Quick Start - Pose Assets
Lip Sync with Pose Assets gives you more controls by providing you tools to integrate in a Professional workflow.
Last updated
Lip Sync with Pose Assets gives you more controls by providing you tools to integrate in a Professional workflow.
Last updated
This guide will help you set up your first lip sync animation using Blender's Pose Assets.
Pose Assets mode is ideal for 3D models with facial Rigs.
A Rig with facial controllers
Pose Assets (in Asset Browser)
An audio file (e.g., .wav, .mp3, .mp4, etc.)
In the Lip Sync panel, select your Language Model
Select your Armature
Click on Add Lip Sync
Ensure that Animation Type is set to Pose Assets
Pose Assets mode is only available if you select an Armature
Basic Rig - Use when animating only standard bone transforms: Location, Scale, and Rotation (Euler/Quaternion).
Advanced Rig - Use when your poses require animating beyond standard bones, such as direct BBone manipulation or custom properties.
Choose Basic Rig if:
You're only animating standard bone transforms
You're using BBones but controlling them through standard bone controllers
Choose Advanced Rig if:
You're directly animating BBone properties (without controller bones)
You're animating custom properties
Rule of thumb: Try Basic Rig. If it doesn't work or gives weird results; try Advanced Rig.
90% of the time, Basic Rig is what you need.
Advanced Rig may handle more rigs BUT it can be way slower than Basic Rig.
This is the most tedious part. You have to manually assign your Poses to Visemes.
If your Pose Assets are not showing, it's probably because you're using an external Poses Library.
You have to load them in your blend file. You can do it automatically with the Refresh Pose Assets button
You can use the same Poses for different Visemes. It can be useful if you only have a few Poses for your character.
Animation settings enable you to fine tune how Keyframes are inserting.
Lip Close Duration
Time required to close your character mouth. It will be used when a silence is detected and insert sil keyframe accordingly
In between
Minimum Threshold between two keyframes.
If a keyframe should be inserted before, it will be ignored.
e.g: 417ms means that next keyframe will be at least at 417ms from the previous one.
Decreasing this value will ensure you to animate all phonemes but may result in jerky motion.
Increasing this value will give a smoother motion but sync will be less precise.
Silence
Time to have between 2 words to be considered as a "Silence".
Too short value will force you character to close their mouth often. Too high value will force your character to have longer mouth transition between two words.
Prioritize Accuracy
Animate all important visemes regardless of timing constraints.
Prevents critical mouth shapes (lip contact sounds like P, B, M, F, TH) from being skipped when they occur in rapid succession.
This feature is only available for Pose Assets Mode.
Use this if your character speaks at high rate.
While this increases accuracy, it may require manual adjustments afterwards.
Timing is very important and need to respect this constraint:
Silence should always be greater than Lip Close Duration and In between
Add a Video Sequencer Editor
Drag & Drop (or Add) your media file
Press Bake Audio and wait!
Lip Sync works with all Frame Rates (24 fps, 30 fps, 60 fps, etc..)
Important: Lip Sync creates a new Action with a new Slot.
This means you need to blend the lip animation with your existing Actions and Slots, otherwise you'll only see the lip animation and lose your original character animation.
You can easily do this in the Nonlinear Animation Editor (NLA):
Add a Nonlinear Animation editor
Select your Rig
Select your character's animation (e.g an idle animation)
In NLA, Clic on "Push Down Action"
Now select the Track named "No Action" on the left, and select Lip Sync's Action and Slot on the right sidebar
Now your animation is complete! You can press Play to see it.
