Quick Start - Shape Keys

Lip Sync with Shape Keys allows you to give more life to your Character, without having to configure complex rig mechanism.

⚡ Quick Start: Shape Keys

This guide will help you set up your first lip sync animation using Blender's Shape Keys.

Shape Keys mode is ideal for 3D models with no facial Rigs.

🎨 What You Need

  • A character with at least one visible face

  • Shape Keys representing phonemes

  • An audio file (e.g., .wav, .mp3, .mp4, etc.)

You need to define your own Shape Keys.

Sadly, this is a thing Lip Sync cannot automate.

🪄 Step-by-Step Guide

Configure Language

  • In the Lip Sync panel, select your Language Model

Lip Sync works with ~25 languages

Lip Sync will download your Language Model and store it in cache so you won't have to download it again.

Models are downloaded from https://alphacephei.com/vosk/models. Only small sized models are downloaded.

Add Lip Sync to your Model

  • Click on Add Lip Sync

Set Up Animation type

Character made by Blender Studio <3

  • Ensure that Animation Type is set to Shape Keys

Viseme Settings

This is the most tedious part. You have to manually assign Shape Keys to Visemes.

Use Drop-down lists to assign your Shape Key to corresponding Viseme

You can use the same Shape Key for different Visemes. It can be useful if you only have a few Shape Keys for your character.

sil viseme is the position for "Silence".

UNK viseme is the position for unrecognized phonemes.

Animation Settings

Animation settings enable you to fine tune how Keyframes are inserting.

Fine tune your animation
Settings
Description
Infos

Lip Close Duration

Time required to close your character mouth. It will be used when a silence is detected and insert sil keyframe accordingly

In between

Minimum Threshold between two keyframes.

If a keyframe should be inserted before, it will be ignored.

e.g: 417ms means that next keyframe will be at least at 417ms from the previous one.

Decreasing this value will ensure you to animate all phonemes but may result in jerky motion.

Increasing this value will give a smoother motion but sync will be less precise.

Silence

Time to have between 2 words to be considered as a "Silence".

Too short value will force you character to close their mouth often. Too high value will force your character to have longer mouth transition between two words.

Timing is very important and need to respect this constraint:

  • Silence should always be greater than Lip Close Duration and In between

Add Audio

  • Add a Video Sequencer Editor

  • Drag & Drop (or Add) your media file

You should only have one enabled media File. Otherwise speech recognition may product unexpected results.

Baking

Press Bake Audio and wait!

Lip Sync inserted 1372 Keyframes in a few seconds!

Baking will remove previously inserted keyframes on defined Shape Keys.

For Shape Keys, Keyframes use a Linear Interpolation

Lip Sync works with all Frame Rates (24 fps, 30 fps, 60 fps, etc..)

Now your character is talking! 🎬

PreviousQuick Start - Pose AssetsNextQuick Start - Sprite Sheet

Last updated