Lip Sync Speed

[Ewen]

Hello! I was wondering if anyone can help me.

I'm trying out lip sync for voice for the first time using test-based lip syncing. I've uploaded all my mouth movement frames (for each letter sound) and it works really well.

However, when the text on screen is a bit longer, the animation speed from frame to frame increases. Is there any way to keep text at a universal speed?

I'm not sure if I've explained this well enough but thank you

[Eon_Star]

Hi,

Did you check your animation frame speed options?
You can delay your animations by editing the frame delay options. Check the frame properties window to do so.
I will take a deeper look on this topic.

Until then. Be safe.

[Snarky]

I'm trying out lip sync for voice for the first time using test-based lip syncing. I've uploaded all my mouth movement frames (for each letter sound) and it works really well.

I assume you mean text-based lip sync? Are you actually playing a voice sound clip at all?

However, when the text on screen is a bit longer, the animation speed from frame to frame increases. Is there any way to keep text at a universal speed?

This shouldn't be happening in the first place, I don't think. Text display time should scale linearly with text length, and the animation speed remain roughly constant. There was a bug around this in earlier engine versions, but it is listed as being fixed in v3.6.1 (Patch 6):

- Fixed text-based lipsync's frame delay calculation: the existing formula was causing the delay value to "jump" back and forth with the slightest difference of speech text length, resulting in the speech lines of close length to be played with the notably different animation speeds.

What AGS version are you using?

[Ewen]

Thanks so much both for getting back to me!

Did you check your animation frame speed options?

Yes. I'm pretty good with animations and frame speeds and everything seems fine on that end, as far as I can tell.

I assume you mean text-based lip sync? Are you actually playing a voice sound clip at all?

Yep. Sorry. I did indeed mean text-based. And I am playing voice clips which are working as normal.

What AGS version are you using?

I'm using V3.6.1

I'll try and provide some more detail, if it helps:
I'm using Lucasarts style text
I'm using the SayAt option to place the text in a consistent location.
I'm using Speech.SkipStyle = eSkipKeyMouse;

Everything else in the game is working a treat, its only the sync (something I've never used before) which is causing issues.

The only work around I've found so far (which is a really long one) is to use game.lipsync_speed after every line to manually line it up which is not ideal!

Any further help would be much appreciated.

Thanks again for your help so far.

[Snarky]

Yep. Sorry. I did indeed mean text-based. And I am playing voice clips which are working as normal.

Hmm, when you have voice clips, the length the line is shown is governed by the length of the clip. I'm not sure how the text-based lip sync animation speed is affected: it may mean that the speed is adjusted to match the length of the clip.

So I would test it without the voice clips to see if the inconsistent animation speed is due to that.

I'm using V3.6.1

Since the bug fix was made on a 3.6.1 patch, we need to know the fourth version number part as well. (Or just if you downloaded it quite recently. Patch 6 was released in September 2024.)

The only work around I've found so far (which is a really long one) is to use game.lipsync_speed after every line to manually line it up which is not ideal!

Line up with what? Do you mean that you are trying to get it to line up with the voice sound clip?

If so, that is never going to work reliably. Text-based lip sync is just based on the letters in the text, with particular letters or letter combinations linked to a particular frame, and proceeding through the text at an even speed. It cannot take into account the ways spelling differs from the sounds actually made (especially in English), how some sounds are longer in duration than others, or the way someone might vary their speed within a sentence.

Text-based lip sync is really meant for games without voice audio, where it will make the animation look pretty convincing.

[Ewen]

So I would test it without the voice clips to see if the inconsistent animation speed is due to that.

I removed the audio and the problem continues unfortunately. The mouth animation is still dependent on the length of the displayed text.

Since the bug fix was made on a 3.6.1 patch, we need to know the fourth version number part as well. (Or just if you downloaded it quite recently. Patch 6 was released in September 2024.)

I just downloaded the most up-to-date version after reading your post and the problem is still there.

Line up with what? Do you mean that you are trying to get it to line up with the voice sound clip?

Yes - that's what I was aiming for, only as an experiment to fix the problem but it is not a solution I will use.

I understand what you are saying about text-based sync not really meant for games with voice and that's totally fine. If the mouth speed was consistent, then I'd be happy with the voice not 100% lining up. I just hope to resolve the problem with the mouth speed zooming off as it is quite bizarre to watch!

Thank you

[Snarky]

In that case it sounds like an engine bug that still needs to be fixed.

[Ewen]

In that case it sounds like an engine bug that still needs to be fixed.

Ah! Okay. Thanks very much for your help anyway. I'm at least glad that its not an issue on my end.

Is there a way I can report the bug? Sorry - I'm fairly new to the forum.

Thanks

[eri0o]

If you are trying to lipsync with voice, why not do that? Also I would suggest that with the TotalLipSync module. You need to prepare the lipsync files beforehand for the module if you go that way.

[Ewen]

If you are trying to lipsync with voice, why not do that? Also I would suggest that with the TotalLipSync module. You need to prepare the lipsync files beforehand for the module if you go that way.

Oh cool. I'll certainly look into that. Is it compatible with Lucasarts speech style?

[Crimson Wizard]

Text display time should scale linearly with text length, and the animation speed remain roughly constant. There was a bug around this in earlier engine versions, but it is listed as being fixed in v3.6.1 (Patch 6)

The lipsync animation speed was always slower with shorter texts and faster with longer text, that's a historical behavior, and I did not change it. My fix was regarding the unproportional speed jumps occuring between texts of near lengths, this was happening because of the mistakes introduced by integer divisions. But I have left the original overall behavior untouched.

The formula used to calculate lipsync frame delay in the engine is this:

FrameTime = (((TextLength / LipsyncSpeed) + 1) * FPS) / TextLength

Notice this "+1" constant added after "(TextLength / LipsyncSpeed)". It's essentially forcing a minimal frame time for short texts, making lipsync animated slower. When I tried removing it, lipsync animation started to play lightning fast for the short texts. I could not guess how else to keep the old behavior, so kept this constant there. (Only adjusted its value little, since it's now working in floating math, so it's now 0.75 instead)
With the increase of the text length, this constant's effect gradually diminishes.

[Snarky]

Right, I saw that formula somewhere, but I assumed that the + 1 was negligible for anything but very short strings. But on closer inspection I see that unless TextLength is significantly bigger than LipsyncSpeed (15 by default), it will actually make a significant difference. With a value of 1 I'd guess the length needs to be around 40 before the slowdown stops being noticeable, and maybe around 30 with a value of 0.75.

BTW, is TextLength simply the character length, or does it take into consideration the mapping to sound tokens, so that if you have OH mapped to one frame, "Oh!" will have a length of 1? Since that is the number of frames that will actually be played, it would be the more correct value.

I would suggest a behavior where the "minimum lipsync animation duration" is maybe 0.5 seconds, and once the duration reaches 1 second (double the minimum) the animation speed (frame length) is constant (simply FPS/LipsyncSpeed). For durations between 0.5 and 1, the frame length is interpolated. (Currently this would apply to TextLength 1-15, but maybe the default value of LipsyncSpeed is too high in the first place and should be reduced to 10?)

[Ewen]

With the increase of the text length, this constant's effect gradually diminishes.

Thanks so much for this explanation. This makes sense now and I understand the issue. Really interesting. I'll definitely check the module out and try that method going forward.

Thanks for all the help, everyone. I appreciate it.