MODULE: TotalLipSync v0.5

[eri0o]

@Snarky uhm, does getting the time of the voice clip in milliseconds would be better? Because I think you can get it from it's audio channel.

I saw this in the manual,

One channel is always reserved for the speech voice-over and you cannot change that, so you have 15 at your disposal.

I wonder if this means the voiceclip AudioChannel is fixed. The other approach is to ask the player to pass this audio channel and let them figure it out.

[Snarky]

Because I think you can get it from it's audio channel.

Yes, that could work (voice clips always play on channel 0), though I've never quite trusted AudioChannel.PositionMs (a long time ago I had some trouble when it didn't seem to work properly).

I do also want to support syncing without an audio file, for more general-purpose synced/scripted animations, but the frame counter should be fine for that purpose.

[Dave Gilbert]

So I'm wondering if the problem lies with the rhubarb end, or if me changing the code like this messed things up. For example, what was the reason that frame 7 was originally unassigned? Was it essential to keep that?

Hmm, changing the mapping should be fine (but see below), though I would strongly recommend that instead of changing the AutoMapPhonemes implementation in the module code, you set up a manual mapping outside of the module when you initialize it—so, instead of calling AutoMapPhonemes, you use those exact AddPhonemeMapping calls yourself to set it up the way you want it.

Ok, that I can do. Thanks! Just to be sure, I did this on game_start():

Code: ags

  TotalLipSync.Init(eLipSyncRhubarb);    // Or whatever lip sync format you're using
  //TotalLipSync.AutoMapPhonemes(); <--- commented out!
  TotalLipSync.AddPhonemeMapping("X",8);
  TotalLipSync.AddPhonemeMapping("A",0);  // mbp
  TotalLipSync.AddPhonemeMapping("B",1);  // other consonants
  TotalLipSync.AddPhonemeMapping("C",2);  // EH/AH/EY etc. (bed, hut, bait)
  TotalLipSync.AddPhonemeMapping("D",3);  // AA/AE/AY (father, bat, like)
  TotalLipSync.AddPhonemeMapping("E",4);  // AO/OW (thaw, slow)
  TotalLipSync.AddPhonemeMapping("F",5);  // UW/OY/UH/OW (you, toy, poor)
  TotalLipSync.AddPhonemeMapping("G",6);  // F/V (fine, very)
  TotalLipSync.AddPhonemeMapping("H",7);  // L (letter)	

The reason why the auto-setup is arranged the way it is (skipping frame 7) is because the other lipsync data formats, Moho and Pamela/Annosoft, distinguish between "ooh" sounds and "w" sounds, but Rhubarb uses the same phoneme/mouth shape for both. The auto-setup maps the frames the same way for all the formats, allowing you to set up the animation view once, and then use whichever lipsync data format you like. You can even switch formats along the way, if for example you want to hand-sync some scenes in Pamela format and do others automatically in Rhubarb (though you'd need to reset the module with the new format and mappings whenever you switch). I don't remember precisely why it's frame 7 that is the "optional" one rather than the last frame, but I would guess I based the order on some standard or convention for lipsync setups.

Okay. So since I never use anything aside from Rhubarb, I don't need to worry about that? I can use my version of the code?

Anyway, to get to the point of your question: looking over the code, it does assume that Frame 0 is the non-speaking frame, as in the AGS convention for NormalView and SpeechView; this frame is set when the animation file does not specify a frame. (This is necessary because some of the data formats can have gaps, but I don't remember if this applies to Rhubarb.) You could try to change it here:

Aha! Yes I have noticed that when the characters stop speaking they always end up on the wrong mouth shape. I added some code to manually change the frame when they stop speaking, but your solution is better! Thanks!

edit: Hmm. I notice that the mouth shape still finishes on the wrong frame. That's on me (because the way the mouth shape frames are imported) so if there's no way to fix that, I'll just continue using the workaround code I created.