The Custom Speech API proposal

Started by Crimson Wizard, Wed 17/12/2014 11:53:40

Previous topic - Next topic

Crimson Wizard

The main problem of custom speech was that it cannot be binded with standard Say commands in game script and speech lines in Dialog script. You cannot make the engine call your custom functions, you have to do that yourself by manualy placing their calls.
Not only that takes more time, but also you cannot take advantage of otherwise automatic features, such as character animation, voice playback and speech skipping: you have to code these yourself.

The most logical (in my opinion) solution would be to add support for speech callbacks, which work in a similar way as custom dialog options rendering: you give your functions predefined names and engine calls them instead of built-in speech processing.

There was already some discussion about this a while ago: http://www.adventuregamestudio.co.uk/forums/index.php?topic=48518.0
Since then I changed my mind a bit though.

My proposal is to support at least two callbacks. We may later extend them, as well as the contents of the callback parameter.

Code: ags

function speech_start(SpeechRenderingInfo *info);

This is called before displaying a speech. User should set up some of the SpeechRenderingInfo properties to initialize the speech overlay

Code: ags

function speech_render(SpeechRenderingInfo *info);

This is called when the speech requires a repaint (default - only once). Here the user should get a drawing surface and do paint operations.

The contents of SpeechRenderingInfo might be:
Code: ags

struct SpeechRenderingInfo
{
  //
  // the Overlay settings
  //
  attribute int X; // the X co-ordinate of the top-left corner of the dialog options
  attribute int Y; // the Y co-ordinate of the top-left corner of the dialog options
  attribute int Width;  // the width of the dialog options
  attribute int Height; // the height of the dialog options
  attribute bool HasAlphaChannel; // should the drawing surface have alpha channel
  readonly attribute DrawingSurface* Surface; // the surface that the speech is rendered to
  
  //
  // Speech data
  //
  readonly attribute Character *SpeakingChar; // the character who is talking
  readonly attribute String     Text; // the text to display
  readonly attribute int        VoiceID; // the id of the voice file (taken from "&CHARXXXX" part of the speech text)
};



I would like to propose game developers to consider if they will be able to implement their actual custom speech using this system. Are there any other callbacks and/or speech data that would be required?

Snarky

Nice work!

My only experience with custom speech has been for animated text (primarily to progressively type out the text typewriter-style, but I've also experimented with things like flashing, shaking or changing text). If I understand this API correctly it wouldn't be able to support that, but it might very well be that it falls outside of what it's meant for, and that my requirements not suited to a built-in API.

Crimson Wizard

Quote from: Snarky on Wed 17/12/2014 12:34:30
Nice work!
Well, thank you, but it's just the design paper. I have no change to code yet, and frankly it may take some time, because the speech code is pretty messy in AGS.
So first I would like to know what we are doing.

Quote from: Snarky on Wed 17/12/2014 12:34:30
My only experience with custom speech has been for animated text (primarily to progressively type out the text typewriter-style, but I've also experimented with things like flashing, shaking or changing text). If I understand this API correctly it wouldn't be able to support that, but it might very well be that it falls outside of what it's meant for, and that my requirements not suited to a built-in API.
Aha, right, the typewriter effect.
Yes, we might need an extra "repexec" callback for this.

Calin Leafshade

Bravo!

Custom speech is something I have hacked in a few times using an empty font but having some interface for that would be great

Adeel

Quote from: Crimson Wizard on Wed 17/12/2014 11:53:40
Code: ags
  // Speech data
  //
  readonly attribute Character *SpeakingChar; // the character who is talking
  readonly attribute String     Text; // the text to display
  readonly attribute int        VoiceID; // the id of the voice file (taken from "&CHARXXXX" part of the speech text)
};

Can you please another attribute for the portraits too? The default 'alternative' style doesn't work all the time and often produces sleazy results. AprilSkies can testify to this. She He even told me that he uses a custom speech function just to display alternative portraits correctly.

Scavenger

While we're talking about rep_exec callbacks, there's no chance that we could get the capability for non-blocking speech in with it, is there? It's the one thing that would make dialogues really shine, since right now we can't really control the pace of the conversation without hacking in fake lines of speech, nor have overlapping speech. The most hacked speech thing for me is definitely background speech, and interrupting. Unifying all the speech types (do we really need a SayBackground anymore?) would reduce the amount of hacking needed to be done with speech. No more speech lines being stored as sound effects, either!

Crimson Wizard

Quote from: Adeel S. Ahmed on Wed 17/12/2014 13:49:07
Quote from: Crimson Wizard on Wed 17/12/2014 11:53:40
Code: ags
  // Speech data
  //
  readonly attribute Character *SpeakingChar; // the character who is talking
  readonly attribute String     Text; // the text to display
  readonly attribute int        VoiceID; // the id of the voice file (taken from "&CHARXXXX" part of the speech text)
};

Can you please another attribute for the portraits too? The default 'alternative' style doesn't work all the time and often produces sleazy results. AprilSkies can testify to this. She He even told me that he uses a custom speech function just to display alternative portraits correctly.

Gurok made some fix for default alternative portraits I think.

I am not sure which attribute do you have in mind?

Calin Leafshade

Instead of callbacks, what about providing an interface instead?

We have managed objects that can inherit other objects, couldnt you just pass an instance of one of those to a function which inherited a base interface struct?

The advantage of that approach is that it's easily modularized and interchangeable. It's also more OOP.

Crimson Wizard

#8
Quote from: Calin Leafshade on Wed 17/12/2014 14:36:27
Instead of callbacks, what about providing an interface instead?

We have managed objects that can inherit other objects, couldnt you just pass an instance of one of those to a function which inherited a base interface struct?

Because AGS Script has no RTTI, nor virtual table, nor function pointers I cannot make engine call member function of any random object.
Simply put, the AGS does not have late binding, only static binding.

The only way that exists is to call a function of predefined name, because we have a map of function names.

Adeel

Quote from: Crimson Wizard on Wed 17/12/2014 14:00:26
Gurok made some fix for default alternative portraits I think.
In that case, I missed that fix. I'll make sure to ask him about this.

Quote from: Crimson Wizard on Wed 17/12/2014 14:00:26
I am not sure which attribute do you have in mind?
The attribute or property of being able to call the portraits either left or right at any given time, depending on your need.

Crimson Wizard

Quote from: Adeel S. Ahmed on Wed 17/12/2014 15:02:25
Quote from: Crimson Wizard on Wed 17/12/2014 14:00:26
I am not sure which attribute do you have in mind?
The attribute or property of being able to call the portraits either left or right at any given time, depending on your need.
No, you don't seem to understand this idea.
It is you who are supposed to draw portraits in this function. And you may do so the way you like.

Adeel

Quote from: Crimson Wizard on Wed 17/12/2014 15:11:59
Quote from: Adeel S. Ahmed on Wed 17/12/2014 15:02:25
Quote from: Crimson Wizard on Wed 17/12/2014 14:00:26
I am not sure which attribute do you have in mind?
The attribute or property of being able to call the portraits either left or right at any given time, depending on your need.
No, you don't seem to understand this idea.
It is you who are supposed to draw portraits in this function. And you may do so the way you like.
My bad, sorry for that...

Scavenger

Oh, I know what attribute is missing! The Pamela lip sync data (like, the current phoneme et al)! That's not opened to the script, but if we're going to do lip sync with our custom speech functions, we'll need it!

SMF spam blocked by CleanTalk