Music and sound in AGS

[Trent R]

Don't have much to add, mostly cause I haven't used AGS's audio anyways, besides just reading manual entries.

However, I do think that there should just be a Audio namespace (accompanied by an Audio folder in the Game directory), rather than Music, Sound, and Speech. However, I think it would be nice to have subfolders within the Audio folder, so that you can seperate them that way for yourself (optional too).

Also, if you can implement Metadata into a Media manager, that would be the best! Even if just reading it at this point, and worry about writing metadata later.


~Trent

[Vince Twelve]

One more note regarding lumping music/sound/speech together: Speech should probably be kept separate so we don't lose the ability to have a separate speech.vox for optional speech or multiple language packs.

[Dualnames]

If something like this :

channel[1].PlaySpeechFile(cEgo,1);

can be placed, it would be really really cool.

[Trent R]

One more note regarding lumping music/sound/speech together: Speech should probably be kept separate so we don't lose the ability to have a separate speech.vox for optional speech or multiple language packs.

If we had access to metadata tags, then you could specify which ones are speech, which are sound, and which are music. Then when you want the build the .vox file, it could search for files with those tags and only build them.

Actually, even if it's not metadata. If you could specify your own tags in AGS, it'd be good enough.

~Trent

[HammerBlade]

Would it be possible to get technical information on MIDI's used in the game?  I know you can seek the position of a MIDI by beat, but it would be nice to be able to use information such as tempo, current measure, etc.  for other things (such as synchronizing animations to line up with background music).  Think of the potential for action-rhythm games!  (or at least automatic adjustment of animation speed of environmental objects based on tempo: "fast-forwarding"/"Slow-down" effects?)

A list of commands such as

channel[1].AsMidi.getBPM(Beat); //gets the beats per minute of the MIDI at beat 'Beat'.
channel[1].AsMidi.getTimeSignature(Beat); //gets the time signature (such as 4/4) of MIDI at beat
channel[1].AsMidi.getPitch(Beat); //gets the pitch of the beat
channel[1].AsMidi.getVolume(Beat); //gets the volume of the beat

channel[1].AsMidi.get...(etc.)

...Okay, so most minute properties of MIDI's aren't important enough for most games to need such intricate data on a MIDI...but the BPM at least woud be nice to have access to.

[RickJ]

sExplosion.Play(optional channelID);
sExplosion.Stop();
sExplosion.SetVolume(60);

It would seem like this approach would offer more possibilities and would be my preference.   Prersummably there would be a an Explosion.wav or Explosion.mp3 file in the a Sound directory.

Important to me is the ability to play speech sounds from custom scripting, and not just from the Say command, as its something I come across alot when writing modules. Also, the whole task of creating a voice pack for a game is very onerous, simply in the admin of matching huge numbers of speech files to text in the game. Perhaps another option is some kind of matching system that uses MP3 tags or filenames to match against speech lines semi-automatically. Or even word-based speech files.

If the above sound object approach was used then instead of tryiung to match dialog text to voice files the sound object itself could also contain the dialog text.

   sExplosion.Text = "Get down there's going to be an explosion!";
   Display(sExplostion.Text);

Actually MP3 tags and other sound file attributes could also be exposed in this manner as well. 

However, I do think that there should just be a Audio namespace (accompanied by an Audio folder in the Game directory), rather than Music, Sound, and Speech. However, I think it would be nice to have subfolders within the Audio folder, so that you can seperate them that way for yourself (optional too).

I would agree with ther need for subfolders as well.

If something like this :

channel[1].PlaySpeechFile(cEgo,1);

can be placed, it would be really really cool.

I would prefer something like:   

sSound1.Play(eChannel-1);

or

cEgo.Say(sSound1, eChannel-1);

In the latter case if the sound object contained text then the text would be displayed and the sound played.

[Pumaman]

Thanks for your feedback, guys.

I think, in conclusion, it makes sense to go with the following:
* Keep the Speech folder for speech, and keep it numbered. This allows the Auto-number Speech Lines feature to continue working, and to have a separate Speech.vox file. Add a manual PlaySpeech script command.
* Combine Music and Sound into a single Audio folder, and allow each file to be addressed by name instead of number.
A priority and loop parameter when playing the audio would allow you to treat the audio as Music, Sound or Ambient Sound, as appropriate.
* The script would have no concept of Sound Channels. Instead, AGS would use the Priority setting of each Audio clip to destroy an existing lower-priority sound if it ran out of channels.
However, the audio.Play() method would return some sort of Channel/Sample object that you would be able to use later to adjust the volume or stop the audio.

Does anyone see any problems with this approach?

[TheMagician]

Combine Music and Sound into a single Audio folder

I'd like to see what 'Trent R' suggested: the possibility for subfolders in the audio folder. At least one for music and one for sounds.

And I hope - although the concept of channels will be dropped - we will still be able to modify the volume of each playing sound separately.
With channels we were able to do: "whatever is playing on channel 3 - make it louder". This seems more difficult to keep track of with the new "priority" concept.

[Joe]

The approach is right for at least... My suggestion doesn't fit here but it would be nice if sound wasn't compiled in the game exe, and create a separate file such as music.vox, eg: sound.vox

[Trent R]

Combine Music and Sound into a single Audio folder

I'd like to see what 'Trent R' suggested: the possibility for subfolders in the audio folder. At least one for music and one for sounds.

But you can prefix your filenames with something like soundBigExplosion.ogg

Like I said before, I haven't used Audio much at this point. But I think your final decisions sound good.


~Trent

[Shane 'ProgZmax' Stevens]

I don't see any problems with the approach, but I would like you to elaborate on whether these changes will extend to reworking MasterVolume to operate across the entire spectrum (overriding room sound settings) and to addressing the ambient sound playback issue (inconsistent sound levels between room changes, etc).  I understand that these aren't directly related to your changes, but if you are indeed overhauling the sound system I think it's important to fix some of the pre-existing issues with it as well?

[GarageGothic]

Would you somehow be able to retrieve the sounds currently playing, or would it be necessary to keep track of this in a custom play function? One example that springs to mind is a feature in my current game where calling up the pause menu will fade out the room music and play a pause menu tune. Returning to the game will fade the pause music out and resume the previous music at the same point it faded out. With the new system it seems I would have to store the ID for the background music when it originally started rather than just before fading it out?

[GrogGames]

* The script would have no concept of Sound Channels. Instead, AGS would use the Priority setting of each Audio clip to destroy an existing lower-priority sound if it ran out of channels.
However, the audio.Play() method would return some sort of Channel/Sample object that you would be able to use later to adjust the volume or stop the audio.

Then,  to play a sound,  we'll use audio.Play(), but to stop or adjust volume we'll use chanel[ x ].Stop.
Then... we will need a command that returns the channel where an audio file is playing,
or -1 (or 0) if the file isn't exist or is not being played.


----------

Also, it would be useful to reproduce midis from files, during the game, such as it is able to load a bmp or pcx image with DynamicSprites, now that i´m making a tool for the creatives of our group.

[Trent R]

Garage's post made me wonder... If there's going to be a new Audio object, will we be able to have pointers to it? (In his case, storing the currently played music into a pointer, then calling it when the pause menu is removed)


~Trent

[Dualnames]

Thanks for your feedback, guys.

* Add a manual PlaySpeech script command.

Marry me!!!

*Gets the hang of it*
Thanks!!!!

[Pumaman]

I'd like to see what 'Trent R' suggested: the possibility for subfolders in the audio folder. At least one for music and one for sounds.

Sub-folders are certainly a possibility, though they could easily be added later and wouldn't be essential to the initial version.

And I hope - although the concept of channels will be dropped - we will still be able to modify the volume of each playing sound separately.

Yes, of course.

I don't see any problems with the approach, but I would like you to elaborate on whether these changes will extend to reworking MasterVolume to operate across the entire spectrum (overriding room sound settings) and to addressing the ambient sound playback issue (inconsistent sound levels between room changes, etc).

The problem with this is how to maintain backwards compatibility.
In an ideal world I'd rip out all the sound and audio code from AGS and start again, but I can't really make a change which would break everybodys games. Therefore, I need to do some further investigation to see how well I can fix it without breaking existing games.

Would you somehow be able to retrieve the sounds currently playing, or would it be necessary to keep track of this in a custom play function?

Good idea, something like this should really be provided.

Then,  to play a sound,  we'll use audio.Play(), but to stop or adjust volume we'll use chanel[ x ].Stop.
Then... we will need a command that returns the channel where an audio file is playing,
or -1 (or 0) if the file isn't exist or is not being played.

Garage's post made me wonder... If there's going to be a new Audio object, will we be able to have pointers to it?

Ok, let me provide an example to clear this up. What I'm proposing is something like this:

SIMPLE SCRIPT:
aBullet.Play(ePriorityHigh, eLoopNone);

ADVANCED SCRIPT:
SoundChannel* channel = aBullet.Play(ePriorityHigh, eLoopNone);
...
channel.SetVolume(5);
channel.Stop();
etc

Because one sound effect could be playing several times simultaneously there needs to be some sort of separate concept of a Channel in order to control the individual playing instance of the sound.

[RickJ]

I was wondering if there will still be a limit on the number of simultaneously playing sounds?  If so then perhaps there could be a sound channel array that would work similar to the way the character array works.  When a sound is played it would automatically be assigned to an available channel or alternatively played on a specific channel specified by an optional parameter.

[Ryan Timothy B]

SIMPLE SCRIPT:
aBullet.Play(ePriorityHigh, eLoopNone);

ADVANCED SCRIPT:
SoundChannel* channel = aBullet.Play(ePriorityHigh, eLoopNone);

With the Advanced script like this, wouldn't it cause some problems?  Also seems a little confusing for those who might not understand scripting.
The 'SoundChannel* channel' should be part of the PlaySound command (as optional).
I feel the code to play sound should be like this:

aBullet.Play(eLoopType, ePriority, Optional Channel);

To make things easier, would be to make an  ePriorityNone.  That way you don't have to mess with priority if don't wish to (the way PlaySound used to be).  Or with ePriorityNone, it could help the advanced scripters who know what channel each sound plays on, and doesn't care about Priority.
I imagine they could always just use ePriorityLow on all sounds and they would still play over each other?

Although if  aBullet.Playing  would return a Boolean, you could also have more control over the sounds.


Also will there be more channels available with the new sound system?


I'll have to give it some more thought.

[Pumaman]

I was wondering if there will still be a limit on the number of simultaneously playing sounds?  If so then perhaps there could be a sound channel array that would work similar to the way the character array works.  When a sound is played it would automatically be assigned to an available channel or alternatively played on a specific channel specified by an optional parameter.

Yes there would still be a limit, but the question is how the sound channels are rationed. I think we seem to be going round in circles with this discussion now so I don't want to keep going over the same ground, but basically the options are:

1. don't allow channel numbers to be specified in the script, and use a Priority for each sound instead so that AGS manages them automatically
OR
2. allow the scripter to specify a particular channel, like PlaySoundEx at the moment.

My plan is to go with option (1) because it seems simpler and more intuitive to use.

With the Advanced script like this, wouldn't it cause some problems?  Also seems a little confusing for those who might not understand scripting.

Well that's why I've called it the Advanced scripting. Beginners would just use a simple scripting line like the one I posted to just play a sound. But if you wanted more control over the sound (eg. to change its volume etc later on), you could save the pointer that Play() returned in order to do so.

The 'SoundChannel* channel' should be part of the PlaySound command (as optional).
I feel the code to play sound should be like this:

aBullet.Play(eLoopType, ePriority, Optional Channel);

I'm not sure what the advantage of this is. And of course, Priority could be an optional parameter to simplify things if you didn't want to worry about it.

[RickJ]

Yes there would still be a limit, but the question is how the sound channels are rationed. I think we seem to be going round in circles with this discussion now so I don't want to keep going over the same ground, but basically the options are:

Sorry abouyt going around and around, I was just had a a little lack of understanding of the current proposal.  Option #1 appeals to me the most and is, IMHO, most consistent with the OO paradigm.   

If all the sound channels were busy and a new sound was played then how would the priority scheme work?  Would the new sound not be played or queued until a channel was available or would the new sound be played on a channel playing a lower priority sound?   If there were more than one channel playing sounds of the same priority which one would be canceled, the oldest one perhaps?

With this option how would one get a list of the currently playing sounds?