Music and sound in AGS

[Pumaman]

Ok, so these days the sound and music system in AGS stands out as being the last main part of AGS that you can tell was written back in the dark ages and hasn't really changed much since then.

You have to access all sounds and music by numbers (though the Media Manager Plugin does help with this), setting the volume correctly is a bit of a mystical art form, and relating sound channels to the sounds playing on them is not easy.

But rather than me just diving in and changing it all, I'd like to hear from you guys who work with it on a daily basis, how you think it should work.

Please don't suggest extra file formats. AGS will continue to support OGG/MP3/WAV/MIDI/MOD/XM/etc as it does at the moment, and I'm not planning to change that.

What I do want to change is the way that the sound and music is controlled from the script. Here are a couple of ideas:

1. Channel-based audio system

You'd do commands like:

channel[1].PlaySound(sExplosion);
channel[1].Stop();
channel[1].SetVolume(60);

2. Sound-based audio system

You'd do commands like:

sExplosion.Play(optional channelID);
sExplosion.Stop();
sExplosion.SetVolume(60);


Those are just two possible options I'm considering. While they eventually lead back to the same thing, which approach is most useful depends on how you like to script and control the audio. Other suggestions on how it should work are welcome.

Should you even have to worry about channels at all? Maybe you would give each sound a priority from 1-100, and AGS would automatically cut off a lower priority sound if all the channels were full?

[Wonkyth]

I only fiddle around with this sort of thing irregularly , so my opinion might not mean much, but the channel based audio system seems the best for what I like to do.

[Ryan Timothy B]

Hmm.. at the moment I don't have any real suggestions, but I really like the ability to assign a name to a number.  Would be very user friendly.

I imagine you're going to be adding a sound manager, from the sounds of it anyway.
The only problem I see from having a sound manager (if it were at least similar to the sprite manager) is that once you edit or switch the sound you'll have to re-import it just to have it in the game.  Could be frustrating for the programmer when the sound guy gives his updated/edited sound files.  Just something I was curious about.

Unless you plan on making it so that you assign which file number the 'sExplosion' will have in the sound manager, and hitting play will just play it from the sound folder, without any importing.


Anyway, I'll definitely think about any audio ideas I may have.

[Matti]

Giving the sound and music files a name is not only an imrovement, it's everything I'd ask for really..

I think I's prefer the sound-based audio commands to the channel ones... but it doesn't really matter to me. I guess I just would like to have the name of the sound at the beginning of the command..

[SSH]

Important to me is the ability to play speech sounds from custom scripting, and not just from the Say command, as its something I come across alot when writing modules. Also, the whole task of creating a voice pack for a game is very onerous, simply in the admin of matching huge numbers of speech files to text in the game. Perhaps another option is some kind of matching system that uses MP3 tags or filenames to match against speech lines semi-automatically. Or even word-based speech files.

[Ubel]

Important to me is the ability to play speech sounds from custom scripting, and not just from the Say command

Yes, I agree. This is an issue that has been bothering me when I've wanted to create my own speech scripts too.

[Goldenrod111]

Like matti, I prefer the sound-based system, but the only improvement I'd ask for would be to be able to name the sounds. Ryan Timothy brings up a good point, though, and while I wouldn't have much problems with it (I have never made (or edited) any sounds--probably why most of my games (all unfinished) are silent), I think that something should be done for the people who do edit their music, like assigning a script name to a file, allowing the file to be changed (as long as the name stays the same) without having to do anything in AGS to accomidate the new version.

[Nickydude]

2. Sound-based audio system

You'd do commands like:

sExplosion.Play(optional channelID);
sExplosion.Stop();
sExplosion.SetVolume(60);

Maybe you would give each sound a priority from 1-100, and AGS would automatically cut off a lower priority sound if all the channels were full?

This is the system I particularly like. Accessing sound by name would definitely help. 

[Vince Twelve]

Ooh, if you're overhauling the music/sound system, I'm indifferent about how the scripting works, but I do want a few features:

1) Ability to get what sound/music is playing on each track and at what volume.
2) At least two music tracks. (allow manual crossfading)
3) Toggle looping for each individual track.
4) No more music skip or stutter when changing rooms or loading large animations/sprites
5) Working music volume controls!
     a)  Track volume (set from 0 to 200)
     b)  Room sound & room music volume modifiers (set from 0 to 200)
     c)  Global sound & Global music volume modifiers (set from 0 to 200)

Actual volume would be a percentage of the regular sound file volume computed by multiplying the three percentages together.  If the percentages multiply to less than 100, the sound or music is quieter, if the percentages multiply to more than 100, the sound or music volume is boosted.  Setting any of them to zero would result in silence for the track/room/whole game respectively.

This way, you can change individual track volume for a sound that's too loud/quiet by adjusting the track volume, you can adjust the room volume if you want one room to just be quieter, or you can adjust the global volumes (like in a settings panel) to affect the entire game. 

Perhaps there should also be something similar to a PlaySoundAtVolume(int sound, int volume); (adjusted to match whatever new scripting you decide on) that would set whichever track the sound would be placed on to the specified volume, play the sound, then return the track it to it's previous volume.  Also, playsound functions should return the track that the sound was placed on.

[Shane 'ProgZmax' Stevens]

I'm glad you're getting around to this, CJ!  Aside from Vince's suggestions, I'd just like to make sure there will be a way to set music volume to actual zero (so it is not audible) and to start it again.  Merging SetVolume and SetMasterVolume into one coherent volume setting that covers the range from 0 to max (as I suggested elsewhere) is probably the thing I'd like to see most.  I do like the idea of sound based since it will automatically assign the sound to any available channel (it's cleaner that way).  Would it be possible to have the sound-based system handle positional sound (sound emitting from a specific location) like you can with ambient sound? 
Just a setting in the sound playback to specify whether it's ambient or not would do the trick, I think?  I'd also like to see the way ambient sounds are handled worked on, since right now it's a bit hit or miss to be honest (you can go to a consecutive room and the audio will fade, but then you jump to another room and the music/sound is loud again).  I think one way to handle this would be to allow the user, at the time the sound is created, to specify a range of rooms and volume settings for the ambient sound.  Or, you could add an emitter option to the room edit and allow the user to place a sound emitter in a room and specify it's values.  I'm sure there are other ways, but I definitely think ambient sound playback needs some tweaking.

[GarageGothic]

Great initiative, CJ. The sound system definitely could use an overhaul, especially the volume settings. A few things I would like to see:

1) Ability to play speech files without using the Say command, as mentioned by SSH.
2) An analogue to Get/SeekMP3PosMillis() that works with any kind of sound file rather than just music. Possibly something like channel[1].GetAudioPosition.
3) And in extension of this, a GetAudioLength function that can either be called on a sound file or on the channel where it's played.

[Pumaman]

Thanks for your feedback so far.

Thinking about this further, do you guys think it's necessary for AGS to make a distinction between Music, Sound and Speech? Should there just be a single thing called Audio?

Conceptually an ambient sound is identical to music -- it's just a looping piece of sound. If, as has been suggested, the speech system was changed so you did something like this:

player.Say("&EXPLAINITALL I've been meaning to explain this for ages.");

which then played EXPLAINITALL.OGG, would we still need the distinction between the three types of audio?


I think the biggest hurdle in any change is likely to be backwards compatibility with existing scripts, but that's for me to worry about I guess.

[freshpaint]

Have been frustrated at times by what seems like an artificial distinction among sounds -- has meant having to duplicate something in sound and music or hacking stuff together in Audacity if I wanted to loop or control volume better.  Would love to be able to queue and layer anything via channels or otherwise.

Would like full property panes for all sounds/music/speech -- ability to put name, description, duration, source, custom property, etc. (I haven't used the addon sound module yet, so this may already be available).  Also, mark a sound/music as "include in compiled file" vs
. "put in separate file".

Channel object vs. sound object -- would love both (somewhat like views and characters -- sounds have channels).

Better control of sounds etc. as events and rooms are exited and entered would be great -- especially duration/fading/adding silence.

That's all.

[Vince Twelve]

player.Say("&EXPLAINITALL I've been meaning to explain this for ages.");

which then played EXPLAINITALL.OGG, would we still need the distinction between the three types of audio?

I think that this way would be extremely cumbersome in a game with thousands of lines of dialogs.  As long as there is still an auto numberer (with the ability to customize it to search for string arguments to other functions) this would be fine.

Also, I would definitely want anything like cCharName.Say("&EXPLAINITALL Blah blah blah."); to play charnameEXPLAINITALL.OGG.  So much easier to keep track of that way.

As for music and sound.  As long as you can set specific tracks to looping/non-looping then there doesn't really need to be a distinction between sound and music.  Ironically, in Linus Bruckman, I did all the music as sound and all the sound as music because I needed the ability to fade the volume all the way out (can't do it properly with music) and play more than one piece of music at a time (also can't do it properly with music).

The only issue I can think of where someone might want to have them separate is for separate music/sound effect volume sliders in a settings GUI.  But you could just have them control certain tracks, and make sure to only play music/sounds on the proper tracks.

[Shane 'ProgZmax' Stevens]

I don't think there needs to be a great distinction aside from what Vince mentions with being able to control music and sound levels independently.  One set of inclusive functions for doing everything you need to with audio would work fine, I think, and save you a heap of time code wise and in the future.  For dialog, I agree it could work a bit better than it does now.  For example, SayBackground doesn't play speech lines, which is something I've never really understood.  This forces you to play the speech as a sound instead.  Allowing people to use any kind of filename for dialog would work fine, I think.  It would be up to the user to make sure their filenames don't have duplicates, and the engine would then be able to read in any 'new' names added without renumbering the whole thing.

[skuttleman]

Allowing people to use any kind of filename for dialog would work fine, I think.  It would be up to the user to make sure their filenames don't have duplicates, and the engine would then be able to read in any 'new' names added without renumbering the whole thing.

What about "Auto-number Speech Lines"? I'm rather fond of that feature. I prefer to write, re-write dialog endlessly until I'm happy with it and at the end, I don't want to go all the way back through it and add a filename to each one.

[Shane 'ProgZmax' Stevens]

The problem with auto-numbering is it really doesn't work well.  You can't 'really' write and re-write lines endlessly without completely renumbering all of the lines as it stands, and if you add some new ones here or there it breaks the auto-numbering completely, forcing you to number the rest of them yourself.  The numbering system either needs a rework so it can intelligently detect new lines and add them or it should be left in the creator's hands by allowing them to make unique filenames.  If the numbering system would continue where it left off I'd have no problem with it, but as of now it does not.

[Vince Twelve]

Yeah, having the option in the wizard to only number un-numbered lines would be great.  I suggested this a few days ago in the 3.1.2 wishlist thread.

[GrogGames]

hi!

We are using a Tool that i've made in AGS. In this tool we can add image files for background, and play MIDI files too, but only if they have the name "music1.mid", or "music2.mid", etc.
I think that a playmusicfile("nameoffile") , compatible with MIDI, would be a great add.

thank you!

[Scavenger]

The suggestions here have been really quite good, and mostly align with what I would like to see, but in case I have some facet left untapped:

- Perhaps some way of organising the numbered speech lines, so we have the number of the speech line and the containing text available for viewing all at once (in a kind of database), that we can attach sound files to in the editor. Example, we have dialogue as such:
&1 I am Ego, Hear me Roar.
&2 Jolly Good, I hope that turns out well for you.
&3 That doesn't work.

Now, if we had thousands of lines, we'd have to make sure we're numbering all the files right, and we would be working pretty much blind. However, if we had something as simple as a menu list/database with:
[SPEECH NUM.][TEXT][PLAY |>] [SOUND FILE NAME]
For each line of the dialogues, we'd be able to quickly assign our audio files, and be able to test it there and then. The resulting data could then be easily changed and reassigned if you need to place new voice files in, without having to blindly stumble around the dialogue.

- (I'm a little out of the loop, but...) Is it possible to adjust the individual channels of a MOD/IT/XM file to, say, reduce the volume of MOD Channel 1 to 0, leaving the other three MOD channels playing normally? This would be a great help in creating dynamic music, but not that important.

- I like the idea of sound emitter objects for ambient sounds. In this update, will ambient sounds be unrestricted in how many channels they take up? (2-4 channels for ambient sounds might be as many as anyone needs, lest it turn into a cacophony.)

- I like the channel based system. It looks more precise and controllable - that way we can control exactly which channels we want to change at any one time.