AGS 3.3 chopping voice OGG files?

[m0ds]

Hi guys, ags devs, I have a simple question that is plaguing our latest game.

It would seem that during the game, some of the voice OGG vorbis files will play about 3 seconds worth and then cut out. This happens with multiple lines, generally if they last longer than 4 or 5 seconds.

The fix I found so far is to keep them as WAV files*, but as soon as some of them become OGG files, AGS starts playing half of them but not the second half. Ie a line "This is a very big doorway, I wonder what's behind it" comes out as "This is a very big doorway, I wo..." and continues to display the text of the line.

Is there a known OGG issue? I've passed the game onto testers and it happens to them too. Could it simply be my ogg format? I've tried a couple of different settings. Perhaps audio channel settings within the game or something?

A possible issue is "stereo" ogg files, because when I switched them to mono, AGS seemed to either do it less often or stop (I can't quite remember).

Anyway, I'm using the same ogg files I always have (same hz, sample settings etc) and never had a problem in 3.2. But since moving up to 3.3 this problem has begun.

Any ideas/thoughts please? Many thanks

* Of course, this means large sized vox which isn't ideal for distribution.

[Crimson Wizard]

Have you tried running them as normal audio by Play command? Are they cut the same way then?
Do you use threaded audio in your project?
OGG player was patched by JJS, but I vaguely remember what was that; I need to refresh my memories.

[selmiak]

have you tried another converter when converting them from wav to ogg? VLC and SUPER are the first to come to mind, sadly these xiph guys don't recommend any converter, hopefully there are some more converters out there as ogg is the open source format. I don't know why, but sometimes changing small things works...

[Baron]

I think Ponch told me once that .ogg format messed up sometimes when the file size aligned with perfect increments of storage size, like exact multiples of 24Kb or something.  Try adding a fraction of dead air to the end of an offending track (or winnowing off a fraction) and see if it still messes up.

[Crimson Wizard]

Mods sais this started when they moved to 3.3 ...

BTW, have you actually tried to play these files in 3.2 project?

[arj0n]

@CW:
So could this be related to this post?

Changes since version 3.2.1:
* Fixed OGG speech ending prematurely if the audio file size is a multiple of 32 KB.

[Crimson Wizard]

Changes since version 3.2.1:
* Fixed OGG speech ending prematurely if the audio file size is a multiple of 32 KB.

Yes, that's what JJS did. But it sais "fixed" .
Still waiting any answer from Mods.

BTW, since I am at it, Mods, can you send me one of the OGG files that fail to play?

[m0ds]

Thanks. You've all provided me a few new things to try, I'll let you know!

PS. No file sorry I'm still looking. I thought it was random and not line specific. Will try a couple of things suggested above first

[m0ds]

Ok, so after various testing and hunting the issue down - I'm 99% certain it's the multiple of 32's causing this. All the broken files are multiples of 32 size. It doesn't matter if it's the size or size on disk, if one of those is a multiple of 32 - speech lines cut out. ie 56kb, 160kb, 256kb etc. All the non-multiples are working fine.

If I make it so they're a few kb less or more than the multiples, they work again.

I'm using AGS 3.3.0.

AGS Editor .NET (Build 3.3.0.1162)
v3.3.0, April 2014

We can only conclude that the supposed fix - isn't fixed? Not sure why I'm experiencing this and no-one else has mentioned it, surprised it hasn't affected Golden Wake or other recent talkies etc but perhaps they're using a slightly older version, or some other format than ogg?

FYI - Two people have edited our lines (me and Arjon) and both used different conversion tools, yet both sets of lines have some at multiples of 32 and they cut out, so that rules out the conversion software pretty thoroughly.

Out of curiosity - what could've happened to the OGG decoder to make it do this now and not in previous versions?

[Crimson Wizard]

Hmm, I can take a look at this, although to be honest this will be the first time I see OGG running code.

Can you send me any file that causes this?

[m0ds]

sure: www.screen7.co.uk/junc/POLI92.ogg

[Crimson Wizard]

The sound is cut only when the OGG is used for a speech. When played as normal sound it works fine. There's a technical difference in how these two types are processed internally (speech is being streamed, not preloaded), so this may have explanation.
The speech is cut both in 3.3.0 and 3.2.1. This means that whatever JJS was fixing he did not fix that to the end.

EDIT: hmm, this might not be related to what JJS was fixing. It appears the OGG playing library is returning error on that last chunk of sound... Some further investigation needed.

EDIT2: I sort of misread information about multiple of 32. JJS was fixing a problem with OGGs which size was multiple of 32 Kilobytes (32768 bytes). There was nothing made regarding multiple of 32 bytes.
I am not sure what's going on there yet. I can only tell that on some point, when there's 320 bytes left in the OGG stream, the OGG player returns error. Unfortunately, AGS does not checks errors in this case and the speech "hangs" for unlimited time (not really playing anything), until user clicks button to skip it. At least that's what is happening to me.

EDIT3: Interesting fact. I increased the streaming buffer, making it equal to the file size under examination, and it played flawlessly to the end.
The error OGG were returning is "buffer underrun" which means that it can't find all data it wants. It looks like there may be a byte sequence that OGG player treats as incomplete and "stumbles" over it.

[m0ds]

Thanks. Yes, they hang infinitely for us too.

I kind of understand what you're saying. Can we expect a fix anytime, or should I go ahead and use the workaround? Is there any way we can increase the 'streaming buffer' our end?

It wouldn't surprise me if we're using longer lines of text and speech than various other games with speech at the moment hence why we've suffered it but no-one else has. It's typically only the longer lines that are affected.

[Crimson Wizard]

I do not know how to fix this properly yet. I never worked with OGG streaming. This probably should be simple, but library documentation is somewhat lacking.
So far I managed to force it play further in case of error and read remaining piece of file. It almost worked, but there is a noticeable jump a fraction of second in length.

And, no, there's no way to change the size of buffer game properties, or script atm.

It wouldn't surprise me if we're using longer lines of text and speech than various other games with speech at the moment hence why we've suffered it but no-one else has. It's typically only the longer lines that are affected.

Hmm, maybe it is length that matters, rather than size multiplier?

[m0ds]

I can't imagine so, there are far longer lines than the broken ones which work just fine. It just seems to happen to the lines that are longer than a single sentence, and like I say, they're all multiples of 32 in size whether that's a cause or just co-incidence.

[m0ds]

Edit. Ok, yes, you're right. It seems to happen in 3.2 also.

[Crimson Wizard]

Are you sure it is not happening to previous versions?
As I wrote earlier, I have tested the OGG you sent me in 3.2.1 and got same results.
This is important to clarify.

If you can confirm that 3.2.1 works well with your OGGs, I could make a set of builds from various stages of development and see when it started to fail.

E: Just tested with 3.2.0, it cuts off too.
E2: Tested with 2.72 and it cuts off too...

[m0ds]

Sure, you're right. It occurs with other versions. Back to the drawing board...

One of the most perplexing mysteries ever for me...!! Sorry I grumbled about 3.3 but you're right, there must be something else to it.

Something about my PC... because OGG files encoded by me on this PC and encoded by Arjon in NL have had this problem, so something about it compiling on my crap old PC perhaps...? hm But then it can't be that if you're able to reproduce the error in a project file for yourself. So weird

[Crimson Wizard]

I wish I knew! Last time I was investigating this I left with feeling that there's a bug in ogg library AGS is using.
It has something to do with OGG stream fractioning, because when I made the size of buffer equal to the size of file, this error did not occur.
I am interested to see how the Linux port works because it should be using more recent versions of the libraries.
In the very last case we could add a hack, some option in config perhaps, that would force OGG player to always load whole speech file in the large buffer...

[SpeechCenter]

The problem is that ov_read returns 0 even though it's not the end of file.
Indeed this is likely a bug in libvorbis. Check out https://bugzilla.redhat.com/show_bug.cgi?id=505610 reporting similar behavior. So if the Linux library uses libvorbis-1.2.2 or above it's likely to not reproduce there. Alternatively, update the library in Windows and see if it works.

[Crimson Wizard]

I built the engine with libvorbis-1.3.4, but have same results.

Meanwhile I learned that the file was encoded with "Lafv" (which stands for "libavformat" coding library). Is it possible that this codec writes the file in a slightly non-standart format?
I noticed that while Media Player Classic and Windows Player play this file well, the VLC player, for instance, don't (although it plays other OGGs I have) - the sound stutters, and few file parts play in different order (!).

It seems I would need to dedicate some free time for this... Maybe ask around on the web.

[SpeechCenter]

Perhaps open a bug for at xiph.org for libvorbis (https://trac.xiph.org/). They'll probably be able to tell if this file is encoded incorrectly or if it's a software bug.
By the way, the file does play correctly on my VLC, but there's a problem to have it play from a different location than the beginning.

Regardless, it's a good idea to update the library, the current one is very old. It would also help building all dependencies from source, if one would like to do that.

[Crimson Wizard]

Mods, do we have a right to use your speech OGG as an example in public? I mean, it contains some in-game lines which may be considered spoiler. Maybe you have something more generic between the files that fail to play?

[m0ds]

It's ok CW to use that one. If someone takes the entire plot for the game from that one line, Kudos to them

Also, it's interesting to see both your points. Unfortunately I don't seem to get much choice in what kind of OGG to convert it to (ie choose any different kind of encoder), other than a basic "OGG Vorbis" choice. But I can understand why "Lafv" might be a cause of the issue.

[Crimson Wizard]

I am back to this again, and my situation changed. I wrote a test based on ogg vorbis example, which simply decodes the ogg file and writes progress into a text log. I found out that my test program parses the file just fine to the very end.
I write parameters I send to ov_read, and the resulting values.
I made AGS write similar log, and then compared them. They appeared to be identical, until the moment when ov_read failed.

Then I noticed that the ALOGG (an Allegro-based library which works with OGG for AGS) has its own callbacks for reading, seeking etc the data.
So now I am questioning if it is ALOGG that has a bug inside.
What I found so far is that AGS uses version 1.0.3 of ALOGG, which is as old as august 2002 .

The latest version I could find in the web is ALOGG 1.3.7 of 12 december 2005. The problem is that I can't use it out of the box, because its program interface has changed absolutely.

Searching through the change-log, I found these entries:

Fix stream stopping and some occasional pop sounds
Fix the encoder not setting the end of stream properly
alogg_stream now avoids cutting the stream too early when adding echo

etc, so who knows, may be one of them bug fixes is related.

Problem with ALOGG is also that it's very old, and many links that supposedly reference its download pages are not working anymore. I found only versions 1.3.4 and 1.3.7, and both are way too different from the one AGS is using to make a quick update.


EDIT: I will make some more tests to see if I can at least find a difference between standard vorbis stream reading and ALOGG's one (thankfully both sources are open).

//-------------------------------------------------------------
UPD: Okay, I practically found the case when ALOGG's callback finds out that the provided source buffer is emptied and returns EOF value to OGG Vorbis library. So ALOGG is the culprit after all. Question is, why didn't it asked for another piece of data in time...

[Crimson Wizard]

Ehhh... alright, I take all my words about ALOGG and OGG vorbis back.
It looks like I did not properly understood how this thing works from the beginning.

Basically, it is a problem in AGS that it does not handle the "buffer underrun" case.
In short, it is the situation when OGG vorbis needs N bytes of data to make correct parsing, but buffer has less. So it returns error code, which signals that program must add more data to already existing buffer. But AGS does not care about such things...

Probably I am on the right tracks now.

[Crimson Wizard]

Mods, I have this patched acwin.exe here:
http://www.mediafire.com/download/d2jbi88jdufibav/acwin_3.2.2_ogg_fix.zip
It is based on AGS 3.3.2 (though I believe you can use it in pair with 3.3.0 editor).

It is not fixed to the end, there's a noticeable "jump", maybe a fraction of second in length. I think a very small piece of sound gets lost when the error occurs. But at least it plays to the end and does not "hangs up" the speech forever. You may use this at least for a test, if you like.

I would need more time to investigate how the library works and if it is possible to fix this. Unfortunately, at the first glance, the sound lib AGS is using is not meant for handling situations like this itself, so may be I'll have to hack it too.

[Crimson Wizard]

Mods, here's another try, now it seems fine, but I can't be 100% sure. The library code is very old and frankly I do not want to dive too deep into it. Maybe if you could test this with more varied examples, we'll know better.
http://www.mediafire.com/download/d2jbi88jdufibav/acwin_3.2.2_ogg_fix.zip

If you say this works I'll add the fixes to upcoming AGS 3.3.3.

[Dualnames]

The same issue has been reported with primordia as well. It has been around forever, i can find you posts where i reported it and everyone just nodded and did nothing.

[Crimson Wizard]

Alright. Current situation is following: my fix is not perfect, it makes a tiny piece of sound get lost (not played) which results in a quick "scratch"-like effect (something similar to what happens when the phonograph needle jumps). The speech continues to play as normal after.

I spent some times trying to figure out a better fix, but my conclusion is that the library AGS uses as an intermediate between itself and OGG Vorbis is written incorrectly and does not provide any means to solve situations like this. And I have no interest in hacking it further, because I have little clue about how much I'll have to rewrite.

My intention is to use this fix for now (in AGS 3.3.3) to at least let the speech play to the end without freezing; for the future we might look into using a newer version of this library, or another library.