BUG: Dynamic arrays are borked!!1

[monkey0506]

For some reason in developing the EasyBake module I've found that if I create a large static array (specificially of 3086 members) of the type int and pass in static values to each index (accessing each one individually and statically, not via a loop), the game will hang. The thing is, it doesn't hang during the array allocation, or even during setting each of the indices to its respective value. The game will repeatedly hang after the array has been allocated, values assigned, and the array released.

The code looks like this:

Code: ags

void Bytes()
{
  int bytes[] = new int[3086];
  bytes[0] = VAL; bytes[1] = VAL; bytes[2] = VAL; bytes[3] = VAL; // VAL is different for every index, this is just an example
  // ...
  bytes[3082] = VAL; bytes[3083] = VAL; bytes[3084] = VAL; bytes[3085] = VAL;
  // use the array now that the values are all seeded in
  bytes = null;
}

function game_start()
{
  Bytes();
}

game_start finishes executing. The game hangs during room 1's fade-in process. I can upload the specific source that I'm experiencing this problem with if it's relevant.

So, I've actually inadvertently discovered that I can work around this. The way I'm doing that is by exploiting what would probably be considered another bug. If I allocate a char array 4 times the size of the int array I need (int is 4 bytes compared to char being 1 byte), then the game runs perfectly. So basically:

Code: ags

void Bytes()
{
  int bytes[] = new char[3086 * 4];
  bytes[0] = VAL; bytes[1] = VAL; bytes[2] = VAL; bytes[3] = VAL; // VAL is different for every index, this is just an example
  // ...
  bytes[3082] = VAL; bytes[3083] = VAL; bytes[3084] = VAL; bytes[3085] = VAL;
  // use the array now that the values are all seeded in
  bytes = null;
}

function game_start()
{
  Bytes();
}

This code works. As I said, this would probably be considered a bug as it's reminiscent of the C++-style *(pointer + index) type of logic (because I'm assigning 4-byte values into 4 indexes of a 1-byte array)..which is something that AGS tends to stray very much away from..but the fact of the matter is that it (presently) works perfectly.

I haven't currently released any code exploiting this method, so if I shouldn't (because it's something likely to be "fixed"), then I will either have to wait until the former bug is diagnosed and corrected, or just stick to using the original char values instead of using the bitwise operators to smash 4 of the chars together and save space.

If it's relevant, I'm running Windows 7, 64-bit.

[Wyz]

What was the error AGS returned, or it simply hangs? Hmm this could have something to do with breadcrumbs in the heap... 

No seriously you might have so much code in that function the virtual machine overflows or something. Why don't you encode the bread in base64 or something.

[monkey0506]

AGS doesn't have a 64-bit type, nor does it allow 64-bit operations*. Merging 4 single-byte values into a single 32-bit int was simply a way to cut down on the size of the module.

AGS doesn't give any error at all. If it gave an error that would be an error, not hanging up, which is what I described.

The game begins fading in and then simply stops. It doesn't finish fading in, doesn't give me control, doesn't apparently process any user input, it just..stops and hangs there. Again, this is only if I'm using an int array.

This issue isn't present in the current version of the module, as I said, but that's also because I've made other exceptions. I've actually managed to hit the compiler's token limit for uniquely accessed tokens, so in the current version of the module I'm using a char array of size 8000, and then using a separate function to assign the remaining 4344 bytes. I was able to go as high as 8300 (and some change), but I wasn't able to program the rest of the module without hitting the limit again. As a matter of fact, if importing the module into a game which uses several variables, that might still be an issue. I don't know if the limit applies only to each script or if it applies to all of them collectively.

Edit: *It will compile 64-bit operations (and 64-bit numbers), but the result is always incorrectly a 32-bit value.

Code: ags

Display("%d", 11158325035008 >> 32); // should display 2598

Incorrectly displays 2147483647 instead of 2598.

Code: ags

Display("%d", 2598 << 32); // should display 11158325035008

Incorrectly displays 2598 instead of 11158325035008.

Code: ags

Display("%d", (2598 << 32) >> 32); // should display 2598

Correctly displays 2598, but for the wrong reason.

[monkey0506]

I just wanted to make the comment that in exploiting the latter bug the reason I say I'm injecting a single 4-byte value into 4 slots of the char array while I'm using indices that increase by 1, and not 4, is that apparently at runtime the integer indices to the char array are automatically being multiplied by 4. So if I make the char array exactly 3086 bytes then an index of 3085 is passed through at runtime as 12340. This is probably in conjunction with the fact that I'm forcibly injecting integers instead of char values.

[Wyz]

Base64 is a way to encode binary data into a string. But I've encoded binary data in strings before with AGS and you can do it with less overhead.
AGS's character encoding accepts every character in string literals except chr(0), chr(10), chr(13) and ". What I usually do is open the file in notepad or something similar, change them to \n \r \" and also \ to \\. The only value that can never exist in strings is the termination character: \0. You need some sort of escaping sequence to include 0 values. Then you can encode images in strings. If you think this way is too much of a hassle you can also convert it into a hexadecimal string for twice the overhead and twice less the work .

[monkey0506]

The inability to pass '\0' into an AGS String is the reason I switched to using an array to begin with. I just wanted to get the thing out, so I went for the first method that worked. In any case, this is off-topic for this thread, which isn't about the how or why of what I was doing, it's about bug(s) with the way AGS is handling dynamic arrays.

Edit: However, yeah, I misunderstood what you meant. Base64 would actually work pretty nicely, and I wouldn't have to worry about losing any data so long as I read the file a byte at a time, which I'm already doing for the same reason.

I'll take a look into doing it that way because even though I'd have to split the data into a few String segments to make it manageable for my purposes, it should still save a significant amount of data over the way I was doing things.

[monkey0506]

Okay, I was taking a look at implementing some conversion functions to work with Base64, and it seems that any index passed into a dynamic char array is always being multipled by 4, including variable indices. This only happens at runtime, and only at the last-moment. Checking the value immediately before passing it to the array is of no use, because it returns the actual value. It's only after passing it into the array that this conversion is happening.

I tried to forcibly pass the binary data by using some bitwise shifts and such, but it was no avail. Since this is not the desired behavior, I'm reverting now to using an int array. Man, exploiting this bug was fun until I didn't want it any more. :