SpriteCache::removeOldest attempted to remove a sprite which doesn't exist

[bx83]

Hi, been having a strange intermittent problem.

I'm using AGS 3.4.1.15 NMP.

This sprite error has happened twice, with the same sprite number, 6436. The sprite is nowhere near the View frames for Julius (main character, walking along to left), or the Farmer Cow (secondary character, running to left special animation) sprites. It has happened during the Farmer's running animation to the right in room x, and Julius walking in room y. They're both part of the same scene, but the rooms are 3 and 47, unrelated and made a long time apart, so god knows.
It's happened once while talking to the farmer (in room 47), which culminated in him running away right (then 2 seconds, sprite error); once with Julius walking towards the Farmer (room 3) - 2,3 seconds, sprite runtime error.

The sprite is close to 6435 and 6437, which occurs in the section I was importing/redoing after the sprite cache was corrupted (when I changed it from compressed to uncompressed; errors and some sprites replaced with blue teacup; turned it back, re-imported sprite, all good apparently). My thought is that somehow, the sprite number was corrupted, redone, but something of the corruption remained in the sprite library for some time. Not sure. Not sure how to fix or read an error log :/

Truly, truly hoping I don't have to manually re-import all sprites, all views, and spend the next 6mo redoing this one part of the game. it has 6-8000 sprites. I mean, originally I just went to general settings, sprite library, compressed->un-compressed->save->f'd up->compressed (after realizing the sprte library was now too large)->re-import. If this causes an immediate and lingering error, whys it even allowed?

Hoping for some good advice. Also, *it may not be* the sprite library (the sprites this probabaly effected, 6435-6437, are not part of a view which is used yet anywhere in the game), but I don't know.

Lost and frustrated :/

pics:
[imgzoom]https://redrom.ltd/img/room3.png[/imgzoom]

[imgzoom]https://redrom.ltd/img/room48.png[/imgzoom]

[bx83]

ps. it's not changing view (except for Farmer, but the error crashes everything after the view change - and even then, it's just lock/unlock animation of a view, not a ChangeView())
nor is it a Dynamic sprite - they're technically used in my custom dialogue script, however this error doesn't happen 150 times during all conversation, just this one (of the two) particular ones.
Not sure what else it could be. Julius (room 3) might have changed direction while walking and that caused the error? It wasn't a sprite close to his walking cycle.

[bx83]

Happened again, this time with a different character. Immediately after conversation, same sprite number. Here's the code for dialogue script:

Code: ags

//----------------------------------------------------------------------------------------------------------------------
// DIALOGUE
//----------------------------------------------------------------------------------------------------------------------

//----------------------------------------------------------------------------------------------------------------------
// Dialogue Options Get Dimensions
//----------------------------------------------------------------------------------------------------------------------

function dialog_options_get_dimensions(DialogOptionsRenderingInfo* info)
{

// Create a 1366x64 dialog options area at (0,704)

  info.X = 0;
  info.Y = 672;
  info.Width = 1366;
  info.Height = 96;
}

//----------------------------------------------------------------------------------------------------------------------
// Draw Dialog Options
//----------------------------------------------------------------------------------------------------------------------

function DrawDialogOptions(DrawingSurface* ds, DialogOptionsRenderingInfo* info)
{
  int i = 1, ypos = 0, xpos = 0;  
  ds.Clear(COLOR_TRANSPARENT);
  while (i <= info.DialogToRender.OptionCount)
  {
    if (info.DialogToRender.GetOptionState(i) == eOptionOn)
    {
      String str = info.DialogToRender.GetOptionText(i);  //get glyph number from option text
      int cur_img = str.AsInt;                            //current image is that number
      ds.DrawImage(xpos, ypos, cur_img);                  //draw this glyph
      xpos += 96;                                         //xpos+=width of glyph
    }
    i++;
  }
  //if (MyDynamicSpriteForTheFakeGUI != null) MyDynamicSpriteForTheFakeGUI.Delete();
}

//----------------------------------------------------------------------------------------------------------------------
// Dialog Options Render
//----------------------------------------------------------------------------------------------------------------------

function dialog_options_render(DialogOptionsRenderingInfo* info)
{
  DrawDialogOptions(info.Surface, info);
}

//----------------------------------------------------------------------------------------------------------------------
// Dialog Options Repeat Exec
//----------------------------------------------------------------------------------------------------------------------

function dialog_options_repexec(DialogOptionsRenderingInfo* info)
{
  int i = 1, xpos = 0;
  while (i <= info.DialogToRender.OptionCount)
  {
    if (info.DialogToRender.GetOptionState(i) == eOptionOn)
    {
      if (mouse.y >= info.Y && mouse.x >= xpos && mouse.x <= xpos+96)
      {
        info.ActiveOptionID = i;
        
        //transplanted in from dialogue_options_mouse_click() to get rid of background/old buttons in real time
        MyDynamicSpriteForTheFakeGUI = DynamicSprite.Create(info.Width, info.Height, true);
        DrawingSurface* ds = MyDynamicSpriteForTheFakeGUI.GetDrawingSurface();
        DrawDialogOptions(ds, info);
        ds.Release();
        
        return;
      }
      xpos += 96;
    }
    i++;
  }
  gFakeDialogOptions.Visible = false;
}

//----------------------------------------------------------------------------------------------------------------------
// Dialog Options Mouse Click
//----------------------------------------------------------------------------------------------------------------------

function dialog_options_mouse_click(DialogOptionsRenderingInfo* info, MouseButton button)
{
  if (info.ActiveOptionID > 0)
  {
    MyDynamicSpriteForTheFakeGUI = DynamicSprite.Create(info.Width, info.Height, true);
    DrawingSurface* ds = MyDynamicSpriteForTheFakeGUI.GetDrawingSurface();
    DrawDialogOptions(ds, info);
    ds.Release();

    gFakeDialogOptions.BackgroundGraphic = MyDynamicSpriteForTheFakeGUI.Graphic;
    gFakeDialogOptions.Visible = true;
    info.RunActiveOption();
  }
}


// END DIALOG FUNCTIONS

This is the only area in the game which has dynamic sprites.

[Crimson Wizard]

Are the reported sprites actually ones you have imported?

Note that they do not have to do anything with the current room. When running the game sprite cache continiously loads up and clears sprites to allow space to load new ones. "removeOldest" tries to dispose oldest sprite loaded in the cache (which could be a sprite from the first room, for example).

Such error occurs if sprite cache tried to remove a sprite marked as dynamic. "attempted to remove a sprite which doesn't exist" message may be misleading, in reality it means "attempted to remove sprite which was not loaded from game data". This is because "removeOldest" is forbidden to remove dynamic sprites.

I would not completely ignore this possibility, but I do not think this particular error may be related to game files corruption. The flags that mark sprite as dynamic or non dynamic are applied when game is loaded for run, regardless of contents of the sprite file.

In the past we've found couple of errors in sprite cache (and I rewrote parts of it in 3.5.0 trying make it work better).

I think most errors were related to dynamic sprites. But also I noticed in the past that sprite cache behaves unreliably if it's too small for the game. Since your game is very hi-res, maybe you need to increase cache's size. Perhaps try to open winsetup's Advanced tab and choose highest value in the sprite cache size, and see if it improves anything.

I mean, originally I just went to general settings, sprite library, compressed->un-compressed->save->f'd up->compressed (after realizing the sprte library was now too large)->re-import. If this causes an immediate and lingering error, whys it even allowed?

It is allowed because it supposed to work. If it does not, please give more details what happened.

[bx83]

Are the reported sprites actually ones you have imported?

Yes, probably, but they don't exist anymore. My my sprite library is a little bit of a rats nest - imagine starting out knowing nothing and recovering from a stroke, and then 2.5 years later being on the verge of releasing a game -- it's like that, growing pains

Well it would appear to be eureka, same error hasn't occurred once

I increased the cache size in the debug game (/game/_debug/acsetup.cfg with Build->game setup), recompiled; it seems to work. It was at 128mb before, now at 512mb. Could it be higher? In text of the game setup file anyway? Currently 'cachemax=4194304' which is about 4200mb; in the game options under General Setup it's 4GB. Not sure about these two sizes or whether I'm reading it wrong, but either way it does work better now and error free (fingers crossed) with 4 times the cache space.

settings:

Code: ags

[sound]
digiid=-1
midiid=-1
digiwin=-1
midiwin=-1
digiindx=0
midiindx=0
digiwinindx=0
midiwinindx=0
[misc]
game_width=1366
game_height=768
gamecolordepth=32
antialias=1
notruecolor=0
cachemax=4194304
user_data_dir=
shared_data_dir=
titletext='Lost on Cow Island' Setup
[graphics]
driver=OGL
windowed=1
screen_def=scaling
game_scale_fs=stretch
game_scale_win=max_round
filter=stdscale
vsync=1
render_at_screenres=0
[language]
translation=
[mouse]
auto_lock=0
speed=1

All good for testing now, I'll get back to you in 24 hours if it comes up again

ps. as for corrupt sprite cache, why is the corruption caused by asking it to uncompress/recompress? Any experience with bugs in the sprite cache with compression algorithm, anyone else done this? It doesn't blow up in my face (ie. it doesn't just say 'sprite file corrupted, oh well, hope you had fun trying'), but several (or 10s or 100s) sprites turn to the blue teacup and have to be re-imported.

[Cassiebsg]

Well, without me being an expert at this, it's a bit logic that that would happen, since the sprite file in this version can't be bigger than 2GB. So when you turned your compressed file into uncompressed, all sprites will fill more space, and once the file reaches the 2G, it just turns all remaining sprites to be uncompressed into a blue cup (default behavior). There is simply no more space. Maybe selecting the compress->uncompressed process should come with a warning about this being a possibility... (not sure if warns since I never tried it myself). Anyway this is soon a dead point, since this size limit on the sprite file has been removed on the new beta. 

[Dualnames]

Here's how u fix this. Before every sprite's creation

DynamicSprite.Create

put a

if (dynamicspritename!=null) dynamicspritename.Delete();

That way you ensure the sprite being handled properly.

[Crimson Wizard]

Well, without me being an expert at this, it's a bit logic that that would happen, since the sprite file in this version can't be bigger than 2GB. So when you turned your compressed file into uncompressed, all sprites will fill more space, and once the file reaches the 2G, it just turns all remaining sprites to be uncompressed into a blue cup (default behavior). There is simply no more space. Maybe selecting the compress->uncompressed process should come with a warning about this being a possibility... (not sure if warns since I never tried it myself).

Ah! Good point, that might definitely explain what happened.

I guess we may add a special case in 3.5.0 since its now being close to release (although sprite file limit is much higher there, so idk if such problem may occur in practice again).

Here's how u fix this. Before every sprite's creation

DynamicSprite.Create

put a

if (dynamicspritename!=null) dynamicspritename.Delete();

That way you ensure the sprite being handled properly.

Actually, any dynamic sprite should be deleted automatically as soon as there's not a single pointer to it.

[bx83]

UPDATE:
With the new cache size, the game goes for an hour of play (instead of 10minutes or less), but it now gets the same error as before, and says sprite 70 is the problem. No sprite for 70 has been imported.
Reduce number of sprites..?
Added 2 new sprites; number of the second one is 70. So, now there *is* a 70. Next time, different number?..
Also could mean the sprite stack is off by 1...?

Is 3.5.0 going to be released soon?
Could certainly do with no max size for sprites.

[Crimson Wizard]

With the new cache size, the game goes for an hour of play (instead of 10minutes or less), but it now gets the same error as before, and says sprite 70 is the problem. No sprite for 70 has been imported.

Well, this means it's a dynamic sprite that gets incorrectly scheduled for cache cleanup.
I will try to find time and look around the 3.4.1 code again to see if I can guess when this may happen.

[bx83]

Hasn't happened so far on 3.5.0b5...

[eri0o]

You mean 3.5.0.14, right?

[bx83]

Yes, not once. The sprite file in 3.1.4 doubled in size each time I added a sprite back and saved, till I gave up a left it compressed.
The uncompressed sprite file is 3.5.0b5 (which decompressed with no issues) is 3.4GB.

[Crimson Wizard]

I rewrote some logic inside sprite cache in 3.5.0 too, so that could fix the bug with "removeOldest", although to be honest, I still do not know what it was earlier. Guess it was fixed by luck.