AGS 3.6.0 WIP (Alpha 21) - SDL2-based engine + Unicode support

Started by Crimson Wizard, Thu 25/03/2021 02:28:54

Previous topic - Next topic

Crimson Wizard

Quote from: nightmarer on Sun 28/11/2021 23:32:28
Any update about the PNG compression for this new version?

If you mean changing the compression method for the game sprites, there were no actual plans for this within 3.6.0.

EDIT: ags currently has a ready support for LZW compression which is used with room backgrounds for some reason. It might be curious to experiment also using this with sprites (optionally, if certain setting is on) and see what compression ratios it can achieve and what will be a perfomance impact.

Quote from: nightmarer on Sun 28/11/2021 23:52:42
BTW I see at the beginning of my game that the black background, which it was blac at 3.5.1 now is pink.
After that everything is fine.

Do you mean the room's background, or before the first room fades in? Does this depend on the graphics driver selection?

nightmarer

Quote from: Crimson Wizard on Sun 28/11/2021 23:56:36
Quote from: nightmarer on Sun 28/11/2021 23:32:28
Any update about the PNG compression for this new version?

If you mean changing the compression method for the game sprites, there were no actual plans for this within 3.6.0.

EDIT: ags currently has a ready support for LZW compression which is used with room backgrounds for some reason. It might be curious to experiment also using this with sprites (optionally, if certain setting is on) and see what compression rates it can achieve and what will be a perfomance impact.

Our sprites are 1,6 GB and our acsprset file is 3,4 GB. It is a shame that our game is going to be so huge in disk space.

Quote from: Crimson Wizard on Sun 28/11/2021 23:56:36

Quote from: nightmarer on Sun 28/11/2021 23:52:42
BTW I see at the beginning of my game that the black background, which it was blac at 3.5.1 now is pink.
After that everything is fine.

Do you mean the room's background, or before the first room fades in? Does this depend on the graphics driver selection?

There where is empty background before it was black, and now it is pink/purple. This purple screen starts as soon as there is a fade in. Maybe it is showing the alpha layer as purple, when it should be invisible.

Crimson Wizard

Quote from: nightmarer on Mon 29/11/2021 11:46:39
There where is empty background before it was black, and now it is pink/purple. This purple screen starts as soon as there is a fade in. Maybe it is showing the alpha layer as purple, when it should be invisible.

Is the "fade" layer itself purple, or something behind it? Do you use standard fade effect, or other methods, like gui with changing transparency?
Could you record a video of what is happening?
Do you know approximately since which build this started?

nightmarer

Not sure from which build started, but I will provide you a video.

Crimson Wizard

Quote from: nightmarer on Mon 29/11/2021 14:32:20
Not sure from which build started, but I will provide you a video.

Actually, I've been testing some of my own games, and it looks like similar thing happen if you have a room without any imported background, then there's a 320x200 magenta rectangle drawn instead of regular black one.


Meanwhile, I made an experimental build that lets select a "LZW" compression for sprites:
https://www.dropbox.com/s/d51t7xnmnwfkzh8/ags-3.6.0--experiment-spritelzw.zip?dl=0
it replaces regular "Compress sprites" option to "Sprite compression type", which has None / RLE (current) / LZW.

Could you check that out and see what difference does this make to your game sprites? I tried this with my games, but the effect was kind of small (but my largest sprite file is around 15 mb).
NOTE that it's best to use this on a side copy of a project, not the main one.

If this does not help much then I may look into adding Deflate (compression used by PNG) or else.

arj0n

Tried the 3 sprite file compression settings on one of my latest project.

Filetype sprites before importing into ags: png,
largest being still pretty small: 35,5 KB (36.369 bytes)

Game:
Resolution: 640x480, 32-bit true-colour
Total sprites: 1855 / 90000
acsprset.spr filesize: 63,2 MB

Results after compile:
None: game.exe filesize: 550 MB (576.947.304 bytes)
LZW: game.exe filesize: 138 MB (144.977.308 bytes)
RLE: game.exe filesize: 105 MB (110.966.094 bytes)

Crimson Wizard

Quote from: arj0n on Tue 30/11/2021 23:35:10
Game:
Resolution: 640x480, 32-bit true-colour
Total sprites: 1855 / 90000
acsprset.spr filesize: 63,2 MB

Results after compile:
None: game.exe filesize: 550 MB (576.947.304 bytes)
LZW: game.exe filesize: 138 MB (144.977.308 bytes)
RLE: game.exe filesize: 105 MB (110.966.094 bytes)

Could you post sizes of acsprset.spr after changing compression and compiling, not full game.exe?
Also, do you have an estimate on the total size of source sprites?

arj0n

Quote from: Crimson Wizard on Wed 01/12/2021 00:48:19
Could you post sizes of acsprset.spr after changing compression and compiling, not full game.exe?
Also, do you have an estimate on the total size of source sprites?

sure:

source sprites: 2.352 png files; 6,03 MB (6.328.527 bytes)

Sprite file compression:
None: acsprset.spr filesize: 475 MB (498.297.296 bytes)
LZW: acsprset.spr filesize: 63,2 MB (66.327.300 bytes)
RLE: acsprset.spr filesize: 30,8 MB (32.316.086 bytes)

Crimson Wizard

Quote from: arj0n on Wed 01/12/2021 17:49:02
source sprites: 2.352 png files; 6,03 MB (6.328.527 bytes)

Sprite file compression:
None: acsprset.spr filesize: 475 MB (498.297.296 bytes)
LZW: acsprset.spr filesize: 63,2 MB (66.327.300 bytes)
RLE: acsprset.spr filesize: 30,8 MB (32.316.086 bytes)

This is horrendous...

Anyway, it's curious how LZW is twice as worse in your case, because in the hi-res 32-bit game I tested LZW was around 2% better than RLE:
raw: 14238 KB, RLE: 2693 KB, LZW: 2650 KB.
I'd wish to see other various projects tested, as results will likely depend on the amount of graphic detail and colors in a single sprite. But for now LZW does not seem worth adding, unless someone finds a project that displays better results with it. I suspect it may be best for larger photorealistic images.

Meanwhile, there are 2 more methods I am planning to add for this experiment, one is PNG's deflate algorithm, and another is basically to store sprites as indexed images whenever possible (if the total number of used colors is 256 or less). Latter has been suggested couple of years ago and i've been keeping that in mind for a while. Indexed format may in fact be used combined with other compression for even bigger effect.

nightmarer

Quote from: Crimson Wizard on Tue 30/11/2021 02:10:07
Quote from: nightmarer on Mon 29/11/2021 14:32:20
Not sure from which build started, but I will provide you a video.

Actually, I've been testing some of my own games, and it looks like similar thing happen if you have a room without any imported background, then there's a 320x200 magenta rectangle drawn instead of regular black one.


Meanwhile, I made an experimental build that lets select a "LZW" compression for sprites:
https://www.dropbox.com/s/d51t7xnmnwfkzh8/ags-3.6.0--experiment-spritelzw.zip?dl=0
it replaces regular "Compress sprites" option to "Sprite compression type", which has None / RLE (current) / LZW.

Could you check that out and see what difference does this make to your game sprites? I tried this with my games, but the effect was kind of small (but my largest sprite file is around 15 mb).
NOTE that it's best to use this on a side copy of a project, not the main one.

If this does not help much then I may look into adding Deflate (compression used by PNG) or else.
I'm afraid it didn't change the size of the asprset.spr. Do I need to do anything or just run it?

Crimson Wizard

Quote from: nightmarer on Wed 01/12/2021 21:51:58
I'm afraid it didn't change the size of the asprset.spr. Do I need to do anything or just run it?

You need to choose a "Sprite compression" in General Settings. I dont remember if saving is enough, so perhaps rebuild the game to be sure. The "RLE" is standard compression, "LZW" is a new choice added.

nightmarer

Ok I see, what it needs to be changed.
LEH 3.481.618 KB - I had this one selected
LZW 2.934.687 KB

nightmarer

Quote from: nightmarer on Mon 29/11/2021 14:32:20
Not sure from which build started, but I will provide you a video.
BTW I sent you the videos through Discord

Crimson Wizard

Reuploaded the experimental version with the new method that stores 16-bit and 32-bit images as 8-bit with palette so long as they have below 257 different colors in them.
https://www.dropbox.com/s/d51t7xnmnwfkzh8/ags-3.6.0--experiment-spritelzw.zip?dl=0
This new method is called "Indexed" for short.

My own test results vary depending on the game's graphic style. In the game with good quality antialiased sprites the effect is barely noticeable (reduces size by 10%) as most sprites have >256 colors with varying alpha channel values; but in the game with "old-school" pixel graphics it's comparable to RLE.
Might try another version of indexed image store where alpha channel is saved separately, which may significantly reduce the number of color variations and fit them into the 8-bit palette.

Also would need to make it possible to combine "indexed" format with other compressions (e.g. Indexed + RLE or Indexed + LZW).

arj0n

640x480, 32-bit true-colour
Total sprites in project (F2):   1855 / 90000
source sprites: 2.352 png files; 6,03 MB (6.328.527 bytes)

Sprite file compression:
None: acsprset.spr filesize: 475 MB (498.297.296 bytes)
Indexed: acsprset.spr filesize: 118 MB (124.652.814 bytes)
LZW: acsprset.spr filesize: 63,2 MB (66.327.300 bytes)
RLE: acsprset.spr filesize: 30,8 MB (32.316.086 bytes)

Crimson Wizard

Quote from: arj0n on Thu 02/12/2021 09:00:23
640x480, 32-bit true-colour

Sprite file compression:
None: acsprset.spr filesize: 475 MB (498.297.296 bytes)
Indexed: acsprset.spr filesize: 118 MB (124.652.814 bytes)

Well, it looks like it saved virtually all of your 32-bit sprites as indexed 8-bit, reducing size 4 times as expected :).
But yes, this is maximum of what it can do, so for better results it has to be combined with the other compression types.

arj0n

Given all sprites in my test do have below 257 different colors in them, this seems like a good result to me already :)

nightmarer

320x200, 32-bit true-colour
Total sprites in project:   35703 / 90000
PNG folder: 1,61 GB (1.734.822.663 bytes)

Sprite file compression:
None: acsprset.spr filesize: 8,77 GB (9.425.965.010 bytes)
Indexed: acsprset.spr filesize: 7,66 GB (8.229.703.633 bytes)
LZW: acsprset.spr filesize: 2,79 GB (3.005.119.147 bytes)
RLE: acsprset.spr filesize: 3,32 GB (3.565.176.753 bytes)

arj0n

That's quite a different result, nightmarer.

My 32-bit background sprites seem to have a max amount of only about 17 different colors in them.
When looking at your 'indexed' compression result, I suspect your sprites have a higher amount of different colors?

nightmarer


SMF spam blocked by CleanTalk