antialiasing problem (with screenshots)

[EnterTheStory (aka tolworthy)]

This picture should explain my problem:


I am using 16 bit color. I realise that 32 bit color is needed for edge antialiasing, but I only need to antialias faces. All my characters have white edges and most of the backgrounds are white, so I don't care if the edges are jaggy. But where the heck are these dark artefacts coming from? There are no dark colors anywhere near the edges of the sprites, and the transparent color is pink. I've even added two pixels of white around the sprite, and the scaled screenshot is pretty close to 100 percent sprite size. But still the dark lines appear! What am I doing wrong?

[EnterTheStory (aka tolworthy)]

Eleven hours later...

I see I am not alone in noticing this: over a year ago, Adamski described the same problem.

Adamski is a lot smarter than me (he even has "moderator" in his title) and yet nobody replied to his question. May I conclude that this problem is intractible? Perhaps even (whispered voice) a bug?

[Pumaman]

I believe this was traced to the anti-aliasing library anti-aliasing the image against the pink background, which leads to artefacts around the edge of the image. I did try out AASTR2 as suggested in that thread, but it had problems of its own.

Can I just clarify that the "antialiasing" you're talking about is the option in Setup to smooth scaled sprites? If so, then it might be worth trying out AGS 3.0 with the Direct3D driver, which does this in hardware and should not have these effects.

[EnterTheStory (aka tolworthy)]

Sorry, yes, I meant smoothing. It's good news if this has been fixed in 3.0 - thanks.  However, It's important to me that my game works in Linux, so I'll have to stick with 2.72 for the foreseeable future.

I guess I'll need to disable smoothing and blur the faces to compensate (so eyes and mouths don't disappear when scaled). Is there a way for me to disable the smoothing option in the standalone setup program so users don't click on it?

[Pumaman]

Currently it's not possible to disable the option. However, I'm trying to remember if this problem was solved, because if the anti-aliasing was always doing this then it would pretty much render the feature useless ... unless most of the time people use sprites with well-defined edges...

[GarageGothic]

Well, although you can't remove it from the setup, you can override it by setting "game.disable_antialiasing = 1;"  in the game_start script.

[EnterTheStory (aka tolworthy)]

if the anti-aliasing was always doing this then it would pretty much render the feature useless ... unless most of the time people use sprites with well-defined edges...

Or mid-range backgrounds? The "dark" edge is often a grey of 50 percent brightness.

Maybe I should redesign my game along the lines of the Cramp Twins)

you can override it by setting "game.disable_antialiasing = 1;"  in the game_start script.

So the user would tick the box but nothing would happen?

[GarageGothic]

So the user would tick the box but nothing would happen?

Exactly. The game would behave the same whether the box was ticked or not.

[EnterTheStory (aka tolworthy)]

Thanks. If I can access the smoothing variable to disable it, then I guess that I could use it to trigger an alert when the game is played? Like "you clicked on X but for technical reasons..."

[GarageGothic]

Strangely, game.disable_antialiasing doesn't actually read the current state of anti-aliasing. It always returns 0 until you've changed it yourself.

However, you could use File.ReadRawLineBack() to check acsetup.cfg for the line "antialias=1", and - if it exists - display a message about the antialiasing being deactivated for technical reasons. Something like this:

Code: ags

File *cfgfile = File.Open("acsetup.cfg", eFileRead);
if (cfgfile != null) {
   while (!cfgfile.EOF) {
      String cfgline = cfgfile.ReadRawLineBack();
      if (cfgline.Contains("antialias=1") != -1) {
         Display("You've selected smooth character scaling, but for technical reasons this feature has been disabled.");
         }
      }
   cfgfile.Close();
   }
game.disable_antialiasing = 1;

You could set another variable to finish reading the file as soon as the correct line is found, but since the .cfg is so short and "antialias=1" is often the last line, I didn't bother.

[EnterTheStory (aka tolworthy)]

Thanks