Direct3D scaling implementations

[Alan v.Drake]

Hello,
as pointed out countless times, DirectDraw mode and Direct3D mode do not draw characters/objects with different zoom levels the same way:


Left D3D, Right DD

Obviously the right one is the right way, so I'll push for it to become the default.
But for those who have somehow built their game around the non-standard D3D scaler, I can give the option to keep using the old scaling mode.

The working name for the feature is "scale_native_resolution".

Now the big question, should I add a checkbox inside winsetup ? Only from acsetup.cfg ? As a game setting ?
I could even make it changeable at runtime, but I'm not sure I should.

Let me hear your thoughts!

- Alan

[selmiak]

is the right version generated inside AGS from the left version? damned, this is interesting for scaling things. Or is this another version of the sprite completely?

[Crimson Wizard]

I believe more elaborate explanation is a must...
For instance, I do not even remember how to reproduce this. Does it happen all time in D3D mode when you have zooming walkable area, or you need more than that to get this effect?

is the right version generated inside AGS from the left version? damned, this is interesting for scaling things. Or is this another version of the sprite completely?

It is the same sprite, left version is rendered in Direct3D, right version in DirectDraw5.

Now the big question, should I add a checkbox inside winsetup ? Only from acsetup.cfg ? As a game setting ?
I could even make it changeable at runtime, but I'm not sure I should.

My personal opinion is that this mode belongs strictly to game design, because it can change the game looks too much. It is not like scaling filter too, which is applied over all the game image, keeping it consistent; it makes particular parts of the game look different from the rest of the image.
So I vote for game property, similar to Alpha Blend modes, which we already have to keep backwards compatible "incorrect" behavior.
Of course, nothing will prevent to add config option, or script command to change this at runtime later, but I honestly doubt that would be asked for. Maybe just for testing/debug purposes...

[Scavenger]

To reproduce this, you must have a game running with a scaler, such as 320x200 running at 2x nearest neighbour. Any characters standing on a scaled walkable area will be scaled down using the resolution the game is scaled to (with the example being 640x400), not the resolution the game was created at (320x200).

This wasn't so much of an issue back in the old days when people would simply scale 2x at most, or keep it at 1x, but this age of scaling to 4x or 5x the size, it is extremely noticable.

[Danvzare]

Ah yes, I've noticed this.
And I also realize I'm in the minority when I say I'm more of a fan of the left one.

Although considering Thimbleweed Park is going to have scaling like the left one, then clearly I'm not in too much of a minority.

Still, it would be awesome if you could put in the option to choose between them. Preferably in Winsetup.

[Crimson Wizard]

Still, it would be awesome if you could put in the option to choose between them. Preferably in Winsetup.

Welp, I was wrong .


I am very interested to know WHY that happens like that in the first place, i.e. what code makes it stretch up before rendering properly, because that may point to the proper solution.
This really bugs me, and the reason I want to underline this is that logically thinking the fix should remove certain operation or alter one, rather than adding additional one, like I've seen in the old Draconian code.

[Radiant]

Still, it would be awesome if you could put in the option to choose between them. Preferably in Winsetup.

I disagree. This affects the visual design of the game, and should be the choice of the game design team.

[Danvzare]

Still, it would be awesome if you could put in the option to choose between them. Preferably in Winsetup.

I disagree. This affects the visual design of the game, and should be the choice of the game design team.

The HQX filters affect the visual design of the game much more, and no one has ever complained about those being in Winsetup (to my knowledge).
And while it might not be comparable, choosing to use a voice pack or muting the music, would affect the audio design of the game, yet we still give the option.

People should be free to change it if they want to. If they want it exactly how the developers designed it, they won't change it. Who are we to judge how others play our games?

[Radiant]

The HQX filters affect the visual design of the game much more, and no one has ever complained about those being in Winsetup (to my knowledge).

You are incorrect. Most of the artists that I regularly work with have asked me if I could remove this option from winsetup.

[Danvzare]

The HQX filters affect the visual design of the game much more, and no one has ever complained about those being in Winsetup (to my knowledge).

You are incorrect. Most of the artists that I regularly work with have asked me if I could remove this option from winsetup.

Oh, well that put me in my place.
I still prefer giving people more options, so they can play the games the way they want to play them.

And my point still stands when it comes down to voice packs and music.

[Alan v.Drake]

So I vote for game property, similar to Alpha Blend modes, which we already have to keep backwards compatible "incorrect" behavior.

I don't think it should be a game property, unless we plan to actually support it. (And if we do, should we try to make the software renderer do the same ?)
Let's also keep in mind that this mode violates the game resolution property, which leads to some important side effects regarding DrawingSurface, DynamicSprite, and savegame screenshots.

To recap, the current D3D approach has the following quirks:

• objects and characters can be scaled with a higher resolution than native, resulting in mismatched pixel sizes
• objects merged into the background will become pixelated
• savegame screenshots are affected by playing resolution and filters
• DynamicSprite.CreateFromBackground() can give inconsistent results depending on resolution and scaling

There are multiple issues to consider here:

• DrawingSurfaces and savegame screenshots shouldn't be getting the bitmap from the screen output: the result from these call should be predictable, the developer might be using the input for special effects (e.g: a flashbang effect simulating retina burn, etc). Savegame screenshots wouldn't increase in size with the resolution and would look the same, whatever filter the player is using.
• if we want to actually support the "incorrect" behaviour how should we cope with all the side effects ?

I wouldn't officially extend support, mostly because AGS is ill equipped to handle this kind rendering the proper way. We'd need vector based movement and walk areas, a different approach to drawing surfaces

Still, it would be awesome if you could put in the option to choose between them. Preferably in Winsetup.

I am very interested to know WHY that happens like that in the first place, i.e. what code makes it stretch up before rendering properly, because that may point to the proper solution.
This really bugs me, and the reason I want to underline this is that logically thinking the fix should remove certain operation or alter one, rather than adding additional one, like I've seen in the old Draconian code.

Because

This might be the flipside of Dave Gilbert's problem. Is it possible to fix the Direct3D scaling so it respects the game's original resolution when it's running with a 2x, 3x or 4x filter (at least the nearest-neighbor filter)?

The problem is that the scaling is done with hardware acceleration in D3D mode, and as far as I know it's not possible to tell D3D only to scale to a certain resolution.

The only way of fixing this would be to do the scaling in software mode instead, but then you lose the benefits of using D3D in the first place.

I think it all started because the old code was written assuming you couldn't do it the directdraw way, so the rendering was performed directly to the presentation surface, thus skipping the part where all is rendered on a surface of native resolution that is later stretched to screen. For that reason the _renderSprite() function calculates the relative drawing position of sprites, unlike software mode.



Anyway, browsing the source of the OpenGL driver, I just noticed a variable called "_render_to_texture", that might be doing the same thing I'm doing on D3D.
If that's the case we should bring that into the equation.



- Alan

p.s: it's like we are coding a damned psx emulator... native resolution... backbuffer effects...
p.p.s: I've been writing this damn post since early this morning, I'm just submitting and hoping I haven't left out anything vital

[Crimson Wizard]

EDITed original post.

I don't think it should be a game property, unless we plan to actually support it.

I guess I cared too much about keeping all backwards compatibilities all this time, but I honestly think we should support it.

if we want to actually support the "incorrect" behaviour how should we cope with all the side effects ?

We just don't! The support will only cover effect that was present in the past.
If there would be ideas how to extend this support for other things you mentioned, then it will be implemented, if not, it is okay to keep it like this, with a warning in a manual.
E: This is also why I think that it should be game option, and winsetup should have it only if game developer enables it, because only game author can know if it can break the game or not.
(I.e. option could have following values: On (strict)/Off (strict)/On (by default)/Off (by default), with last two allowing player to change it).


All this obviously was an unintentional effect by mistake, but there were people who were using this incorrect behavior for their purposes for years. Some of the game developers have explicitly stated that they want this to be kept. I think we need to give them something in return of removing this unintentional engine mistake they were using.

I mean, there might be a way to implement this as a distinct feature, even though having issues with particular operations, in which case it also can be used to run old games which were intended using this effect as a part of game design.


PS.

AGS is ill equipped to handle this kind rendering the proper way. We'd need vector based movement and walk areas, a different approach to drawing surfaces

AGS engine is bad for many kind of things... and the real, although harsh, question here is whether it is wise to expect big changes with it.
In the past we discussed that many times, and frankly I tend to think that if you want to have an engine with all of those issues fixed, then there should be two branches of the engine: the one that intentionally keeps all the backward compatibility, and another that is completely rewritten, having many old code throwed out, and not aiming to support backwards compatibility at all.

[Danvzare]

My last two cents on this subject: https://blog.thimbleweedpark.com/gdc2016
The first comment, and Ron Gilbert's response.

If the legend that is Ron Gilbert can give options to the pixel-purists, then why can't we give options to the pixel-unpurists.

Also it needs to be supported for compatibility reasons, like CW said.

[Alan v.Drake]

Also it needs to be supported for compatibility reasons, like CW said.

It could be *tolerated* for compatibility reasons

- Alan

[sonneveld]

I'd probably see it as a game setting, and should be available in all modes (software, direct3d, opengl, future sdl etc)

[StillInThe90s]

I feel obliged to rant a bit, since that is what I usually do whenever this issue is brought up, every two years or so... So bear with me.

//Start of rant:

Spoiler

The D3D mismatched pixel issue is the main reason why I'm still using Alan's 3.2.1 build. I don't want to give players the choice of adding butt ugly mixed res scaling in my game, just like I don't want them to swap my purpose made font for Comic Sans, or add their favorite Lionel Richie tracks instead of the music I chose/wrote/recorded for it.
Apparently, people are designing games around the D3D scaling issue, so dumping it would be a bit harsh on them. However, adding more cluttering, non optional stuff to winsetup would be a bad idea imo. There used to be discussions about minimizing the setup because a lot of players had no idea what all the settings were for. So keeping winsetup clean is probably still relevant(?)
Also, the issue seems to be a potential game breaker as well, making non optional implementation a very bad idea:

This is also why I think that it should be game option, and winsetup should have it only if game developer enables it, because only game author can know if it can break the game or not.

...which makes CW's suggestion sound pretty neat:

(I.e. option could have following values: On (strict)/Off (strict)/On (by default)/Off (by default), with last two allowing player to change it).

Imo, HQX filters should also be optional if possible (I'm using an old build so maybe they already are).

[close]

//End of rant.

In short: Either in-editor option like CW suggested or if this is not possible, then good riddance.

EDIT: I'm sorry, is this thread about the main build or the new Draconian? (Rant applies to both I suppose... )

[Alan v.Drake]

(Rant applies to both I suppose... )

Yes.
Beware of switching to the last Draconian, it's based on 3.4.0 and is still missing a few things (my junk filters for DD), be sure to make a backup!

- Alan

[Danvzare]

or add their favorite Lionel Richie tracks instead of the music I chose/wrote/recorded for it.

If you put in the ability to change music volume (which you should), then people can simply mute your music and play their own background music.
I've actually done that with some games that have horrible soundtracks. Only about two I think.

Although I think that's besides the point. Why do you want people to experience your games in such a specific way?
Are you also going to force them to have a specific monitor, with a specific resolution, and with the brightness levels set to a specific value? Perhaps you're going to make sure they have a specific keyboard that makes a specific clicky noise when you press the keys?

What you're saying would the equivalent of a movie maker only wanting his film to be seen in a cinema, because a television wouldn't be able to give the full experience.
What your'e saying is like James Cameron's Avatar only having a 3D version, because that's what it was made to be watched in (screw those guys who prefer 2D).

When you release your game into the world, you've got let it go to some extent and understand people will want to play it their way. It's why mods are so popular on the PC!
You can't be so draconic in how it's played (and yes, pun completely intended).

[StillInThe90s]

Calm down, Danvzare.
If you want to make a extensively moddable game, that is fine. And If I don't want to include stuff that I think looks like crap in my game, then that should not be a huge issue. Sure people can ruin my game for all I care. I just do not want to ship it like that.

Beware of switching to the last Draconian, it's based on 3.4.0 and is still missing a few things (my junk filters for DD), be sure to make a backup!

Was that a joke or should I wait for a later version? Is it unstable or just missing bits?

[Crimson Wizard]

Danvzare, in my opinion, your examples are completely off scale with the setting in question. Everything you mention - brightness, etc - as well as previously mentioned gfx filters - they make an effect on all the game, in equal sense, and its core playability. While this option we were talking about makes an effect on particular objects' relative looks. It's like demanding that there were separate gamma setting or custom tinting slider for characters. Sure, thay could come handy in some cases, but forcing game developers to implement one regadless of what they want?

When you release your game into the world, you've got let it go to some extent.

But what defines this extent if not game designer? And why the method particular sprites in game are drawn should be an obligatory element in player settings?
When Ron Gilbert adds it to his game, he probably knows that his game will look fine with either variant. But that is not necessarily true for all games.

[Danvzare]

Calm down, Danvzare.

Danvzare, in my opinion, your examples are completely off scale with the setting in question.

Ok, sorry for overreacting.

And yeah CW, you make some good points.

I guess I just feel as though I'm the only one who rather likes the mismatched pixels, and thus feel as though I have to defend them. Which sounds pretty stupid when I put it into words.
Sorry again everyone.

(I.e. option could have following values: On (strict)/Off (strict)/On (by default)/Off (by default), with last two allowing player to change it).

Personally, I like this suggestion. Although I suppose to make things easier for the engine developers, a strict on/off might be better.
While I'd prefer it in the WinSetup, I guess I can understand why the developer would prefer it if they had control over it.

[Snarky]

I don't like telling people what they should spend their volunteer efforts on, but it all sounds like a whole lot of work and complexity just for optional backwards compatibility with a bug that should never have been there in the first place (and which behaves inconsistently between the two renderers). If ever there were a case for unceremoniously breaking backwards compatibility, I think this is it: Just fix it, and if people like the old, broken behavior they can use an old version of the engine.

[Crimson Wizard]

I don't like telling people what they should spend their volunteer efforts on, but it all sounds like a whole lot of work and complexity just for optional backwards compatibility with a bug that should never have been there in the first place (and which behaves inconsistently between the two renderers). If ever there were a case for unceremoniously breaking backwards compatibility, I think this is it: Just fix it, and if people like the old, broken behavior they can use an old version of the engine.

That was pointed out in this thread (and was said before) that while this behavior could have been introduced unintentionally, it may be valid feature if properly implemented (see Ron Gilbert's engine example).
So this is not simply about keeping backwards compatibility.

Now, from what I know, current problem Alan has is not in keeping the old behavior, but finding out a clean and optimal way to implement NEW one. Because so far we could not find what code specifically is making characters drawn that way. As far as I know, until now Alan was not able to actually remove the old behavior, so in his Draconian version he overrides it by adding extra operation. (He may correct me if I am wrong, but that was what I previously saw in his code).
E: to elaborate further, the problem here is not that it is objectively hard, but because neither of us have big knowledge in Direct3D. This also may be related to overall AGS's entangled code state...

Finally, most of the discussion here is about whether to have or not have an option in winsetup, which is a non-issue from the coder's point of view.
Don't mean to be rude, but really that's the case when there is more talk than work...

[Snarky]

That was pointed out in this thread (and was said before) that while this behavior could have been introduced unintentionally, it may be valid feature if properly implemented (see Ron Gilbert's engine example).
So this is not simply about keeping backwards compatibility.

Perhaps the effect might be of use if the whole engine was built around it (the Thimbleweed engine supports "sub-pixel" scrolling and things like that), but you already said:

We just don't! The support will only cover effect that was present in the past.

I simply don't think this "effect" (i.e. bug) by itself is worth preserving in any form, and certainly not with the limitations currently in place.

As for why it happens... isn't it simply that the screen is scaled and then the sprites are drawn, rather than the other way around? Drawing everything to a game-resolution texture and then scaling and rendering that to the screen (which I thought was what Alan was doing) seems like the proper approach to me.

[Crimson Wizard]

I simply don't think this "effect" (i.e. bug) by itself is worth preserving in any form, and certainly not with the limitations currently in place.

I've been asked by certain game developers in the past to keep it. How else is it decided whats worth and whats not?
The point is, whether this is needed to anyone as a part of game design, and if yes, whether it will be complex to keep it as an option. I am not the one to answer on the first question, because I do not make games with AGS. But answer on second question is: "probably not".

As for why it happens... isn't it simply that the screen is scaled and then the sprites are drawn, rather than the other way around? Drawing everything to a game-resolution texture and then scaling and rendering that to the screen (which I thought was what Alan was doing) seems like the proper approach to me.

He told me he tried doing that and something did not work well, so he is back to original code he used. And that code looked tad like a hack to me, doing some excessive operations. Then again, it was long ago when I checked it last time.
I did not have time to look into this myself now, so I do not know details.


EDIT: As I see this, also refering to your explanation above, the possible solution is in separating sprite rendering code into a function that accepts destination multipliers as parameters. Depending on setting, this function is called either before stretching whole scene with multipliers = 1.0, then streches the scene, or called with  correct multipliers enough to stretch all sprites without stretching the scene afterwards.
This may actually be what Alan did in Draconian too, but as I said, I do not quite remember this.
That may not be the most perfect way, but it should work for renderer which has accelerated scaling. Honestly, I do not think that really will work for software renderer, because it is slower, and doing two-pass rendering will be more complicated, because of how walk-behinds work. But that's a separate issue IMHO.

[Crimson Wizard]

Well, I found old Alan's code I loaded to my pc couple of years ago, and frankly, since I now understand what is going on in the code much better, I believe it could be way to go, just with certain cleanup & optimization. Adding an option to turn it between two rendering modes also seem to be super-simple to do.

[Alan v.Drake]

Well, to be fair Thimbleweed's engine is not really a good example to justify our broken scaling. That engine seems to run at a high native resolution and the assets are streched up by design.
The pixel purist mode will surely just be a shader that scales down or pixelizes. That's the exactly opposite thing of what's happening on AGS.

- Alan

[Crimson Wizard]

In case anyone curious, this switch was just added to master branch in official repository.
It allows to either set up exact fixed mode for the game, or let player toggle it in setup at will.
The switch currently works only for Direct3D mode.

[Crimson Wizard]

Hrm, well, this is kinda silly, but Alan had issues making a convenient name to this option . At least I think that it is very confusing as it is now.

Currently this setting is called "Scale native resolution", with ON making old Direct3D result, and with OFF making correctly scaled pixels (as DirectDraw).

I honestly do not know how to understand "scale native resolution" (cannot draw connection between the name and what it does), and I fear that will be super puzzling to the users who had never heard of this problem before (and maybe even those who had).

Currently I am looking for a good way to rename this to. My latest variant is "Subpixel sprite scaling". Although I am aware that subpixel rendering is kind of different thing, they have something similar in what the renderer is trying to achieve.
EDIT: another variant: "Use high-res pixels for sprite scaling".

I also invented following explanation to this property to be shown in the editor (and possibly manual):

"When drawing zoomed character and object sprites, AGS will take advantage of higher runtime resolution to give scaled images more detail, than it would be possible if the game was displayed in its native resolution."

I need opinions on whether it is elaborate or not.

[Danvzare]

Yeah, "Scale native resolution" is confusing.

Personally, I can't think of a good name for it, but I kinda like "Subpixel sprite scaling".
We'd have to hear other people's thoughts first.

That explanation to be shown in the editor though, is perfect!

[Crimson Wizard]

Are following variants correct in terms of English?

- Scale sprites in screen resolution, or
- Render sprites in screen resolution

[morganw]

I think it would be "at screen resolution".

[cat]

Will there be a new setting in the setup program? I was glad when the setup was finally slimmed down, adding another option doesn't sound like a good idea. Especially when I can and definitely will prevent the player from changing the sprite scaling.

[Crimson Wizard]

Will there be a new setting in the setup program? I was glad when the setup was finally slimmed down, adding another option doesn't sound like a good idea. Especially when I can and definitely will prevent the player from changing the sprite scaling.

There will be an option on Advanced tab. Game developer can enforce particular selection for their game (set in General Settings), which would render the checkbox in setup program always greyed out. If they did not, player may toggle it on and off.

[cat]

Ok, thanks for the clarification. Looking forward to the new version

[selmiak]

just asking, why is it greyed out and not just completely gone? curious players will feel cheated and maybe even complain to the gameauthor, well, just because...

[Crimson Wizard]

just asking, why is it greyed out and not just completely gone? curious players will feel cheated and maybe even complain to the gameauthor, well, just because...

That complies with other options that also get greyed out when they are not applicable to particular game.
Also that is plain simplier to do, if option gets removed there will be noticable empty space on window between controls, and we do not have automatic layout in setup.

[Crimson Wizard]

I found that this option already existed in OpenGL renderer all this time but was only available for Android (and probably iOS) ports. It is called "Supersampling".

[m0ds]

Hiya! Sorry to necro this, but what is the current method to achieve what's seen in the right hand image please?



If it's a particular version of the engine or what not that should be used, please let me know? So you can run it in directX mode and still retain the pixel feel of sprites. This thread seems to hint at things but I can't see a clear cut solution.

I need a solution that doesn't involve redrawing things...

[Crimson Wizard]

Hiya! Sorry to necro this, but what is the current method to achieve what's seen in the right hand image please?

If it's a particular version of the engine or what not that should be used, please let me know? So you can run it in directX mode and still retain the pixel feel of sprites. This thread seems to hint at things but I can't see a clear cut solution.

You need AGS 3.4.1, that has this looks by default. Optionally you may change "Render sprites at screen resolution" option in General Settings to either enforce particular look for your game or let players choose preferred looks in winsetup.

[m0ds]

Great, thanks CW! And there I was using 3.4.0 ... so close! But thanks for that, going to do it now.