Engine versioning

[Crimson Wizard]

Here I want to discuss following questions:
1. How often (when, in which cases) should the version change?
2. Should the engine and editor versions strictly correspond? For example, if we change something in the engine, but nothing in editor, should we still increase editor's version? What about opposite situation (some editor features reworked, but engine stays the same)?


Some technical information to be considered.
AGS engine uses three built-in version string constants:
AC_VERSION_TEXT; currently = "3.21 ". Used only when printing startup information to console. The string looks like

Adventure Game Studio v3.21 Interpreter
Copyright (c) 1999-2011 Chris Jones and 2011-20xx others
ACI version 3.21.1115

Notice, there are two version strings here, first being AC_VERSION_TEXT, and another ACI_VERSION_TEXT (explained below). It looks like AC_VERSION_TEXT is simply the "major" version string, without build number, used solely for informational purposes.

ACI_VERSION_TEXT; currently = "3.21.1115"SPECIAL_VERSION, where SPECIAL_VERSION may be empty string, or "NMP" for builds without MP3 support.
NOTE: editor uses AGS_VERSION = "3.2.0" string when compiles the game.
This is very important string, because it is tested against one stored in game data file, as well as savegames, to know if this engine is compatible with loaded data. On other hand, there's some strangeness in how it is done, and I think there's sense in revising it in future.
First of all, the game data file contains numeric id of the version too, which is simplier to compare. When game is loaded, first engine checks if its numeric version id is not lower than required. If this check fails, engine quits. Then, it compares ACI_VERSION_TEXT with the string, stored in game file. If the result of string comparison is > 0 (the required string is alphabetically 'higher'), engine displays a mere warning and continues execution. This purpose of this extra check is a mystery to me. Besides, it relies on string comparison, which may simply fail if the "required version" string has any extra characters attached to it. For example, if the engine version is "3.3.1200", and the one saved in data file is "3.3.1200NMP" (no-mp3 build), the test will fail, because "3.3.1200NMP" has higher alphabetical value.
This may not have big consequences for loading game data, but it will have consequences for loading saved games, where ACI_VERSION_TEXT check is the only and decisive version test.
When savegame is written, ACI_VERSION_TEXT is saved in there. When it's loaded, current ACI_VERSION_TEXT is compared with the one, read from savegame. If current ACI_VERSION_TEXT has 'lower' value, test fails. Personally, I see a problem here, because increase in version does not always mean breaking compatibility. I would suggest to save "lowest compatible version" to savegame instead. In such case the savegames made by newer engine build will load on (some) older ones, which is good both for testing and returning to previous interpreter in case new one introduced any play-blocking bugs.

LOWEST_SGVER_COMPAT; currently = "3.20.1103"SPECIAL_VERSION.
This string is used to test savegame for being saved by "too old" engine. If the engine version, stored in savegame, is alphabetically 'lower' than LOWEST_SGVER_COMPAT, the engine will refuse to restore the game.

[Crimson Wizard]

More on this. Along with engine version string, AGS defines following version numbers:
Game File Version. This is a numeric constant, which defines general game data format. Vanilla AGS 3.2 does not support loading games with lower game file version, newer engine allows to load those of version 2.5 and higher (although it does not guarantee 100% compatibility at the moment). Important thing is that game file version is not strictly bound with engine version. Two engine versions may work with same game file version (if data format wasn't updated in-between their development).
Room File Version. A numeric constant, which defines room data format. Similarly to general game format, it may stay the same between engine releases.

Concerning game format, I must correct myself:

When game is loaded, first engine checks if its numeric version id is not lower than required. If this check fails, engine quits. Then, it compares ACI_VERSION_TEXT with the string, stored in game file. If the result of string comparison is > 0 (the required string is alphabetically 'higher'), engine displays a mere warning and continues execution. This purpose of this extra check is a mystery to me.

The "numeric id" checked on game load, is not an engine version, but game file version (its format). Obviously, if the engine does not support certain game format, it won't be able to load it. But the difference in engine version simply means, that this engine build may run the game differently, than the one it was made for.

Concerning savegames. As I said, there's version string written to them, and if it does not strictly match current engine's, engine refuses to restore the game. I was thinking that in our case we have a hugely modified new version of an engine, but with no differences to data formats and the ways to interpret loaded data. If we increase the engine version, however, the savegames made by new engine will become incompatible with original games made on vanilla 3.2. This is illogical, since technically savegame format, and saved data is totally the same. That's why my suggestion is to write not the current engine version string, but "earliest backwards-compatible" version string to the save games, so that, in case of any need, earlier versions of the engine could restore saves made by newer engine builds.

[JJS]

1. How often (when, in which cases) should the version change?

Till now the engine version was increased every time there was a "significant" change. The bigger the change, the higher the version jump. This sounds reasonable to me because there is no fixed release schedule that could dictate a version number.
For the current development version 3.3 would be suitable I think.

2. Should the engine and editor versions strictly correspond? For example, if we change something in the engine, but nothing in editor, should we still increase editor's version? What about opposite situation (some editor features reworked, but engine stays the same)?

In my opinion: yes to both. This avoids confusion and is also how things were handled in the past. It could be specified that the "build" number (the last four digits) of the version don't have to match since increasing that should be more of a small bugfix thing than a feature change.

Concerning savegames. As I said, there's version string written to them, and if it does not strictly match current engine's, engine refuses to restore the game. I was thinking that in our case we have a hugely modified new version of an engine, but with no differences to data formats and the ways to interpret loaded data. If we increase the engine version, however, the savegames made by new engine will become incompatible with original games made on vanilla 3.2. This is illogical, since technically savegame format, and saved data is totally the same. That's why my suggestion is to write not the current engine version string, but "earliest backwards-compatible" version string to the save games, so that, in case of any need, earlier versions of the engine could restore saves made by newer engine builds.

I agree with this.


Also: For a release based on the current sources the game and room file version should stay the same as 3.2.1.

[BigMc]

For the current development version 3.3 would be suitable I think.

Normally version numbers should exactly define what the code is. So one commit could be tagged with the version and that's the version. For development versions that does not work. We could use odd minor versions for development versions and even versions for releases, so that we would have version 3.3 now and some time in the future there will be tagged commits 3.4, 3.4.1, 3.6, 3.8 etc.

[scottchiefbaker]

I think a BUILD_STR feature would be a handy thing to have

[BigMc]

Btw, I consider the version 3.21 to be higher than 3.3.

[Crimson Wizard]

Btw, I consider the version 3.21 to be higher than 3.3.

It should be 3.30 then.

[s_d]

In most software, the major version signifies some significant API breakage.  Perhaps that could be reserved for save-game file format changes, or no longer supporting old games or whatever.  The minor version is discretionary, but seems like we all agree that it should, most importantly, represent Editor and Engine compatibility.

Then, the final ("micro", or "build") would mostly be used for issue tracking, fixes, and very minor features that introduce no regressions and are not germane to the Editor (e.g. new CLI options, debugging print statements, etc), or in the Editor and have no effect on the Engine (e.g. GUI changes).

Seems like that's sort of what you guys have agreed on, yes?

[Crimson Wizard]

Seems like that's sort of what you guys have agreed on, yes?

I think it is.

One more question though: should the last 4-digit (or "micro" version, if you like) be reset when larger numbers change, or incremented by 1?

[BigMc]

One more question though: should the last 4-digit (or "micro" version, if you like) be reset when larger numbers change, or incremented by 1?

Reset. If it needs 4 digits or should just start with 1 for the first bugfix release is another question. Apart from the fact that no major release will ever get 1000 bugfix releases, that would still be possible without always having 4 digits.

[Crimson Wizard]

Reset. If it needs 4 digits or should just start with 1 for the first bugfix release is another question. Apart from the fact that no major release will ever get 1000 bugfix releases, that would still be possible without always having 4 digits.

Current AGS version is .1115

[BigMc]

Did that really come from 1115 bugfix releases or is it just a made up number?

[BigMc]

Ok, let me sum up what I think would be a sane approach.

Use only version numbers a.b.c or a.b for ACI_VERSION_TEXT, where a,b,c are integers. The same for engine and editor. For comparisons, first compare the, a's then the b's and then the c's, where a missing c means it's 0. No more string comparisons, and let's get rid of SPECIAL_VERSION. Such information can be stored in the engine in a different way. SPECIAL_VERSION should be ignored for comparisons in the future.

Odd b's are for development versions and they are used over the time period between two releases. Even b's for releases. If the next release is 3.30, we could use 3.29 right now.

[Crimson Wizard]

I just found, that the SPECIAL_VERSION causes buffer overflow if set.

Check this:
https://github.com/adventuregamestudio/ags/blob/master/Engine/main/engine.cpp#L1218
The ACI_VERSION_TEXT is being copied to the limited buffer of 10 chars:
https://github.com/adventuregamestudio/ags/blob/master/Engine/ac/dynobj/scriptsystem.h#L29
The only thing that save No-MP3 versions is that "int reserved[5]" at the end...

[s_d]

Use only version numbers a.b.c or a.b for ACI_VERSION_TEXT, where a,b,c are integers. The same for engine and editor.

Very reasonable.

For comparisons, first compare the, a's then the b's and then the c's, where a missing c means it's 0.

Very standard version number handling;  we see this everywhere.  Definitely follows the principle of least astonishment.

No more string comparisons, and let's get rid of SPECIAL_VERSION. Such information can be stored in the engine in a different way. SPECIAL_VERSION should be ignored for comparisons in the future.

Certainly an elegant idea, but are there any legacy considerations?  I.e., is this, in any way, used by game scripting, or is it informational only?  IMO, string comparisons are a rotten way to perform versioning checks.

[Crimson Wizard]

If we are to change engine version to X.X.X.XXXX format - to make it consistent with editor, can we use last 4 digits to mark intermediate development builds? I have a feeling Chris Jones did that in the past, because I saw games having different 4-digit number than official release (Gemini Rue is 3.2.1.1111, while official version is 3.2.1.1115).
I must confess that I am not sure I understood the idea behind using odd/even numbers for public/development versions (the need to do such thing).

[BigMc]

I must confess that I am not sure I understood the idea behind using odd/even numbers for public/development versions (the need to do such thing).

When the source is publicly available, every git commit will be used by someone. Right now there are many different git commits where the version is 3.30. So when someone says he uses 3.30, we don't know exactly what code he used. When we use even version numbers for releases, then we know that someone who says he used say 3.32 used exactly the released version, while someone with version 3.31 used some intermediate development version.