Easing myself into the Engine source code.

[Sslaxx]

I've been thinking about getting involved more with the engine source, maybe looking at all the -fpermissive warnings I get whenever I compile the engine's source code. What's the best way to go about this? Anyone got any advice? The code isn't exactly well-documented in places either, and looking at documenting the code might be something to do (once I'm more familiar with it's internal workings).

[Crimson Wizard]

huh
No idea what is the best way. I'll mention some things.
NOTE: there are things below which are based on my personal opinion, and may be an object of discussion.

The code in our repository is a vastly split version of original code, that once was stuffed in very few files (one of them exceeded 30.000 lines in length). The first thing I did was splitting them in parts. In retrospect, I made mistakes; probably I should have start with understanding ties even before splitting, but I thought that splitting will help. Hard to tell, what would be best. Anyway, splitting went a bit wrong, and there are files that should not exist, their code rather belongs to others.
One of the reason I wasn't hurrying reorganizing this code is that I believed many things should be rewritten there, making code more compact. But my first assumptions about amount of time that may take were wrong.

Along with AGS code, there are also sources of some parts of Allegro and few accompanying libs. I think that's because Chris Jones made changes there. IIRC these library codes are main source of various warnings. So far I prefered to simply ignore them.

Current solution division is following. The former AGS code is split in 3 parts (folders).
- Common: these files are shared between Engine and Editor, hence - "common". For Windows version I am making a static library of these files, and link to both Engine and Editor. This helps to keep their organization stricter. Other platforms cannot build Editor, and therefore they just take Common & Engine folders and build (almost) everything from them as a single whole.
- Compiler: a split, containing code for script compiler. This belongs to Editor only, Engine does not use these files at all.
- Engine: Engine code, not used by Editor, at all.

Compiler code was not changed much (maybe some reorganization was made only, I barely remember now). Common & Engine folders are those that received most intense changes over last year.
A word of warning: changing Common files must be done with certain care, for they are shared with Editor. JJS and BigMC mostly disregarded this in the past, focusing on the engine and its ports, so I made myself responsible for maintaining Editor's compatibility with Common code. Simply put: if something is changed in Common, someone must test how Editor builds, and make changes there too, if they are needed.

Quick info about Editor. It consists of roughly of two parts: the Editor itself - a gui application, written in C#, and AGS.Native.dll, written in managed C++. The AGS.Native is a bridge between shared engine code and Editor. I try not to touch Editor itself, all the changes we made there were 99% in the AGS.Native. As for Editor development, tzachs is the one who managed it after CJ, and also Alan v. Drake made his Draconian Edition, which earned popularity in the community already. All tzach's changes are already merged in our repo, and I really hope to see Alan's patches there at some point (he did some fixes to engine too), maybe after we complete this Linux release for Wadget Eye.

Anyway, back to the engine.
As being said, it uses the code from Common and Engine folder. I tried to organize files ther in a number of subdirectories with certain meaning:

- ac - I tried to put there all general code, like one related to main game logic, and in-game objects (rooms, characters, etc). The name "AC" is rather a legacy issue, it was the name of the largest *.cpp file in the original source . "AC" stands for "Adventure Creator" - the old name for AGS.
- api - recently created dir, suggested for plugin api.
- core - something you can think as the AGS axle, the initial point; contains only some newer stuff; my vision is that it should contain the engine class that manages loading, starting and running the game.
- debug - debugging stuff, mostly for output.
- device - I thought to put device-managing code there, but currently contains only mouse handler.
- font - font rendering classes.
- gfx - lower-level graphic classes and functions, like bitmap, graphic renderers and filters.
- gui - code related to in-game gui classes.
- libinclude and libsrc - the parts of Allegro and some other libs. IIRC, these rest untouched. Also, source of many compiler warnings .
- obsolete - the code, extracted from engine, that is not really used anywhere. I suspect it was used before, but have no idea how long ago. Maybe, it originates to very first versions of Adventure Creator. I left it just in case, since we really do not have sources for earlier AGS versions.
- main - the main entry point, engine initialization code, and the main loop. Something I wished to be moved to the "core" directory, but not until it is refactored.
- media - code, related to lower-level audio and video handling. Not so low-level, as direct data handling (that's what sound libraries are for), but rather relative to general game logic.
- platform - Platform Driver class, that contains some platform-specific implementations for various things. To be frank, I think it should be split eventually, for it deals with too varied stuff to keep in one class.
- plugin - plugin handling code + main plugin api header.
- resource - icons, and resource declarations for setup program (WinAPI related). I am not sure that everything there is actually needed.
- script - script interpreter stuff.
- setup - embedded setup program, written in WinAPI, therefore non-portable.
- test - simple test units for newly added utility classes.
- util - various utility classes and helper functions: strings, streams, paths, files, library loading, threads, mutex and similar kind.



Current development focuses following tasks:
1. Porting. That's JJS & BigMC's turf, they were working on that before I ever joined.
2. Backwards-compatibility. Vanilla engine did break compatibility with previous versions of the games many times. JJS was making compatibility fixes over time, and now many older games are playable on this engine. In brief, the game data contains version number, which allows to identify data formats and game behavior. Forks are inserted into the code, that switch between logic depending on game's version. This is on-going work, because there's yet many things we do not know about old versions.
3. Refactoring. General ideas and aims are:
a) OOP (where sane);
b) Platform-agnostic interfaces: game logic should not know and should not worry about which OS it runs. If it needs to create a file, it does not use "fopen", it uses our own function OpenFile, and that function does the rest.
c) Library-agnostic interfaces: game logic should not know and should not worry about which low-level libs is there. This is primarily to allow switching from Allegro 4 at some point (either to Allegro 5, or anything else). Originally allegro 4 functions and types were embedded into engine too much. This work is partially done with introduction of Bitmap interface, which hid many calls to Allegro drawing, but still there's much to do.
d) Allowing to change data classes without ruining backwards-compatibility. This is something so extreme that deserves its own paragraph...


What should be kept in mind ALL THE TIME YOU ARE WORKING WITH THE ENGINE.
Do not change data structures unless you are aware about possible consequences and 100% sure they are no longer an issue. DO NOT CHANGE DATA STRUCTURES AT ALL, before we estimated it is safe thing to do.

Problem number one: loading game data / saved games. That's pretty obvious, and rather simple. Also, it is not a big deal any longer, because we have rewritten loading/saving functions. Just remember: if you make changes to data, insert a fork for switching between old version and new version when loading from file (and saving back).

Problem number two: scripts. Scripts in vanilla AGS were made in such a way that they refered to structure members using relative offsets. Those offsets WERE HARD-CODED into script. Meaning, if you change object's size, or change an order of variables, you'll break the scripts.
This problem is nearly solved, by rewriting script interpreter, but there are still few things to be done. So, for now - if you want to change any (original) data structure - ask first, for that may be unsafe.

Problem number three: plugin interface. It included declarations of several structs, therefore restricting their change (except for appending data to the end). This is a blocking issue, and we see only one way to solve it: remake plugin interface and rewrite all AGS plugins ever used by any game . Thankfully there are not much popular plugins around, so, regardless of how crazy this might sound, it may be viable plan. However, nothing is done yet, and we try to keep the code compatible with existing plugins.



What else to say?
Here's our old discussion about Coding Convention: http://www.adventuregamestudio.co.uk/forums/index.php?topic=46329.0
This is a "proposal draft", as it sais, and, frankly, I think only JJS follows it to the letter (I tried too, but used "wrong" identation by mistake). Probably this should be discussed again soon. There were plans to apply autoformatting to AGS code, after we finalize this convention.


I am sure you have a lot of questions, so please ask. It would be simplier to answer certain questions, instead of trying to describe everything we know about the engine.

PS. Also you may want to check this: http://www.adventuregamestudio.co.uk/forums/index.php?topic=46427.0
and this: http://www.adventuregamestudio.co.uk/forums/index.php?topic=46475.0

[Crimson Wizard]

Few Wiki pages we wrote several months ago. Looking at these makes me realize I understand engine much better today, but still, some info that may help to start exploring it:

https://github.com/adventuregamestudio/ags/wiki/_pages

[Sslaxx]

Taking a look through it all now...

[s_d]

Wow, CW, thanks!  This is hugely instructive.  I found myself relying heavily on my tagfile, since it seemed that occasionally things were organized in totally random fashion... and other times, very sensibly and contiguous.  I observed that it must be for some legacy reason, but I'm grateful to know the whole backstory.   

[s_d]

I'm working with screen and game resolution issues, currently (more on that over in the Linux port thread).  I've made some progress while investigating this, i.e., a game can now start up in desktop resolution without probing, as though gfx mode list is empty.  It is, of course, very experimental and I broke stuff while learning how it works, but I've done enough to have specific questions to ask 

First thing I'm having difficulty figuring out is how during initialization the game's intended resolution mode is discovered.  I see where "GameSetupStruct.default_resolution" is read from the game data, and then later, this value is switched on to produce usetup and scrnwid/scrnhit values... but somehow,  both 320x200 and 320x240 games have default_resolution of 2.  Yet, sometime later, the correct resolution is divined.  Does anyone have insight into this magic sauce?

EDIT:  Never mind, I think I figured this out.  Sorry to bother everyone...

[Crimson Wizard]

EDIT:  Never mind, I think I figured this out.  Sorry to bother everyone...

Maybe you can tell what you have figured out then? I don't know the answer too.

[s_d]

Haha, yeah it gets bit nightmarish.  Ok, take a deep breath...

Code: CPP

Engine/main/engine.cpp:initialize_engine(int argc,char*argv[])
    Engine/main/engine.cpp:engine_load_game_data()
        Engine/main/game_file.cpp:load_game_file()
            Engine/main/game_file.cpp:ReadGameSetupStructBase_Aligned(Stream *in)
                Common/ac/gamesetupstructbase.cpp:GameSetupStructBase::ReadFromFile(Stream *in)

That's the call stack from loading the main game data file to populating game.default_resolution a member of the global GameSetupStruct.  But what is it exactly?

Is this mysterious default_resolution a scalar factor?  A multiplier?  Divisor?  Not so much...

Here's a nice comment which is helpful:

Common/ac/gamesetupstructbase.h:struct GameSetupStructBase

Code: CPP

int32             default_resolution; // 0=undefined, 1=320x200, 2=320x240, 3=640x400 etc

This value will be refactored as an enum, for greater clarity.  There's no good reason to leave it a mysterious unknown quantity.  Of course, I won't do that until after I'm quite certain that the Editor folks are on board with the change;  this is deep in the heart of Common code, where it is deceptively easy to misstep and break the universe according to the very helpful comments from the brilliant and legendary IKM.  I might have to thank that guy someday 

At any rate, I believe an enum to be completely harmless, since it compiles down identically to current implementation (assuming we take care to start the enum at 0), hence, it will also serialize identically.

Next up is what is done with GameSetupStruct.default_resolution...

Engine/main/graphics_mode.cpp:engine_init_screen_settings()

Spoiler

Code: CPP

    usetup.base_width = 320;
    usetup.base_height = 200;

    if (game.default_resolution >= 5)
    {   
        if (game.default_resolution >= 6)
        {   
            // 1024x768
            usetup.base_width = 512;
            usetup.base_height = 384;
        }   
        else
        {   
            // 800x600
            usetup.base_width = 400;
            usetup.base_height = 300;
        }   
        // don't allow letterbox mode
        game.options[OPT_LETTERBOX] = 0;
        force_letterbox = 0;
        scrnwid = usetup.base_width * 2;
        scrnhit = usetup.base_height * 2;
        wtext_multiply = 2;
    }   
    else if ((game.default_resolution == 4) ||
        (game.default_resolution == 3)) 
    {   
        scrnwid = 640;
        scrnhit = 400;
        wtext_multiply = 2;
    }
    else if ((game.default_resolution == 2) ||
        (game.default_resolution == 1))
    {
        scrnwid = 320;
        scrnhit = 200;
        wtext_multiply = 1;
    }
    else
    {
        scrnwid = usetup.base_width;
        scrnhit = usetup.base_height;
        wtext_multiply = 1;
    }

[close]

So, that's a big part of it.  Here some defaults get set.  Through init_gfx_mode(), switch_to_graphics_mode(), and IGraphicsDriver->Init() these values get converted to adapt for widescreen monitors (where padding gets added to the drawing surface on the top and bottom, so that 320x240 is pillarboxed and reaches all the way to the top and bottom, but 320x200 has letterboxing and pillarboxing).

The problem I'm running into is that when I skip a lot of this stuff (to hard-code the empty mode list) the game gets painted middle of the top of the screen, some drawing artifacts occur on 320x240 games, and the mouse area is weirdly affected. In particular, the mouse cursor stops on the right or bottom sides of the image, but a "virtual" cursor continues off the side somewhere, and requires a lot of scrolling to "come back" to the game screen.  Of course, it's also comically tiny.  I'm not surprised about that stuff, as I played pretty fast and loose just to do a proof-of-concept.

Should further details of this work be discussed over in the Linux porting thread, probably? I suppose I don't want to distract too much from generic deep engine-sources discussion.  There is surely so very much more to work on and to understand than one specific graphics mode bug in one particular port!