Porting to Allegro 5

Started by monkey0506, Mon 16/02/2015 19:23:03

Previous topic - Next topic

monkey0506

It's been discussed before, but as time goes on it seems increasingly as though porting to Allegro 5 would get AGS where it needs to be as a modern engine.

Allegro 5 is well documented, has better support for modern graphics drivers, has built-in threading support, makes up for a lot of the things AGS has hacked into A4 over the years, has UTF-8 string support, has prebuilt libraries for modern Windows compilers (so we could finally abandon both Visual Studio 2008 and the NoNative solution)...the list goes on.

Porting to Allegro 5 will be no small task. Allegro 4 builds a queue of events which are then processed as the game loop executes. Allegro 5 is instead event-driven, which will require major logical changes to the way the engine is coded. Also, there are many cases where there is no one-to-one conversion, though theoretically everything A4 does A5 can still do.

I have taken a look at starting in on this, but I'm not sure if just diving head-first into this will produce anything meaningful.

Would it be feasible to partition off the engine code into smaller segments, so changes can more easily be reviewed, and others could work on other sections? What might be the best way of breaking this up?

Also, it would probably be useful to use a C++ wrapper layer on top of Allegro 5 for RAII garbage collection and the like. I found one called "ALX", but it doesnt seem to conform to good C++ standards (e.g., it doesn't make use of the default case in switch statements, even when that is the desired result). If needed, it would be relatively simple to write our own wrapper layer.

Thoughts?

Crimson Wizard

#1
My opinion is that the best way would be to write a new engine using cleaner concepts, then look into reimplementing/emulating some of the most wanted AGS features (possibly in a form of script modules).

Even if the aim is to make a clone of AGS, it is better to first make basic "blocks" based on Allegro 5, and re-base old AGS features on top. I do not think it will be feasible to rewrite old code by parts. It will be way too tiresome. Also, a lot of AGS features is just old garbage that no one uses anymore, yet they require extra branches in the engine code.

I would also consider to follow a tip from SkyGoblin's blog and avoid using different compilers for different platforms, and just use GCC (MinGW on Windows).

doimus

Just one big IMO, and please pardon my ignorance as this was probably discussed already, but it just seems too obvious not to try it out, when there's the idea of developing an all new engine.

Perhaps the whole AGS concept should be re-engineered into more modular approach?
Go against the core paradigm of the integrated "studio" concept and instead focus on developing "modules"?

TL;DR version: develop an Adventure Game Script language. Similar to SCUMM+interpreter combination. Or like ScummVM that looks into the future, instead of the past.


Long version:
Separate the whole AGS into three main modules/components:

1) Scripting Language - what game IS. The current AGS scripting language. The new scripting language. Whatever it is, it is (almost) set in stone and changed only on centennial entmoots in the forbidden forest. :=


2) Visual Editor - visual representation of the scripting language. Define room backgrounds, sprites, walkable areas etc... In some more refined form, this editor could allow for code-less game development (like UE4 blueprints). But in essence, all that it does is visually organizing the scripting language. It's 100% not essential.
A subset of this could be an asset packer that produces a serialized asset data for direct use with the interpreter.


3) The Game Interpreter - what makes the game run on a certain platform.
This should be heavily separated from script development, and even more separated from any particular software library on any particular platform.
Since it's all open source, it's up to particular developer to develop their own interpreter. On PC, they may use Allegro, SDL. On iOS, it's Swift. It's ScummVM everywhere else, etc.

There are benefits and drawbacks of this system.

Benefit is the smaller scope of it all. Concentrate on the scripting language itself for start and ignore the visual editor completely. Hack the interpreter in Pygame and Python for all it's worth.
Once the structure of the scripting language is set, focus on the development of "proper" interpreter for Windows written in native C++ with full-out DirectX support. After that, it's time for OSX/iOS interpreter written in Swift.
The windows interpreter works, but OSX is buggy? Nevermind, here's the AGS 4.0 language specification - write your own interpreter using C and OpenGL directly. You want to port your game to ARM Linux? Sure, same download link. But you may want to try out ScummVM interpreter first.

You don't like the current visual editor? Here's the AGS 4.0 language specification link again - write your own. You want to port the editor to you Android TV.... you get the idea.


The biggest drawback is the fragmentation an approach like this would bring.
But fragmentation is given in any open source project anyway. This could be leveraged by only focusing on the scripting language specification and not actual implementation.

Because, and here's a big IMO again, the main focus of AGS development should be: "I want my actor to do this or that" instead of "I want my game to run on xyz".
This may seem ridiculous at first, but once there's a clear separation between a script and an interpreter; and there's clear language definition that negates reverse-engineering; and once it runs at least somewhere somehow - the implementation of it for running somewhere else will come naturally. 


Crimson Wizard

#3
Thank you, doimus; all you say is true , but I am still not sure this thread was started to talk about designing of a new engine. Monkey named it "porting to Allegro 5" not "inventing new engine".
I am afraid your post will deviate the discussion, as it happened too many times in the past.

I was here only to put my two thoughts on a clearly stated technical question about replacing backend library (which I did above). Before saying anything else I would like to be sure that I understand the purpose of this thread right.

doimus

Sorry, kind of got derailed off the track by my own train of thought....

The initial thought probably was something in the line of your reply - ie. making AGS less the hostage of whatever software library there might be.
Oh well, I seem to be in the revolutionary mood today (roll), sorry for the misplaced rant again.


monkey0506

As much as I've proven myself to be an AGS fanboi over the years, it speaks to the ease of use and power of the engine that AGS has hundreds of active users. Disenfranchising them all for some new ideal of what a modern game engine should be seems wrong to me (especially some meta specification that "someone" will implement). I know that improving AGS in its current state is no simple affair, but I don't think that it's a futile endeavor.

I do think that starting from a fresh code base and reimplementing existing core features could be a viable way of moving forward with this, but the end-goal would be to make it a better version of AGS, not just make some brand new engine that couldn't even be recognized as AGS. I think that for this particular upgrade, the focus should be (almost) exclusively on rebasing the core functionality onto the new engine base, which would rely (for now) on Allegro 5. Staying with Allegro should make it simpler to reimplement the portions that currently rely on Allegro 4 instead of converting them over to something like SDL or a purely generic interface. However, I think that using an intermediary wrapper layer for A5 would be a step toward that generic backend. Once we have the main engine rebuilt on top of the Allegro wrapper layer, it will be simpler to extract the interface we need, and allow other implementations from that point I think - I feel like spending too much time focusing on a generic backend interface could kill production.... :-\

Lt. Smash

#6
Sorry, for barging in but I have to give my +1 for starting from a fresh code base and reimplementing existing core features.
People often seem to misunderstand to meaning of a fresh restart. Look at starbound. They started with a totally different codebase (not that efficient, moderate expansion capabilities...), then the game became a huge success and they reimplemented the wheel. For them it is now far more easier to implement new features and fix bugs, but for the player the game is still the very same (just better).

The same goes for AGS. Reimplementing core features in a more efficient, modular and cross-platform way will in no way change the games that will be produced, as well as the developing process for the game developers. But the games will probably get more performant, can have tons of new great features (for example: GL shaders), the scripting language will get many new capabilities and so on.

Clearly, this should be the way for AGS to go. The only true way to make AGS survive the future.

BTW Basing it on generic interface with Allegro 5.1 as a base implementation seems to be a good idea. Allegro 5.1 even supports Android.

Radiant

Quote from: monkey_05_06 on Wed 18/02/2015 23:16:13
As much as I've proven myself to be an AGS fanboi over the years, it speaks to the ease of use and power of the engine that AGS has hundreds of active users. Disenfranchising them all for some new ideal of what a modern game engine should be seems wrong to me (especially some meta specification that "someone" will implement). I know that improving AGS in its current state is no simple affair, but I don't think that it's a futile endeavor.
I completely agree.

New features and scripting cabailities can be and are being added to the current code base; and it is clearly false that the current AGS is not performant enough. None of that is worth splitting the community in half over.

[delete}

#8
Should I wait now learning AGS, e.g. until the new scripting language.

Crimson Wizard

#9
Quote from: Amy on Thu 19/02/2015 15:03:08
Should I wait now learning AGS, e.g. until the new scripting language.
You are speaking of a "new scripting language" that does not exist, nor planned, nor even explained... I don't even know what is it, and if it ever be. :undecided:
So, the simple answer, there is no reason to wait, because nothing has been decided yet.

Quote from: Amy on Thu 19/02/2015 15:03:08
Or would it be a useful idea to get familiar with Allegro (whatever this is...).
Allegro is the low-level programming library that the users of AGS have no access to. You may learn it if you want to write game applications on C/C++ language.

[delete}

#10
Thanks for your quick answer, appreciated!

monkey0506

Quote from: Radiant on Thu 19/02/2015 14:54:08New features and scripting cabailities can be and are being added to the current code base; and it is clearly false that the current AGS is not performant enough. None of that is worth splitting the community in half over.

To clarify, I think what is being most strongly suggested here (and I'm in agreement with) is to create a "new" engine from scratch (totally blank code project) to implement the backend features (graphics, input, audio, etc.) that will be needed to get things off the ground. The benefit behind this is replacing the Allegro 4 backend with a cleaner, more abstracted interface to using Allegro 5 (which in turn has better cross-platform support, better Android support, etc., etc.). Once the core backend features are implemented, then existing features will be reimplemented in a cleaner way - this preserves the features of AGS (ease of use, functionality, etc.), but will allow us to clean up the code base in a modular kind of way. It's not about reinventing AGS, but trying to replace Allegro 4 in-place in the current code would be a lot more difficult to accomplish.

Work could continue on the current version of AGS while the new backend is being constructed, and thus any new features would be maintained as they are implemented into the "new" engine. This all will take a lot of time and effort, but it could be done in way that the final transition for users (devs) would be totally seamless.

Snarky

I'm a bit confused by you calling this a "new engine." To me, the "engine" is primarily the core game-managing logic, which I don't suppose Allegro is much involved in. What it sounds like you're really talking about is the renderer or IO, which should ideally be a separate component, even if AGS doesn't isolate them properly.

Anyway, so the idea is that once the Allegro-relevant architecture (renderer) is in place, you can take the current AGS code, rip out all the old Allegro calls, and replace them with calls to your new abstract interface (hooked up to Allegro 5), probably with some rewriting of messy parts of the AGS code? (But not rewriting all of the rest of AGS at this time.)

Sure, if that's what you think is the best way to refactor the graphics/rendering code, go for it!

Crimson Wizard

#13
Well, my main suggestion would be to make engine more generic by introducing interfaces of... don't know how to call them... subsystems? major features? - that would let you add more variants of implementation later.

Two examples:
1) The "object constructor" interface, used for creating game objects and assigning their initial properties. Implementations could be: a) "default AGS binary data format constructor", which assigns parameters from binary stream, and b) "property map constructor", which assigns data from dictionaries, which it turn could be parsed from human-readable XML/JSON/text files.

When the game is loaded, the constructor is created depending on some setting, but the initialization routine does not care what kind of constructor it is, so it uses them same way.

2) The "pathfinder" interface, used to navigate characters in room. Implementations could be: a) "8-bit mask pathfinder", which uses traditional AGS area masks, and b) "vector pathfinder", which uses polygons.

When the room is loaded, the proper pathfinder is created depending on which walkable areas are used in the room. Again, the engine treats them uniformly, just telling "calculate route from point A to point B".

And so forth.

Snarky

That sounds like a good idea, but is it part of a move to Allegro 5?

Crimson Wizard

#15
Quote from: Snarky on Thu 19/02/2015 21:27:02
That sounds like a good idea, but is it part of a move to Allegro 5?
No :). It's just will be simplier to make in a fresh new code.
Sorry, I keep thinking about new ideas all the time, Allegro 5 is just a part in my eyes.

EDIT:
Quote from: Snarky on Thu 19/02/2015 19:45:39
I'm a bit confused by you calling this a "new engine." To me, the "engine" is primarily the core game-managing logic, which I don't suppose Allegro is much involved in. What it sounds like you're really talking about is the renderer or IO, which should ideally be a separate component, even if AGS doesn't isolate them properly.
<...>
Sure, if that's what you think is the best way to refactor the graphics/rendering code, go for it!
Oh... wait... I now understand what you mean.
As both monkey and me mentioned earlier, the core game code should be changed, because of differences in how Allegro 4 and 5 react to system events. Of course some parts from older code may be copied over then.

If it were me, I'd start from scratch. But... well... it's me :P.

Snarky

Quote from: Crimson Wizard on Thu 19/02/2015 21:30:25
As both monkey and me mentioned earlier, the core game code should be changed, because of differences in how Allegro 4 and 5 react to system events. Of course some parts from older code may be copied over then.

But ideally the core engine shouldn't be that closely linked to the specifics of the renderer implementation. Conceptually, you should be able to have an Interface, or set of Interfaces, that just define the functionality you need, and then an Allegro-specific implementation that deals with those details. (Just like you're proposing for the various constructors, etc.) If you're rewriting the renderer from scratch, that's what you would do, right?

So the changes to the "core game code" should really be to rip out anything that has to do with Allegro. Or am I missing something? Are there "core" engine features that depend directly on Allegro?

I still think you're better off putting the existing AGS codebase (with the necessary changes: if some individual parts are easier to redo from scratch, sure!) on top of this new backend than trying to recreate it piece by piece. In the one case, you have an incrementally better AGS that you can keep fixing and improving, in the other you have parts of an engine that you have to keep adding to in the hope of one day replicating everything AGS can already do.

But we've had this argument before, and it's probably a waste of energy to go over it again. I'll leave you to sort it out.

Radiant

Quote from: monkey_05_06 on Thu 19/02/2015 19:25:18
To clarify, I think what is being most strongly suggested here (and I'm in agreement with) is to create a "new" engine from scratch (totally blank code project) to implement the backend features (graphics, input, audio, etc.) that will be needed to get things off the ground. The benefit behind this is replacing the Allegro 4 backend with a cleaner, more abstracted interface to using Allegro 5
Yes, I agree that's a good idea (I also agree with Snark in that I would call it a new renderer, not a new engine). I get the impression that Smash and possibly Doimus basically want to replace AGS with something new and different (like the forum seems to discuss every couple months), and I'm really not agreeing with that.

Crimson Wizard

I don't want to make something really different, I would like to make something more generic inside, and have AGS on top as a variant of many possibilities.
Hope I make sense. :/

monkey0506

#19
When I said:

Quote"new" engine

I was, of course, referring to replacing the "renderer", as you have said, Snarky (in so far as you mean Allegro as the renderer, which also handles audio, user input, etc.). But the problem is that the engine as it exists today is very much hard-coded around (and thus intrinsically tied to) Allegro 4. I'm not talking about rewriting the entire engine itself from scratch. What I'm saying is that creating a new backend interface (for now, based around Allegro 5, but with generics and abstraction in mind) from scratch is the best starting point to break the engine's ties to Allegro 4 (and eventually, Allegro in general). From that point, what I meant by "reimplementing" things was to pull in the existing engine code except for anything related to Allegro, which would be modified at that point to utilize the new backend interface.

Edit: Actually, reading back over it, I think you (Snarky) pretty much have the right idea. Bear in mind though that the engine isn't going to run on both Allegro 4 and 5 simultaneously, so this particular change can't really be done in an incremental way while preserving the full functionality of the engine at every step (functionality will have to be incrementally restored). Once the engine no longer references Allegro 4, any code that has not yet been updated won't compile. This is why I was saying that the code would have to be pulled back in or "reimplemented" in this engine with the new backend.

SMF spam blocked by CleanTalk