AGS: Let's start over?

[Calin Leafshade]

(Mod note: This discussion was split from this thread. â€" Snarky)

As I learnt the existing code better and about problems that were required to be fixed, I was coming to the thought that AGS is simply not suited for upgrade that everyone wanted. It appears to me that although it reached certain state with existing features and capabilities, it took more than 10 years to evolve into what you have now with the work of one man (Chris Jones), and it will take too much time to evolve further to the next "level" to keep up with the demands of the modern day game developers and game players.
Perhaps it could be better to start a new engine, back couple of years ago, built with completely different design in mind, to replace AGS, but it's too late for me to try that now, especially not alone, especially while other engines and frameworks appear - this makes me feel that my work is getting useless in the perspective.

I know my opinions on this stuff are rarely popular but I agree with CW on the matter of AGSs updateability.

I believe AGS is not suitable in it's current state to be brought into the 21st century for two main reasons.

1) the graphics stack is designed primarily to be software based. This is a consequence of it's reliance on allegro. The hardware accelerated support is very much tacked on the end and that's just not realistic for a modern engine and it's only going to get worse. It would be difficult to fix this because a lot of the API relies on direct access to the surfaces (DrawingSurface support for instance simply cannot be moved to a hardware accelerated system without breaking backwards compatibility)

2) The scripting language is slow and archaic and it would require a huge overhaul to fix. *I know, I know* I've been on and on about this for a long time so I will leave it at that.

However, I see no reason why we can't begin AGS 4. The work CW and his team did on 3.3 was outstanding and it fixes most of the trivial issues people were having like alpha support and other resolutions (in the other branch) so I think it would be perfectly reasonable to leave AGS 3 behind at this point and consider it as done as it can be in it's current incarnation.

In a lot of ways that's kinda sad but I think it's also optimistic to draw a line under something and start again as a community.

I know I speak for everyone when I say that I hope CW would consider being a part of that.

[Monsieur OUXX]

As several people said, sadness is the first feeling here. Thanks a lot for everything you did. I understand what you said about feeling that the task is too big for some unreachable result. But that makes your achievement even bigger: as someone else said, without you we'd still be waiting for AGS 3.3

thanks a lot to the other contributors too -- we don't forget you just because CW is taking a well deserved break. Contributors to the Editor, to the engine, as wel as legendary module and plugins writers.

An idea: couldn't we initiate a transition to a more modern architecture by developing Editor and Engine plugins that would fake AGS' current features, but using the new architecture behind the scenes? Those plugins could be enabled or disabled, in the fashion of hardware-acceleration: "I want to use the old AGS code" or "I want to use the new code". This way, the transition could be done little by little, in 5 to 10 years (let's look far ahead) the code would slowly become less relevant as more and more people switch to the new one.

[Crimson Wizard]

An idea: couldn't we initiate a transition to a more modern architecture by developing Editor and Engine plugins that would fake AGS' current features, but using the new architecture behind the scenes? Those plugins could be enabled or disabled, in the fashion of hardware-acceleration: "I want to use the old AGS code" or "I want to use the new code". This way, the transition could be done little by little, in 5 to 10 years (let's look far ahead) the code would slowly become less relevant as more and more people switch to the new one.

Recently I had a small discussion with Alyssa Milburn, the author of ScummVM AGS port, and she expressed a thought, somewhat similar to the one which occasionally came to me once: would not that be better if the ScummVM version would become the leading interpreter for running original AGS games, while interested AGSers could make a completely new engine, free of old ties, not needing to be backwards compatible?
Who knows, maybe that's what we should have considered to do a while ago...

[xlynx]

Recently I had a small discussion with Alyssa Milburn, the author of ScummVM AGS port, and she expressed a thought, somewhat similar to the one which occasionally came to me once: would not that be better if the ScummVM version would become the leading interpreter for running original AGS games, while interested AGSers could make a completely new engine, free of old ties, not needing to be backwards compatible?
Who knows, maybe that's what we should have considered to do a while ago...

As an outsider to the community looking in from time to time, I like the idea of a complete recode, dropping backward compatibility. It has become painfully obvious even to end users on Steam forums etc. that this is needed, and SkyGoblin's blog is inspiring, even if a full suite replacing AGS would require considerably more effort. Also, many big projects (Apache comes to mind?) have done total rewrites without splitting their communities.

As for the renewed interest from the SCUMMVM project, I see this as a big win for end users in terms of retaining (and creating) accessibility to all our beloved AGS games. I had read some rather long threads here which disappointingly concluded with "not possible while still the active branch under the AGS project", so this could be really exciting. It is the future of AGS 3.x, I'm almost certain.

Lastly I want to thank you for your efforts, and I'm sorry it has been such a large source of distress. I'm sure any reasonable person here would happily encourage you to "do what you need to do". I feel that even just by giving your blessing to ideas such as the quote above, you are still contributing far into the future.

[Snarky]

Personally, I don't think a complete rewrite is a good idea. Slapping together a simple engine may not be too difficult, but AGS is a huge beast, built to support a variety of game types and configurations, with thousands of little tweaks, settings and special-case fixes, and a fairly sophisticated editor/project management app. It took some 15 years of development (plus a couple of years in hiatus) to reach its current state. Think about it: just writing a new manual would be a big undertaking!

Also, while a project like Apache may be able to get the community to go along with a total rewrite, they have a rather healthier dev community to behind the project. As it's currently looking, total AGS rewrite would realistically be a project by two-three people (if that), and I'm not sure there are any potential candidates who have the stature within the community to convince game devs that something with a completely new API, stack, IDE and so on is still AGS.

Finally, while a complete rewrite is not always a bad idea, it usually is: it's a direction developers are often tempted to take even when it's not the smartest approach. The code you have is a mess, but the code you intend to write is pristine, perfect... However, then when it comes down to actually putting it down in text, you're back to the imperfections of the real world. Most of the time, an existing codebase can be fixed bit by bit, with some refactoring here, replacing a library there, rewriting a few individual components there. (As explained in this classic article.)

So if the graphics stack is a weakness of AGS and you know how it should be done, tackle that! You break some aspects of backwards compatibility, so be it. (If the biggest thing that breaks is DrawingSurfaces, and the new version provides a different way to achieve the things people use those for, it's probably not going to be too difficult for people who want to update to rewrite those parts of their game.) You don't need to throw away all the other parts of the engine or editor, and force game makers, players and community members to adopt an entirely new "AGS" to do that. Other changes (such as a new scripting language) can come later.

All that said, it's a do-ocracracy, and since I'm not the person to either do a rewrite or make major improvements to the current codebase, all this is just talk. Maybe I can convince some people, but it's what people who have the inclination, skills and time to get their hands dirty choose to do that matters in the end.

[Crimson Wizard]

I do not have much intent to participate in such discussion, not at the moment at least, but I want to say this all depends on the aim. If the aim is to save as much from old AGS as possible, then you replace what you must and leave the rest.
But, you see, you keep using the word "rewrite" which presumes new implementation of old ideas. I have different opinion; I think many of the concept that AGS uses: scripting language, data format, localization mechanism, game entities and their relations to each other - should be reviewed and designed in a different way.
See RickJ's opinion, for example: http://img28.imageshack.us/img28/9957/entityconcept.pdf
I am not speaking about "realistic" aim now, "realistic" is usually subjective thing, what may be realistic in one condition and with one team will be unrealistic in other case; I just express my honest opinion.
The big flaw of my own work was that I didn't really make up a global aim... There should be one. Are you aiming on improving existing program? Or rewriting existing one which would do the same just better? Are you going to keep backwards compatibility?
Or do you want to write a completely new program that, optionally, may support old behavior?
That's the real question.


EDIT: Also, I am not sure I understand why anyone would need to persuade game devs that new thing "is still AGS". Is there a real need to keep ties to the old program? Original AGS won't go anywhere, it will stay and people may use it if they want. It may even be improved separately (by adding some features).

[Snarky]

The reason to keep the ties to the old program is... well, this is the AGS forums, right? If someone wants to go off and build a new engine that's not AGS, fair deuce to them, but that's really not our concern here. This is about how to develop AGS further.

AGS is not just an engine; it is, perhaps more importantly, a community. That means the engine is not set in stone, but it also means that you have to get the community to buy into any new thing you want to call AGS. What we have is in many ways a proven success: one can criticize aspects of the design, but this is the engine that has become so popular and that has been used to make so many great games. If you were to change everything, for example to a compiler that just takes lua scripts (written in some arbitrary editor, using some completely different API) and a bunch of resource folders (e.g. organized according to RickJ's proposed structure), would you be able to convince the community that this was AGS and that they should make the switch? I doubt it.

I agree that many of the things you mention need an update (I have no opinion on data format), but I would use AGS as it currently is as a starting point, and evolve it from there. I don't think "rewrite" excludes the possibility of redesigning these elements, one by one. Personally I'm not persuaded by RickJ's proposal for entity relations or your idea for localization: both seem to me too abstract, trying to be totally generalizable rather than maximally pragmatic and efficient (in the case of localization, I think you get more usefulness with 10% of the complexity just by adding in Unicode support; you can use existing ability to check the current translation from the script to switch out graphics and voice packs). Similarly, rather than RickJ's deep hierarchy of entities and stuff, I'd be more inclined to just rethink views and loops. In any case, if someone believes in these ideas, nothing's stopping them from trying them out in an experimental branch. Like I said: it's a do-ocracy.

As for backwards compatibility, we've talked about this before. There's a big range between "you load your project in the new version of the editor and it just works" (or even "you can run old games with the new engine") and "you have to throw away your game-in-development and start again". I don't think it's a big deal if a new version deprecates a few API methods, or changes the scripting language syntax slightly. Obviously, if it required you to recreate a bunch of game resources (for example because it no longer had a notion of a "room"), that would be a higher hurdle, but still nowhere near as high as switching to a completely different engine.

[abstauber]

Just something to think about:
What happend to all those other new engines (looking at Dage and XAGE for example). They also had dedicated and talented programmers.
Why did Wintermute 2.0 never happen (yet)?
Those big OSS projects have paid software engineers, coders etc.
I highly doubt that anyone here can invest the time and energy to create a new engine with even 10% of AGS's current features.
Just try to calculate, how many hours we would need to at least get the most basic adventure engine.
AGS is great (community aside), because it came such a long way. Because somebody kept it going for such a long time added features after features.
I completely agree with Snarky here.

[Crimson Wizard]

both seem to me too abstract, trying to be totally generalizable rather than maximally pragmatic and efficient

Generic base makes things flexible, you can then shape it into many kinds of presets. It also allows people to find their own ways to do stuff. It also makes it easier to extend (which in many cases may not require adding new things, but rather reusing existing ones in the new way).

Just try to calculate, how many hours we would need to at least get the most basic adventure engine.

Maybe not so much. Depends on which codebase do you use (existing libraries). Did not Calin wrote his preliminary Lua framework in two weeks?
(I am not making any far going assumptions here, just giving an example)

AGS is great (community aside), because it came such a long way. Because somebody kept it going for such a long time added features after features.

And every simple feature takes more and more time to be added, because its like digging through a pile to reach a place where you can hack it in. And then realizing that there are conflicts with other features you did not see before, because they did not seem to have any obvious connection.

[Snarky]

both seem to me too abstract, trying to be totally generalizable rather than maximally pragmatic and efficient

Generic base makes things flexible, you can then shape it into many kinds of presets. It also allows people to find their own ways to do stuff. It also makes it easier to extend (which in many cases may not require adding new things, but rather reusing existing ones in the new way).

Yes, but only up to a point. Try to make things too general, and you end up with an enterprisey system where every task is obscured behind layers of abstraction. For example, you could say that the AGS concept of a "room" is not general enough. "Room" isn't the right word to describe an outdoor scene, for example. So maybe call it a "Node" instead. Additionally, you have things like menu screens, closeups, etc. that aren't really nodes in the same sense, so maybe make nodes a particular sub-type of... let's call it a "Scene". Scenes, of course, are a game Entity. Of course, you don't want to limit the room displayed to only where the player is, so make a more general property for that... And really you should be able to construct rooms â€" sorry, nodes â€" on the fly, so let's have a NodeFactory class for that...

Hooray! We've generalized AGS... Of course, in order to use it people have to wrap their heads around a bunch of abstract concepts and not-very-descriptive terms, just in order to solve what wasn't really a big deal in the first place, but pfft! it's about the elegance of the design, not the convenience of the user.

I'm caricaturing, but it's easy to drive yourself crazy by trying to generalize everything. (I'm currently deep in a database design task, and for pretty much every relationship I want to represent â€" say for example "Person A owns Car X" â€" there are edge cases and possible complications that makes it tempting to move up one level of generality: "There is some sort of relationship R between one or more People A, B, C... and one or more vehicles X, Y, Z". At some point you're no longer building a database for a particular task, just a generic database for storing anything.)

I've made my comments about the asset localization specifically in that thread.

Just try to calculate, how many hours we would need to at least get the most basic adventure engine.

Maybe not so much. Depends on which codebase do you use (existing libraries). Did not Calin wrote his preliminary Lua framework in two weeks?
(I am not making any far going assumptions here, just giving an example)

With all due respect to Calin, I think there's a huge difference between putting together a basic engine for a particular game, and a full-fledged Adventure Game Studio, complete with IDE, documentation and everything (including lots of features the developer has no personal use for, always a bottleneck when it comes to open source development). It might not take 15 years to do, sure, but you'd have a long way to go before it was comparable with AGS. As opposed to the alternative, where each thing you fix or improve makes a good, popular engine better.

And every simple feature takes more and more time to be added, because its like digging through a pile to reach a place where you can hack it in. And then realizing that there are conflicts with other features you did not see before, because they did not seem to have any obvious connection.

Well, I remember that your initial plan was to first focus on refactoring the existing code. Did you make much progress with that, or did we distract you too much with feature requests?

[Crimson Wizard]

Sorry, I see I am derailing the actual discussion to some hypothetical talk again. I'd rather stop now.

[Calin Leafshade]

I am pro-rewrite. Or rather pro-drastic-change.

The editor could probably be salvaged but AGS is old not only in terms of its technology but also in its approach.
In a lot of ways it is built from the ground up to make retro games. It was built to emulate sierra and lucas arts games and it shows in terms of its architecture and its technology.

However, I do agree with Snarky that AGS is AGS and it's conventions are there because the people in this community like them and for someone to say "let's make AGS 4 completely different" does somewhat miss the point of a community.

With that in mind I think it might be best to make a competing engine. Something that attempts to emulate AGS but in a modern way. Essentially what AGS was to the old Sierra tools or SCUMM. Not a rewrite but rather a homage to the original. It would stray from AGS's vision in a lot of ways just as AGS improved upon SCUMM and its contemporaries.

I'll have something to announce soon on this, depending on rate of progress.

EDIT: On snarky's comment that AGS is a "huge beast". I disagree. It's huge in some ways because CJ doesn't use a lot of off-the-shelf technology and so maintaining the project has become arduous. Adventure games are very simple. It's not that complicated to replicate what AGS does with modern technology.

[Joseph DiPerla]

I agree with a do over in rewriting the engine. But it doesn't need to change what AGS is necessary. For instance, I still think the new  editor should work nearly identically to how AGS 3.3 would work. Maybe some small differences would enhance it of course.

My idea would be this:
*Rewrite the engine using SDL as it is far more portable and works on more OS. As much as I hate to say it, Allegro does not seem to be what AGS needs. SCUMMVM itself is a testimony of what SDL can do and its portability.
*Develop it having LLVM in mind(I think that is what the software is called. I am at work and cant look it up now). The reason why I would use this is so that the project can also be compiled into HTML5 and Javascript so that the engine can work in our browsers. The benefit to this would be that we can then use Phonegap to compile the engine to many portable systems such Windows Phone, iOS, Android, WebOS, Symbian and Blackberry.
*For scripting, I would go with Angelscript. It is the closest language that matches AGS' original SeeR language and what it is now. I would go with that primarily. It is also highly portable. Something I thought of with AGS was that it can also include a secondary language, which would be LUA, giving the user more options and more power. I had seen this with another engine and I can't remember which one it was.


The big change for me would be coding the editor. I would like to propose using B4J(http://www.basic4ppc.com). There are a few reasons why I propose this:

1) It has a basic.net style programming language. This makes the language very similar to C# allowing many developers to participate in this, including me.
2) It is portable to Mac, Linux and Windows and basically anything that can run Java without changing any code between OS.
3) It can use Java libraries and plugins.
4) The interface can be far more attractive with less effort than C#
5) It can easily be ported to Android using Basic4PPC.
6) With small effort and libraries, it can handle all the things the original editor could and more.

If you would like me to do a mock up using B4J, I will.

This is where I think we should go with AGS personally.

[BigMc]

If something new emerges it could find its place in this forum and community even if it's not called AGS right? I think that a new engine could maybe attract more developers than AGS. AGS was open source for years now and most people who looked at the code didn't work on it for long. That's not a coincidence.

[Snarky]

I am pro-rewrite. Or rather pro-drastic-change.

The editor could probably be salvaged but AGS is old not only in terms of its technology but also in its approach.
In a lot of ways it is built from the ground up to make retro games. It was built to emulate sierra and lucas arts games and it shows in terms of its architecture and its technology.

However, I do agree with Snarky that AGS is AGS and it's conventions are there because the people in this community like them and for someone to say "let's make AGS 4 completely different" does somewhat miss the point of a community.

I'm curious about what AGS conventions you think are so out of date and only suitable for retro games, or why the idea of emulating Sierra/LEC games ties you to particularly outmoded technology or architecture. (Ummm... maybe that sounds more confrontational than it's intended. I'm genuinely curious about what parts of the AGS design/architecture/codebase you think should be ditched.)

By the way, I'm not opposed to improving or changing a lot of elements of the engine and editor: replace the graphics stack, ditch globalscript, add support for vector masks... I'm all for it, and I don't think it fundamentally goes against the AGS design principles or conventions. I just think, as a pragmatic matter, that it's better to apply these changes, even architectural changes, to the existing AGS codebase rather than build something new from scratch. (Though it's all academic unless someone's willing to actually do it.)

In other words, how do you (all of you who favor a new start) respond to the Joel on Software article?

EDIT: On snarky's comment that AGS is a "huge beast". I disagree. It's huge in some ways because CJ doesn't use a lot of off-the-shelf technology and so maintaining the project has become arduous. Adventure games are very simple. It's not that complicated to replicate what AGS does with modern technology.

Well, if that's true, couldn't you just do it? When you announced Adore, you described some of the things it couldn't do as well as AGS, but if replicating AGS is so simple, why not put in some extra effort into replicating the user-friendliness and pixel-art support? Base it off the existing AGS editor, maybe add in a more C-like scripting language for people who hate and fear change, and you'd have AGS 4.0, right? It'd come with a built-in community.

Of course if someone can in, let's say, four months go off and rebuild AGS from the ground up with all modern libraries and technology, with all the existing features and a familiar API, taking care of all the driver problems, platform issues, internationalization limitations, future-proofing... fucking go for it!

AGS was open source for years now and most people who looked at the code didn't work on it for long. That's not a coincidence.

By that same logic, if recreating everything AGS does well is so easy, why hasn't anyone done it already? At least we have an AGS 3.3 release; we don't have a competitive release of any of the "AGS Killer" engines.

Personally, I'll believe it when I see it.

[Crimson Wizard]

I want to say that, in my opinion, the choice of existing libraries and or framework is essential. If I learnt something from past few years is that one should stay focused on one level of development at a time. You should not develop utility libraries when your task is to make game engine: this distracts you, steals your time and leaves you hugely dissapointed. Besides, every level requires certain thinking. Low-level routines must be super-efficient, but when you write higher layer of abstraction you want to forget about internal details and just write an engine logic, and if you work on two levels at once you get confused, pay too much attention to unimportant details and miss important ones.
So, don't spend too much time on things that are already done by other people. If you are not 100% sure, keep project modular, so that you can switch some parts in the future years if they prove to be not good enough.

In other words, how do you (all of you who favor a new start) respond to the Joel on Software article?

Aahhh. I remember this article. And I disagree . I've seen too much bad code, I think.
I think every program has a certain lifespan, a limit when it is no longer fun (in general meaning) to maintain, support and improve. The better it is written, the longer lifespan is. But when you reached the limit - you reached the limit. And it is silly to pretend that you will be able to change it further. You might need a different approach, a different inner structure.
It's like building a house. If you have a 2-level house in a suburbs you maybe will build another storey on top, maybe add a shed and garage, but you'll never make a multi-storeyed condominium of it.

It's important to remember that when you start from scratch there is absolutely no reason to believe that you are going to do a better job than you did the first time. First of all, you probably don't even have the same programming team that worked on version one, so you don't actually have "more experience".

I actually LOLed at this. The company I am currently working in we have a HUGE software which was developed since mid-90-ies. It was started when some of the developers were students and basically began to learn C++. Now they plan on creating a "next-gen" software version, writing a program base from scracth, and since I work with their old code all the time I can only agree .

I'll underline, it is not really about writing better code (although if you have more experience now, you may evade some mistakes), but about better architecture, better design solutions.

And, well, I've seen examples when rewriting software from scratch made things better . One of them is AGS Editor .

[hedgefield]

It's sad to see AGS coming to a kind of 'end point', but at the same time I think the editor is at it's peak now, able to support high resolution stuff, and it's extensible should the user so desire. It's not a bad time to draw the line, I mean it will still be useful for a while to come.

On the note of starting something new, I wouldn't nescessarily recommend building an engine from scratch. I wouldn't stop anyone if that's what they would enjoy doing, but looking at what ChrisIcebox is doing with his Adventure Creator plugin for Unity, I'd almost say the smart move would be to support him in that project (if he wants that). That plugin is almost at a point where it can do everything AGS can do (albeit with bugs and caveats) plus Unity is free and compiles fine to a bunch of platforms. It's that AGS is still easier for me to use rght now otherwise I would have already switched. An argument can probably be made for building an engine that doesn't have the infinite bulk of all of Unity's features, but as for getting average Joe to jump on the adventure game bandwagon, I think you could do a lot worse than Unity + Adventure Creator.

[bicilotti]

[..] plus Unity is free and compiles fine to a bunch of platforms.

I think I recall Unity is proprietary and moreover gratis only for certain users (noncommercial + lone devs maybe?).

[Snarky]

Plus the plugin itself appears to be $70.

Being free is one of the key draws of AGS. To say we should not only rewrite the program from scratch, but give up on the whole idea of a free adventure game studio app... Speechless, I am.

[Crimson Wizard]

Also, 8-bit mode...!