Jibble

Author Topic: The future of the AGS engine: STALEMATE!  (Read 32249 times)

monkey0506

  • SEND PIZZA.
    • Best Innovation Award Winner 2017, for his work to help AGS games reach the widest possible audience - through popular distribution platforms (Steam, Galaxy) as well as other operating systems (Android, Linux)
    • monkey0506 worked on one or more games that won an AGS Award!
    •  
    • monkey0506 worked on one or more games that was nominated for an AGS Award!
The future of the AGS engine: STALEMATE!
« on: 23 Jul 2011, 11:10 »
I've been perusing the engine source and looking to see what I can do to try and contribute here, and I have some questions on the technical side of this, specifically for the C++ programmers who know more about it than I do.

First off then, a bit of background on my own knowledge. The furthest I ever really got with C++ was in the book I had for "beginners" I got up to chapter 6, which included an exercise to write your own custom string class. I ended up getting stuck trying to figure out how to program a sprintf-style function because I couldn't for the life of me figure out a way of implicitly casting my type to a POD-type in order to pass it through "...". If you know, well good for you, but I don't even have the source files any more because that was like 5 years ago. :P

I also once did a Windows C++ tutorial which was in the style of designing an MDI text editor, with which I got as far as trying to figure out how to set a dirty flag for each document (which wasn't covered in the tutorial). I asked some questions on the appropriate events that should have been raised, and did actually much later get an answer, but amazingly enough my incredibly awesome text editor completely failed to see the light of day.

So, I do have experience in using C++, even above and beyond the call of duty given out by the tutorials and included exercises, but in my ambition I never really did much with it. AGS already existed, so what more did I need? :=

Anyway, enough back-story. I just wanted to establish that although I'm not well versed in it, I do have functional knowledge of how C++ works and how to program in it and such. But looking over the engine source, which as CJ himself put it, is "a mess", I have been wondering about some things:

- What, if anything, is the advantage of using char*s vs. the std::string class? Presumably it's a bit faster since there's less overhead and direct memory access and such, but is there actually sufficient overhead with the string class to make it non-viable for a project such as this? By that I mean, in refactoring and cleaning up the code, would one be totally remiss to try and replace the usage of char*s with the more user-friendly strings? If so, why?

- Is there any technical reasoning to the programming style that seems prevalent among C++ programmers? Having been working with the editor code and trying to develop a plugin (when I'm not working directly in AGScript for modules and things), I feel I've become a bit spoiled really. Regardless of how some people may feel about it, the OO-model is fantastic for organizational purposes. Is there any reasoning why refactoring the engine source into something more OO would be a bad thing? I don't necessarily mean that everything has to be bound inside of a class, but at least for the organization of grouping functions together, this would be a major step in making the code more readable. Not to mention the fact that for those who are accustomed to languages like C#, it would make the transition between looking into the editor code and the engine code much easier.

- A lot of static limits could be removed by using one of the classes provided such as std::vector, but some of the file structures are currently dependent on the static sizes for the way they encrypt and/or serialize their data. Is there an easy or "simple enough" way to go about rewriting these functions so as to essentially produce the same result without actually locking ourselves into static sizes? I believe the general consensus is that it would be in our best interests moving forward to start with a clean codebase versus a "compatible" one (in reference and relation to the prior versions of the engine), so non-compatibility with the existing functions isn't so much a problem, but presumably the basic gist of what's already being done would be preferable to just rewriting the thing entirely with something new.

Considering this is more about the technical side of programming in C++ than anything strictly specific to AGS directly, I thought that this would probably be the most appropriate forum. Feel free to disagree and move it around wherever you like if you want though. :=

In any case, that's pretty much all I have for now. I'm just trying to get a feel for how other people feel about this. I've done some work in refactoring some of the existing files into OO classes and trying to give it something of a feel more reminiscent of C#, but I'm not sure if everyone would consider that the best thing to do. So please contribute any feedback that would be appropriate as I'm trying to get a feel for how I should be going about all this. Admittedly, the engine source is kind of a huge undertaking, but if we could all reach an agreement on the technical provisions of coding style and conventions, then we could progress forward with making the necessary changes to clean up the code into a consistent state. ;)

Also something to consider is any implications that might be prevalent across various platforms. I have zero experience in cross-platform coding, so I'm not personally aware of any of the caveats or complications that may need to be handled to try and make porting the engine as simple as possible.

Thanks for any information or thoughts!
« Last Edit: 14 Mar 2012, 18:08 by monkey_05_06 »

Ghost

  • Guest
Re: Regarding C++ and the state of the engine source.
« Reply #1 on: 23 Jul 2011, 11:48 »
- Is there any technical reasoning to the programming style that seems prevalent among C++ programmers? Having been working with the editor code and trying to develop a plugin (when I'm not working directly in AGScript for modules and things), I feel I've become a bit spoiled really. Regardless of how some people may feel about it, the OO-model is fantastic for organizational purposes. Is there any reasoning why refactoring the engine source into something more OO would be a bad thing? I don't necessarily mean that everything has to be bound inside of a class, but at least for the organization of grouping functions together, this would be a major step in making the code more readable. Not to mention the fact that for those who are accustomed to languages like C#, it would make the transition between looking into the editor code and the engine code much easier.

Seconding that. Of course it will forever be CJ's life story how the AGS source developed, but I'd guess that it begun mainly procedural, and at some point (when OO became the most viable option) the program was already very huge, and thus never got "fully rewritten for those pesky OO rules". Why break something apart that works?
And as a mere user of AGS I really don't care much about the underlying code, as long as the product does what I want. But I agree (from limited personal experience) that OO is great for organising, re-using code and making things more readable (opening the source the first time was a bit like facing the wall of text in Netherhood). Would take a very commited team to re-factor AGS, though.
« Last Edit: 23 Jul 2011, 11:50 by Ghost »

Wyz

  • anno 1986
    • I can help with making music
    • I can help with story design
    • I can help with translating
    • I can help with voice acting
    • I can help with web design
    • Wyz worked on one or more games that won an AGS Award!
    •  
    • Wyz worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #2 on: 23 Jul 2011, 11:51 »
Quote from: monkey
- What, if anything, is the advantage of using char*s vs. the std::string class? Presumably it's a bit faster since there's less overhead and direct memory access and such, but is there actually sufficient overhead with the string class to make it non-viable for a project such as this? By that I mean, in refactoring and cleaning up the code, would one be totally remiss to try and replace the usage of char*s with the more user-friendly strings? If so, why?
Well first off, the engine started out in C (correct me if I'm wrong) and good ol' C does not have fancy stuff like objects and templates. The string class has quite some overhead but not that much it would drastically increase memory consumption (still if you want to minimize things you should use the cstring for any string that is unlikely to ever change). But the real reason would be: It's already there, it has always been there and if you want to change it that will be a lot of work.

Quote from: monkey
- Is there any technical reasoning to the programming style that seems prevalent among C++ programmers? Having been working with the editor code and trying to develop a plugin (when I'm not working directly in AGScript for modules and things), I feel I've become a bit spoiled really. Regardless of how some people may feel about it, the OO-model is fantastic for organizational purposes. Is there any reasoning why refactoring the engine source into something more OO would be a bad thing? I don't necessarily mean that everything has to be bound inside of a class, but at least for the organization of grouping functions together, this would be a major step in making the code more readable. Not to mention the fact that for those who are accustomed to languages like C#, it would make the transition between looking into the editor code and the engine code much easier.
There is a big difference between modular programming and OO-programming. If you change everything to objects the impact will be quite extensive, that's a no-no. However C++ has introduced a lovely feature called Namespaces. This groups variables and functions together in a name space so you can refer to them using a certain name. If you combine that with modular programming (where everything is divided into modules with each a own code and header file not unlike AGS modules) you have a winning team. This would still be a lot of work but people can work on modules independent from each other as long as the headers are specified uniform. A technique you can use for that is Design by Contract.

Quote from: monkey
- A lot of static limits could be removed by using one of the classes provided such as std::vector, but some of the file structures are currently dependent on the static sizes for the way they encrypt and/or serialize their data. Is there an easy or "simple enough" way to go about rewriting these functions so as to essentially produce the same result without actually locking ourselves into static sizes? I believe the general consensus is that it would be in our best interests moving forward to start with a clean codebase versus a "compatible" one (in reference and relation to the prior versions of the engine), so non-compatibility with the existing functions isn't so much a problem, but presumably the basic gist of what's already being done would be preferable to just rewriting the thing entirely with something new.
Yes you nailed this point, this is where OO would thrive. Adding STL containers would be a good idea, and maybe some custom built datastructures. You can add the serialization also in this fashion, that would greatly simplify it.

Quote from: monkey
In any case, that's pretty much all I have for now. I'm just trying to get a feel for how other people feel about this. I've done some work in refactoring some of the existing files into OO classes and trying to give it something of a feel more reminiscent of C#, but I'm not sure if everyone would consider that the best thing to do. So please contribute any feedback that would be appropriate as I'm trying to get a feel for how I should be going about all this. Admittedly, the engine source is kind of a huge undertaking, but if we could all reach an agreement on the technical provisions of coding style and conventions, then we could progress forward with making the necessary changes to clean up the code into a consistent state. ;)
If you really want to learn C or C++ stay the fuck away from C#, It's the Avatar of the programming languages.
Well, I've said this before but nobody yet responded to it so here goes again:
We need someone to take on the project of refactoring the source. There would actually be three projects, but we're talking about the last one:
  • Maintaining support for the 2.x branch and maybe porting it to scummVM
  • Maintaining support for the 3.x branch
  • Refactoring the source code to eventually to roll out a 4.x branch.


I'm interested in helping in the latter one, I would like to take on the project myself but I don't have enough time for that. Besides I'd be more comfortable if an AGS veteran did it like yourself.

Quote from: monkey
Also something to consider is any implications that might be prevalent across various platforms. I have zero experience in cross-platform coding, so I'm not personally aware of any of the caveats or complications that may need to be handled to try and make porting the engine as simple as possible.
Platform compatibility boils down to this:
  • Using libraries that are platform independent
  • Use some level of abstraction for things like integer sizes and endianess.
« Last Edit: 23 Jul 2011, 11:57 by Wyz+ »
Life is like an adventure without the pixel hunts.

monkey0506

  • SEND PIZZA.
    • Best Innovation Award Winner 2017, for his work to help AGS games reach the widest possible audience - through popular distribution platforms (Steam, Galaxy) as well as other operating systems (Android, Linux)
    • monkey0506 worked on one or more games that won an AGS Award!
    •  
    • monkey0506 worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #3 on: 23 Jul 2011, 14:24 »
It's already there, it has always been there and if you want to change it that will be a lot of work.

Considering the vastly disjointed state, and the tremendously large size of the code, I was going under the impression that the simplest way of cleaning it up would be to basically just start with a fresh script and then reimplement the existing functions in a more organized fashion. That might be a lot more work than other methods, but if the point is to create some sense of unity, it seemed the best route to me. Feel free to explain why I'm being stupid. :P So anyway, with this route, I could just switch it out on the fly as things were being reimplemented.

If you change everything to objects the impact will be quite extensive, that's a no-no.

I'm not sure what you mean here. Is there significant difference in overhead or something that makes objects more memory intensive? I don't understand why moving code into classes would be "a no-no" as you put it. I'm aware of how namespaces work, and indeed I'm currently utilizing namespaces, classes, and structs in some of the code that I've reimplemented, but I don't see why putting things into classes now would be so problematic. As long as the calls and references were properly updated, the only downside I could see is if the overhead involved were really just drastically horrible.

Yes you nailed this point, this is where OO would thrive. Adding STL containers would be a good idea, and maybe some custom built datastructures. You can add the serialization also in this fashion, that would greatly simplify it.

Yes, the Standard Template Libraries. That's what I was thinking of but couldn't remember the name of. :P However, when you say, "you can add the serialization...in this fashion," I'm not sure exactly what you mean. Could you clarify/expand on that a bit?

If you really want to learn C or C++ stay the fuck away from C#, It's the Avatar of the programming languages.

Do you mean James Cameron's or the Airbender kid? Honestly I'm more of a fan of the latter. C# looks very pretty. It's very well organized, which makes it nice to program in. That's all I was getting at, is that I'm trying to improve the total lack of organization into something that would more closely reflect, say, the organization of the editor's source.

Well, I've said this before but nobody yet responded to it so here goes again:
We need someone to take on the project of refactoring the source. There would actually be three projects, but we're talking about the last one:
  • Maintaining support for the 2.x branch and maybe porting it to scummVM
  • Maintaining support for the 3.x branch
  • Refactoring the source code to eventually to roll out a 4.x branch.


I'm interested in helping in the latter one, I would like to take on the project myself but I don't have enough time for that. Besides I'd be more comfortable if an AGS veteran did it like yourself.

We don't even have public access to the 2.x source at all yet, and the only real 2.x branch support I'm aware of within the 3.x branch of the code is the backwards compatible functions (engine side, editor side also has the project import-updater for 2.72 projects). I'm not sure if that's what you meant, but since it's already in the 3.x branch I don't see how that would differ from maintaining the 3.x branch...?

Refactoring the entire engine is going to be a long and arduous task, no matter how you go about doing it. My main concern for this thread was simply coming to terms with what I need to know about C++ to make sure I'm not "doing it wrong". It would be rather silly for me to rewrite half the engine only to have someone come along and tell me, "No, that's not gonna work because.....and......." Once we can establish and agree on some basic programming conventions then we can get into the gritty details of refactoring the engine code itself. Again, as I said, my purpose here in this thread is just to figure out what I should and shouldn't be doing (and why).

Thanks for the responses so far, it's been informative.
« Last Edit: 24 Jul 2011, 11:20 by monkE3y_05_06 »

RickJ

  • fix'n one thing and break'n two ...
    • I can help with scripting
    • I can help with story design
    • RickJ worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #4 on: 24 Jul 2011, 01:31 »
Quote
- What, if anything, is the advantage of using char*s vs. the std::string class? Presumably it's a bit faster since there's less overhead and direct memory access and such, but is there actually sufficient overhead with the string class to make it non-viable for a project such as this? By that I mean, in refactoring and cleaning up the code, would one be totally remiss to try and replace the usage of char*s with the more user-friendly strings? If so, why?
This is actually more of a question of ANSI C vs C++.   In the context of something like the AGS runtime conventional wisdom is that C has a number of advantages over C++.

- ANSI C is considered more portable and is less resource intensive.  C++ implementations tend to vary from compiler to compiler much more so than C.

- ANSI C does not automatically, dynamically allocate/reallocate memory as C++ does.  The latter is fraught with potential problems when executed on unknown or resource limited target platforms.   

- Consequently runtime errors are less likely with ANSI C than with C++

- ANSI C typically has a smaller resource footprint than C++

Quote
Is there any technical reasoning to the programming style that seems prevalent among C++ programmers? Having been working with the editor code and trying to develop a plugin (when I'm not working directly in AGScript for modules and things), I feel I've become a bit spoiled really. Regardless of how some people may feel about it, the OO-model is fantastic for organizational purposes. Is there any reasoning why refactoring the engine source into something more OO would be a bad thing? I don't necessarily mean that everything has to be bound inside of a class, but at least for the organization of grouping functions together, this would be a major step in making the code more readable. Not to mention the fact that for those who are accustomed to languages like C#, it would make the transition between looking into the editor code and the engine code much easier.
I agree with you that OO is a superior organizing principal and that it is not necessary to make use of compiler features to take advantage.  I have been doing this for decades in non-OO languages.   I think it would lead to a better quality codebase as well.

Quote
A lot of static limits could be removed by using one of the classes provided such as std::vector, but some of the file structures are currently dependent on the static sizes for the way they encrypt and/or serialize their data. Is there an easy or "simple enough" way to go about rewriting these functions so as to essentially produce the same result without actually locking ourselves into static sizes? I believe the general consensus is that it would be in our best interests moving forward to start with a clean codebase versus a "compatible" one (in reference and relation to the prior versions of the engine), so non-compatibility with the existing functions isn't so much a problem, but presumably the basic gist of what's already being done would be preferable to just rewriting the thing entirely with something new.
I disagree C++ classes are needed to achieve this.  All that is necessary is for the the AGS editor to pass on the number of each item used.  ANSI C could be used to allocate the required resources at startup.

Quote
Quote from: monkey
Also something to consider is any implications that might be prevalent across various platforms. I have zero experience in cross-platform coding, so I'm not personally aware of any of the caveats or complications that may need to be handled to try and make porting the engine as simple as possible.
Platform compatibility boils down to this:

    * Using libraries that are platform independent
    * Use some level of abstraction for things like integer sizes and endianess.
I pretty much agree with Wyz+ on this although I would suggest more extensive use of abstraction.   

With regard to cross-platform libraries you may find QT interesting. 

http://qt.nokia.com/
http://doc.qt.nokia.com/4.7/gettingstartedqt.html

It features support for cross platform graphics, plugins, network, webbrowser, etc ....  It's also C++ ;)

monkey0506

  • SEND PIZZA.
    • Best Innovation Award Winner 2017, for his work to help AGS games reach the widest possible audience - through popular distribution platforms (Steam, Galaxy) as well as other operating systems (Android, Linux)
    • monkey0506 worked on one or more games that won an AGS Award!
    •  
    • monkey0506 worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #5 on: 27 Jul 2011, 12:35 »
This is actually more of a question of ANSI C vs C++.   In the context of something like the AGS runtime conventional wisdom is that C has a number of advantages over C++.

- ANSI C is considered more portable and is less resource intensive.  C++ implementations tend to vary from compiler to compiler much more so than C.

- ANSI C does not automatically, dynamically allocate/reallocate memory as C++ does.  The latter is fraught with potential problems when executed on unknown or resource limited target platforms.   

- Consequently runtime errors are less likely with ANSI C than with C++

- ANSI C typically has a smaller resource footprint than C++

What makes C-style char* allocation "more portable" than the std::string class? My understanding of the way that the std::string class is written is that it is essentially just a (semi-)managed interface that encapsulates the same thing. Seeing as it removes the need for direct pointer manipulation, it's also significantly less error-prone (despite your claim that char*s are less likely to cause errors, I would say that a memory leak would be significantly more likely than an out-of-memory error on most target platforms, and char*s would be more prone to the former).

You say that "ANSI C" doesn't dynamically allocate/reallocate memory, but I don't see how that could be avoided in any case where the std::string class couldn't avoid it. strings can be passed by reference to avoid creating unnecessary copies, and in the case of any type of manipulation, even char*s would have to be reallocated. I don't see where there would be any difference necessitated between the two in that regard.

If there's any information on the differences between the two and portability I'd be interested in reading it, but I've actually found more information negating your claims than anything supporting them.

I agree with you that OO is a superior organizing principal and that it is not necessary to make use of compiler features to take advantage.  I have been doing this for decades in non-OO languages.   I think it would lead to a better quality codebase as well.

I've actually been working to implement some classes for better organization and OOP-compliancy, such as for enums, properties, and indexers. Specifically enums should really have their own scope (the values being in a sub-scope of the one at which the enum itself exists) to fit with the OO-model. Properties and indexers aren't strictly essential, but they're significantly nicer to work with than directly calling getter and setter methods. Anything that didn't need function-based access could just be implemented as a normal field of course, but for the rest I've got these new classes implemented (after some Googling and extensive trial-and-error ;)).

I disagree C++ classes are needed to achieve this.  All that is necessary is for the the AGS editor to pass on the number of each item used.  ANSI C could be used to allocate the required resources at startup.

I never meant to imply that the STL classes would be necessary, only that they would make things neater in the long-run. C-style memory allocation is more prone to error and memory leaks, whereas if we used the STL classes it wouldn't so much be an issue. Not to mention the fact that even if we wrote some generic methods we'd still be writing a few hundred lines of code that could be simplified by just using an STL class. Unless there's a totally drastic amount of overhead then I simply don't see the aversion to the STL classes at all.

Anyway, I've been doing a lot of reading on various topics and I feel like for the most part what I've been doing will be fine. I'm still working to understand a lot of the engine code so as to try and make sure that I'm implementing the same functionality, but so far it seems to be going pretty well.

Radiant

  • Mittens Knight
  • AGS Baker
  • Return once more to the Two Kingdoms!
    • I can help with publishing
    • I can help with story design
    • Radiant worked on one or more games that won an AGS Award!
    •  
    • Radiant worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #6 on: 27 Jul 2011, 13:06 »
- What, if anything, is the advantage of using char*s vs. the std::string class?
Performance. I don't mean memory, I mean speed.

Bear in mind that AGS was written for DOS and much slower processors than we hav enow, where performance was much more important. A nice benefit thereof is that AGS games run fine on low-end systems even now (whereas there are several retro games on the market that require a 2 GHz processor for no apparent reason).

monkey0506

  • SEND PIZZA.
    • Best Innovation Award Winner 2017, for his work to help AGS games reach the widest possible audience - through popular distribution platforms (Steam, Galaxy) as well as other operating systems (Android, Linux)
    • monkey0506 worked on one or more games that won an AGS Award!
    •  
    • monkey0506 worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #7 on: 08 Jan 2012, 04:17 »
I was reading into this to see what more I could learn about this situation. I am particularly interested in some of the changes that have been made with C++11, and compiler support of those changes.

As far as I understand it, the proposed changes wouldn't affect the compiled code, and so shouldn't bear any relevance to portability (aside from compatible compilers), just to the design-time of the code (correct me if I'm wrong about that).

Particularly the type-safe enums and furthered Unicode support (UTF-8 literals, and the addition of the char16_t and char32_t types (which also have their own literal types)) seem that they would be very beneficial.

From there I was looking at what compilers support C++11, and I was rather surprised by how little MSVC++2010 has been updated to support the new C++ standard. Other than the fact that the existing code has been compiled with MSVC++ 2008, is there any reason why switching to another compiler would be a bad idea? Particularly GCC seems to have good coverage of the C++11 features.

Of course there's likely to be issues in changing compiler, but with the level of refactoring we're looking at, especially if we're looking at a fresh codebase versus maintaining backwards compatibility, it could probably be dealt with...

Oh, and for anyone interesting in educating me, from what I've read about endianness, am I correct in my understanding that it would only be relevant to bitwise functions?


Thanks.

Re: Regarding C++ and the state of the engine source.
« Reply #8 on: 08 Jan 2012, 09:50 »
Oh, and for anyone interesting in educating me, from what I've read about endianness, am I correct in my understanding that it would only be relevant to bitwise functions?
No. Endianness is more pervasive than that. It affects file I/O as well, for just one more gotcha.
Stuart "Sslaxx" Moore.

Wyz

  • anno 1986
    • I can help with making music
    • I can help with story design
    • I can help with translating
    • I can help with voice acting
    • I can help with web design
    • Wyz worked on one or more games that won an AGS Award!
    •  
    • Wyz worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #9 on: 08 Jan 2012, 16:35 »
I've been using MinGW for years and I have to say it some has advantages. MSVS has a good compiler, it has a few quirks a few short comings but uh well. I guess the biggest advantage using MinGW for me has been that it will compile the same on linux and if I pick the right libs I spend little time on porting. 8)
I think supporting both compilers (which is not all that hard when you use macros) is the best solution.
Switching from ascii to UTF-8 would be very nice, but damn, that will be a hellish task :D UTF-8 can be encoded in ascii, but UTF-8 can not easily be converted back to ascii, I found that out the hard way once. :P
The way most engines deal with endianness is by means of a wrapper for all types and io. It is generally the same mostly but some changes to fit the purposes of the application better. Have a look at libs like SDL, those generally have a nice way of handling it. AGS in it's current state also does this to some degree, I haven't really looked into it though so I can't really tell to what extent. If you decide to also add wide char support I suggest also adding it to the wrapper. You won't need templates for this, macros should be just fine.
As for C++11, yes it has some nice features, but compiler portability is also nice. I guess incorporating C++11 specific things in a wrapper only should fix that. That way you can replace parts with libs when certain compilers do not support it yet.
Life is like an adventure without the pixel hunts.

monkey0506

  • SEND PIZZA.
    • Best Innovation Award Winner 2017, for his work to help AGS games reach the widest possible audience - through popular distribution platforms (Steam, Galaxy) as well as other operating systems (Android, Linux)
    • monkey0506 worked on one or more games that won an AGS Award!
    •  
    • monkey0506 worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #10 on: 09 Jan 2012, 07:42 »
I've downloaded Code::Blocks and it seems to be telling me about quite a lot that MSVC++ was letting me get away with, but perhaps shouldn't have. Particularly the code I'm working with involved some usages of variable argument lists, and upon reading into that further I've come to the conclusion that variadic functions cannot safely be considered portable, as they depend on implementation-specific determinations of how parameters are sent through (are they all pushed onto the stack, do they use registers, a combination of both, etc.). For that reason I've started looking into C++11's initializer lists instead of using true variadic functions.

It added a bit of complexity to my code, but with a custom class I was able to create the conversions I need to pass various data into a initializer list, and it actually seems a much safer solution as essentially the initializer_list type is just a specialized array class. I'd actually like it if you could just pass a CSV list in place of an array/pointer parameter, but this is a pretty nice improvement. Since it is a specialized (const) array, the size is automatically determined on creation, and the individual elements are guaranteed to be stored in sequential order, regardless of where exactly they're allocated (at least as far as my understanding).

Regarding supporting multiple compilers, the only "officially" supported compiler for the existing branch of AGS is MSVC++ 2008. Just because it might compile elsewhere doesn't mean it's supported. :P I get what you're saying though, and we should try and keep the code as generic as possible in that regard I feel (limited to preferably no use of compiler-specific extensions and what-have-you). It will make it easier to maintain I'm sure if we're not all limited to the same compiler and can use our own preference.

As for Unicode, it's my understanding, and again, please do correct me if this is wrong, that the standard Unicode implementations map the standard printable ASCII characters the same, right? Such that if I do:

Code: [Select]
char32_t um = U'M';
char m = (char)um;

I should get the correct character back, right? And forgive me if I used the wrong indicator, I forget which one indicates UTF-32 and it's 1:30 in the morning, thank you. In any case, that is my understanding as far as individual character codings. If we did change to UTF-8, -16, or even -32, I can't imagine that we'd have too much need to convert those strings back to ASCII anyway. If we did (say, for a particular library), then what's another few hundred or thousand lines of code, eh? :=

Endianness is clearly something I have no experience in dealing with, so could you give an example of what you mean by a "wrapper"? Do you just mean conditional coding with macros, or a specific function to swap the endianness of a value at run-time?

Oh, by the way, I had asked at the beginning of this thread regarding std::string, which is apparently stricken with its own problems... Why isn't there a serious, modern C++ string class? It seems that you either have to sacrifice usability, portability, or Unicode support. There doesn't seem to be a single class that addresses the needs of the modern C++ programmer. There's a million and fifteen different string class implementations, but they all seem lacking. Or have I just overlooked that one elusive golden child of a class? As for efficiency, I've seen Radiant's post about direct char* access being faster than using a string class, but I can't fathom a situation in which a properly programmed class couldn't deal with that within a reasonable margin of deviance. As I said earlier in this thread, even char* strings would have to be copied, reallocated, etc. in order for them to change their length...and every other operation should be virtually identical short of any overhead implicit in using a class.

RickJ

  • fix'n one thing and break'n two ...
    • I can help with scripting
    • I can help with story design
    • RickJ worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #11 on: 09 Jan 2012, 17:47 »
Perhaps the QT libraries should be considered?
http://qt.nokia.com/

Quote
Qt is a cross-platform application and UI framework. Using Qt, you can write web-enabled applications once and deploy them across desktop, mobile and embedded operating systems without rewriting the source code.

Qt is available for the following platforms:

    * Windows Desktop
    * Windows CE and Windows Mobile
    * Linux/X11
    * Embedded Linux
    * Mac OS X
    * Symbian
    * Maemo/MeeGo

Android is also now supported
http://labs.qt.nokia.com/2011/02/28/necessitas/

Here is an overview of it's API.  It's modular so it's possible to only use what is needed.
http://developer.qt.nokia.com/doc/qt-4.7/modules.html#id-5ff96530-83d8-4e05-a98a-4ca5636bec55

There is also a cross platform plug-in system
http://developer.qt.nokia.com/doc/qt-4.8/plugins-howto.html

A lot of games have been written using QT
http://developer.qt.nokia.com/search?search=games

Bottom line: QT has already addressed portability issues so that it is possible to code once and compile compile that one source for any of the supported platforms.   If we are going to re-factor the runtime in an OO/C++ fashion then QT ought to be considered because it offers so much.  Just my 2 cents.


Shane 'ProgZmax' Stevens

  • Mittens Serf
  • GARBAAAAAGE DAAAAAAY!
    • I can help with animation
    • I can help with characters
    • Lifetime Achievement Award Winner
    • I can help with making music
    • I can help with proof reading
    • I can help with scripting
    • I can help with story design
    • Shane 'ProgZmax' Stevens worked on one or more games that won an AGS Award!
    •  
    • Shane 'ProgZmax' Stevens worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #12 on: 10 Jan 2012, 17:20 »
Wow, solid find there RickJ.  I support this approach!

Edit: Holy shit, the full sdk is 1.6 GB!?  And the download links seem to be down...
« Last Edit: 10 Jan 2012, 17:23 by ProgZmax »

Calin Leafshade

  • Long live King Cat!
    • I can help with making music
    • I can help with voice acting
    • Calin Leafshade worked on one or more games that won an AGS Award!
    •  
    • Calin Leafshade worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #13 on: 10 Jan 2012, 19:16 »
I don't think QT is really appropriate for a game engine. It's primarily a widget toolkit and a gui framework.


Wyz

  • anno 1986
    • I can help with making music
    • I can help with story design
    • I can help with translating
    • I can help with voice acting
    • I can help with web design
    • Wyz worked on one or more games that won an AGS Award!
    •  
    • Wyz worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #14 on: 11 Jan 2012, 00:15 »
I agree with that. If you want to build let's say an editor then Qt is great. Especially now since they made LGPL an possibility, yay!

But ok, portability is always a matter of abstraction and how far you want to go with that. You want to have at least some wrapping so you don't end up with thousands of #ifdef #else #endif blocks. But too much is just as bad (especially as some libs tend to with C++ templates) and will only make your application bloated and hard to debug. So where is the sweetspot for that? Well that really depends on the application and the people that have to maintain its code-base.
Let me illustrate what I mean with wrapping. Let's say we want to read an integer from a file; we will have issues with endianness we need deal with. You don't want preprocessor blocks all throughout your code every time you read an integer so you make a global function and deal with endianness there. There are three practises from there on:
  • use preprocessor blocks in the source of this global function
  • use macros to replace the function name with an existing function or a system specific function
  • supply different source files for each target system


I'd say, use whatever floats your boat. :P

Code: [Select]
char32_t um = U'M';
char m = (char)um;

I should get the correct character back, right?

For UTF-16/32 yes, but I was talking about UTF-8; check Wikipedia for the differences between the two. :P
Just a quick reason why you might want to use UTF-8 instead of UTF-16: UTF-16 takes twice as much space as ASCII, UTF-8 generally not much more as ASCII if you don't write in foreign scripts. But if space isn't really an issue I guess it doesn't matter that much.

To get back to std::string, well sure you can use it if you take the overhead for granted. String operations might as well become slower, in some occations, and faster in other. I don't want to bore people with a talk about data structures so I'll leave it with that. Do know you can use UTF-16 and UTF-32 with std::string actually! Simply use std::string<char16_t> and std::string<char32_t> :)
« Last Edit: 11 Jan 2012, 00:18 by Wyz »
Life is like an adventure without the pixel hunts.

RickJ

  • fix'n one thing and break'n two ...
    • I can help with scripting
    • I can help with story design
    • RickJ worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #15 on: 11 Jan 2012, 09:39 »
Although QT is most well known for it's GUI framework it is so much more.  It consists of a modular platform independent API and a platform specific backend for each supported platform.   I was told many years ago, by one of the developers, that having separate backends allowed them to achieve the appearance, behavior, and performance they desired and that it eliminated most of the ugliness (i.e. endian issues, #ifdefs, etc) and made for a much cleaner solution.   The result is that exactly the same application source (without #ifdefs, etc) can be compiled to run on any supported platform. 

Trolltech (now Nokia) has been at this for close to 20 years in a commercially successful context and have an impeccable reputation for the quality of their work. IMHO, it would be foolish to reject the fruits of their labor without at least taking a look. 

There are over 100 Qt games here. and another 30 here.  These don't include games created by Gluon, a Qt base game engine/editor similar in function to AGS.  There is even a book on Amazon about it, "Open Source Game Development: Qt Games For KDE, PDAs, And Windows",  and an active  Qt Development/Game Development forum on Nokia's website.   And of course some youtube videos here and here.

The notion that Qt is not appropriate for game  and game engine development is just plain silly.

Now there may be good reasons to not use Qt but have yet to be articulated. ;)

Snarky

  • Global Moderator
  • Global Moderator
  • Mittens Lord
  • Private Insultant
    • Best Innovation Award Winner 2018, for his numerous additions to the AGS open source ecosystem including the new Awards Ceremony client and modules
    • Snarky worked on one or more games that won an AGS Award!
    •  
    • Snarky worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #16 on: 11 Jan 2012, 13:34 »
Although QT is most well known for it's GUI framework it is so much more.  It consists of a modular platform independent API and a platform specific backend for each supported platform.   I was told many years ago, by one of the developers, that having separate backends allowed them to achieve the appearance, behavior, and performance they desired and that it eliminated most of the ugliness (i.e. endian issues, #ifdefs, etc) and made for a much cleaner solution.   The result is that exactly the same application source (without #ifdefs, etc) can be compiled to run on any supported platform. 

Trolltech (now Nokia) has been at this for close to 20 years in a commercially successful context and have an impeccable reputation for the quality of their work. IMHO, it would be foolish to reject the fruits of their labor without at least taking a look. 

There are over 100 Qt games here. and another 30 here.  These don't include games created by Gluon, a Qt base game engine/editor similar in function to AGS.  There is even a book on Amazon about it, "Open Source Game Development: Qt Games For KDE, PDAs, And Windows",  and an active  Qt Development/Game Development forum on Nokia's website.   And of course some youtube videos here and here.

The notion that Qt is not appropriate for game  and game engine development is just plain silly.

Now there may be good reasons to not use Qt but have yet to be articulated. ;)


Qt is great (they're based close to my home town!), and certainly has a lot of of features that would be useful for AGS. In fact, one of the potential benefits might be replacing several separate libraries with one single library to handle more or less everything. And the level of cross-platform abstraction is tempting, definitely. But the fact is that it's primarily a widget toolkit, and that therefore many (most?) of the core features aren't relevant to AGS.

If the parts we do need are superior to any of the alternatives, it's not necessarily a huge problem that it also offers other stuff we don't need. The drawbacks would mainly be an unnecessarily complex API, and the library size. Although the SDK is 1.6 GB, the full library binaries are only around 275 MB; I don't know how much space just the libraries we need would take, assuming we would include core, winmain, gui and probably multimedia and opengl; I could also see uses for network, svg and perhaps xml.

From what I see online, the smallest statically linked application execs, which exclude almost all parts of all libraries, are about 1.6 MB on windows. We'd need more than that, although we could probably disable most of the gui widget stuff (which takes up a lot of space). I'm guessing about 8 MB minimum for a Qt-based AGS engine. With dynamic linking this could be reduced, but users would need the 275 MB dll file to run any game. Furthermore, The Qt LGPL license imposes some clauses once you link the library statically; I think we would be OK as long as AGS remains open-source (which would be a requirement!), but it could be interpreted as requiring game makers to open-source their games as well.

It is not clear to me that Qt is superior to SDL or Allegro as a graphics library, or to existing alternatives for other functionality. And in terms of platform support, Android is only in alpha, and there's no iOS support at all (which SDL and Allegro both offer). Going with Qt is certainly not going to solve all issues with portability and the state of the engine source by itself, and my understanding is that it would require a complete rewrite e.g. to use the Qt data types.

monkey0506

  • SEND PIZZA.
    • Best Innovation Award Winner 2017, for his work to help AGS games reach the widest possible audience - through popular distribution platforms (Steam, Galaxy) as well as other operating systems (Android, Linux)
    • monkey0506 worked on one or more games that won an AGS Award!
    •  
    • monkey0506 worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #17 on: 11 Jan 2012, 15:52 »
I looked briefly around the website for Qt, and based on what I've seen, in addition to what Snarky has said, I'm skeptical about the appropriateness of Qt for AGS. I'm not contesting its usefulness for making games or for a game engine, but is this what AGS needs? I'm not convinced.

If we need to use preprocessor code blocks for conditional coding, I would propose that we should wrap them within a specific self-contained class/function/whatever, which would ultimately achieve the exact same effect that Rick mentioned ("without #ifdefs, etc" running amok through the codebase). I'm reasonably certain that the same can be applied to endianness issues. In short, I regard the suggestion that we use Qt for the purposes of avoiding conditional preprocessor coding and/or endianness issues, as bunk. That may certainly be a benefit, but that doesn't bear any relevance as to the issue at hand (whether Qt (as a whole) is appropriate for the AGS codebase). Just because Mozilla Firefox is portable doesn't mean we should base AGS on Mozilla. ;)

They've done a lot of legwork, that we could perhaps benefit from reviewing, but I feel that there's no reason why we couldn't use other libraries that fit what we need (without having to force the users to install a 275 MB DLL to run the game! :o).

I'm also not thoroughly convinced that the overhead of using a class would make such a drastic difference on modern systems. Radiant pointed out that older systems would have reflected a significant performance hit from this overhead, but is that still a realistic issue? I doubt it. AGS has already dropped support for DOS and for Windows 98. Windows XP should be the minimum Windows target version, and I'm all for supporting that platform, but I find it doubtful that many people using XP would suffer terribly from using classes.

Regarding the std::basic_string template, I'm quite aware that it could accept char16_t and char32_t as data types, but I'm concerned about things that I've read regarding different compilers having even slightly varying implementations of the class. From my own understanding, it would likely be a non-issue, but there's a possibility of it causing explosive catastrophe. For this reason I felt that it would be safer to find a very specific implementation that could be included in the source, just to ensure that at the very least we would have the guarantee that the class would have the exact same definition regardless of compiler/platform/etc.

In any case, there's some good and informative discussion going here. I like it! :=

As for portability, how about a target platform list:

Windows: XP and higher
Linux: ???
Mac: OS ? and higher
Android: ?.?? and higher
iOS: ??? (also, I'm sorry, but would this include iPad devices?)
PSP: ???

I think at least the first 5 should be core to the design of AGS 4. What do you guys think?

Shane 'ProgZmax' Stevens

  • Mittens Serf
  • GARBAAAAAGE DAAAAAAY!
    • I can help with animation
    • I can help with characters
    • Lifetime Achievement Award Winner
    • I can help with making music
    • I can help with proof reading
    • I can help with scripting
    • I can help with story design
    • Shane 'ProgZmax' Stevens worked on one or more games that won an AGS Award!
    •  
    • Shane 'ProgZmax' Stevens worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #18 on: 11 Jan 2012, 17:04 »
Anyone have some WORKING mirrors for the sdk?  I'd like to get a closer look at it but the links keep giving me 404 errors ;(.

Snarky

  • Global Moderator
  • Global Moderator
  • Mittens Lord
  • Private Insultant
    • Best Innovation Award Winner 2018, for his numerous additions to the AGS open source ecosystem including the new Awards Ceremony client and modules
    • Snarky worked on one or more games that won an AGS Award!
    •  
    • Snarky worked on one or more games that was nominated for an AGS Award!
Re: Regarding C++ and the state of the engine source.
« Reply #19 on: 11 Jan 2012, 17:59 »
ProgZ, the (Mac) SDK download works for me with the default link. 1.3 GB downloaded in about 1 minute.

Looking at the list of platforms (which I agree with, though I'd also appreciate an MS XNA backend to run games on the Xbox, and ideally a way to run them on the Wii, DS and/or 3DS -- though AFAICT that typically only works on rooted consoles), I wonder if it's at all plausible to design it so that game creators don't have to compile separate versions for each platform, but the engines would all be able to run the same bytecode/resource files (much like ScummVM, in fact). I definitely think games should be distributed with the engine--otherwise the setup barrier becomes too great for casual or non-tech-savvy players--but I worry that many authors won't bother compiling five different versions of their game.