AGS 3.3.x/3.4.0.0 Developer Discussion

Started by Gurok, Sun 09/02/2014 08:45:13

Previous topic - Next topic

Calin Leafshade

I agree that replacing it with another language (AngelScript would probably be best) would be the best course of action but it's a significant thing to do.

Replacing it would be better in the long term because we get all the advantages of a popular, off-the-shelf library but it would take a lot of work.

Gurok

Quote from: Crimson Wizard on Sun 13/07/2014 16:28:31
E: The dumb way is to have a list of offsets for all managed handles inside struct. In any case this means that script interpreter must support some kind of "type description".

Speaking of dumb ways... if you can't solve it, the half solution you proposed on GitHub seems reasonable to me. Disallow non-built-in managed struct pointers inside managed structs. We could then enable them at a later date when we've figured out how to manage them. I know it cripples managed structs, but there's already so much functionality provided by passing in and returning struct pointers. I'd hate to see it shelved. Crippled managed structs also don't do anything bad. They don't box us in for future development.

Quote from: monkey_05_06 on Sun 13/07/2014 18:52:08
If AGScript is replaced by a Java-based language, I will stop contributing to AGS development.

Hear, hear. I don't think anyone who contributes really wants that.

Regarding AGS script, I'm not in favour of replacing it for the time being. AGS script isn't just the language, it's the VM too. We aren't reinventing the wheel. We inherited a project that (for better or worse) reinvented it and we're uh... reinventing a few spokes. Ewww, terrible analogy. Point is that AGS script is 90% there.
[img]http://7d4iqnx.gif;rWRLUuw.gi

monkey0506

^ I basically... Yeah, what Gurok said. ;)

Crimson Wizard

Quote from: Gurok on Mon 14/07/2014 01:53:17
Speaking of dumb ways... if you can't solve it, the half solution you proposed on GitHub seems reasonable to me. Disallow non-built-in managed struct pointers inside managed structs. We could then enable them at a later date when we've figured out how to manage them. I know it cripples managed structs, but there's already so much functionality provided by passing in and returning struct pointers. I'd hate to see it shelved. Crippled managed structs also don't do anything bad. They don't box us in for future development.
Well, if you think that's proper; you then need to fix the compiler appropriately, and reopen/remake the pull request.

Gurok

#84
Quote from: Crimson Wizard on Mon 14/07/2014 07:08:26
Well, if you think that's proper; you then need to fix the compiler appropriately, and reopen/remake the pull request.

I just want to make sure I understand the problem correctly. Managed structs with user struct pointers are bad because the references aren't cleaned up recursively, but structs with pointers that come from the engine (e.g. Character, Object) should be fine as the engine cleans them up when appropriate. Correct? I need to know if I should lock down all pointers or just user ones. Never mind. I just read your GitHub note again. I believe I need to restrict them so that they can't include any pointers.
[img]http://7d4iqnx.gif;rWRLUuw.gi

Crimson Wizard

#85
Quote from: Gurok on Mon 14/07/2014 07:21:05I just read your GitHub note again. I believe I need to restrict them so that they can't include any pointers.
BTW, we need to recheck if any of the existing managed structs contains pointers... if some do, then some alternate solution should be invented.
For instance, we may allow pointers in structs declared as "builtin".

Regarding universal solution. Here's a raw sketch:
1. "Type info" object is created for every basic type (int, float, pointer) and every user-declared managed struct. Type description tells object size and useful info. For the time being we need an array of offsets for pointer types in structs (letting engine know which pointers this object has).
2. When any object is created with new, compiler tells not the size of struct, but an ID of type info, letting object know which struct it represents.
3. We may even make dynamic arrays work the same way: make the "create object" command pass extra "number of elements" parameters. Type info is one of the basic types (int, float or pointer), so array will know how to deal with elements too.

This will require increasing "script data format" number (I recall there's one); the compiler must write the "type infos" chunk, and engine must know how to read it.
Don't know about compiler, but I don't think this should be terribly difficult for engine.

Radiant

Quote from: Snarky on Sun 13/07/2014 18:33:05
I really don't mean this as discouragement or criticism, but I wonder if all the energy put into hacking the scripting language might not be more usefully spent on other parts of the engine. The fact is that AGS Script isn't that unique, and we might be better off simply adopting a standard one with an existing compiler than having you spend so much effort trying to improve what we have.
That strikes me as a very bad idea, as it would throw out backwards compatibility with all existing games and ongoing projects, as well as invalidate most of the skill that our community members have developed with using the scripting language.

The AGS scripting language works, and has worked well for over a decade, and we've got a database of nearly two thousand games to prove it. There's absolutely no good reason to replace it by some arbitrarily chosen "more popular" scripting language.

Calin Leafshade

Quote from: Radiant on Tue 15/07/2014 14:17:04
There's absolutely no good reason to replace it by some arbitrarily chosen "more popular" scripting language.

Speed and upstream support seem like the most obvious ones to me.

Snarky

What Calin said. There's also (depending on the choice of language):

  • Improved documentation and more/better scripting tutorials "for free". Beyond very basic scripting, AGS doesn't really offer much help for learning, and many important features are not even clearly documented â€" at least in the most in obvious locations (for example, any time I need to export/import variables or declare struct members functions I have to go find an existing script that does it; neither is properly explained in the manual). Of course, our documentation could be improved, but that's just making more work: in addition to reinventing the wheel we also have to explain it!
  • Better code debugging/analysis tools.
  • More appeal to new users. It's all well and good to say that you've been using it for ten years and it works fine for you. But we also want to attract new developers, and a home-rolled scripting language is not exactly a selling point.
  • Ability to use standard libraries and code samples.
  • (Speculatively): Ease of porting. While AGS is clearly the bestest engine in the world, for various reasons some poor deluded sods occasionally decide that they'd rather use a different engine after all. (For example: Nelly Cootalot 2 in order to target more platforms.) Scripts written in a standard language could potentially be easier to port, particularly if the new engine use the same one (otherwise there are cross-compilers). This could be an argument for Javascript, which Unity3D supports a dialect of.
  • As a consequence of this, using a standard language might make AGS more popular as a game prototyping tool.

At the same time, I agree that it's important to preserve as much of people's scripting expertise as possible, and make it as easy as possible to port ongoing projects (not to mention existing modules!), so I would favor a language with a syntax as similar as possible to AGS Script. That would probably be AngelScript or C# on Mono, although Javascript is also superficially similar. As to monkey's reaction, none of these options are Java-based.

Radiant

Quote from: Calin Leafshade on Tue 15/07/2014 14:38:20
Speed and upstream support seem like the most obvious ones to me.
I've seen zero evidence that AGS has speed issues. In fact, I've done all kinds of crazy scripting within AGS (including writing first-person maze games and platformers) and based on this experience I'd suggest that AGS's speed is fine.

Quote from: Snarky on Tue 15/07/2014 17:05:27
What Calin said. There's also (depending on the choice of language):
The thing you're overlooking is that AGS's scripting language isn't some weird arbitrary custom design. Rather, AGS's language is, for the intent of learning it, pretty much identical to C++, one of the most commonly used languages in the world (granted, there are some C++ constructs that AGS doesn't support, but nothing that would be used by somebody learning how to program). This also means it's very close to C, C#, PHP, and JAVA; meaning that any novice programmer familiar with any of those can pick up AGS almost instantly, and can use tutorials and coding samples for any of these languages. By constrast, LUA is known to substantially less people.

As far as I've seen, AGS ports just fine to Mac, Linux, and Android already, so I'm not sure what else you'd want to port it to. And I'm not sure why anyone would want to use AGS as a prototyping tool; it's an engine for creating games, not a prototyping tool.

Now I grant that it would be nice to use better debugging tools and be able to import standard libraries, but converting AGS to some other language won't actually accomplish that. Rather, I'd be curious to see what specific examples of debugging and libraries people would need. I'm willing to bet that simply creating these would be way easier than swapping out AGS's scripting language to something else.

Overall, you're suggesting a huge sweeping change, requiring huge time investments with clear and obvious drawbacks, for only hypothetical benefits. That's not a good tradeoff.

Calin Leafshade

Quote from: Radiant on Tue 15/07/2014 20:46:05
I've seen zero evidence that AGS has speed issues.

The core engine dev team even admit there are speed issues. The compiler has absolutely no optimization at all which is especially noticeable in tight loops.

Radiant

Quote from: Calin Leafshade on Tue 15/07/2014 21:51:14
The core engine dev team even admit there are speed issues. The compiler has absolutely no optimization at all which is especially noticeable in tight loops.
Ah yes, that same core engine team that states above that they'll quit if AGS changes to a different scripting language. Sorry, I don't see how you have much of an argument here; hundreds of game developers and 100000s of game players have no issue with AGS's speed.

Crimson Wizard

#92
Quote from: Radiant on Tue 15/07/2014 22:11:05
Quote from: Calin Leafshade on Tue 15/07/2014 21:51:14
The core engine dev team even admit there are speed issues. The compiler has absolutely no optimization at all which is especially noticeable in tight loops.
Ah yes, that same core engine team that states above that they'll quit if AGS changes to a different scripting language.
Now, wait a minute... monkey_05_06 is not a "core engine team", he is a single person with his own opinion :). Also, I do not recall he ever mentioned slowness of AGS script.
I had though, and my opinion is different (I am not saying "opposite" btw). Well, at least I would not quit just because of different scripting language. I guess monkey has his own long history with AGS Script, so he is sensitive when it comes to this topic.

Calin Leafshade

#93
I was specifically talking about CW, yes.

Reference:

Quote from: Crimson Wizard on Wed 27/11/2013 23:37:05
Quote from: Calin Leafshade on Wed 27/11/2013 23:21:56
The AGS compiler, AFAIK, has no optimisation and uses no advanced CPU instructions.
Unfortunately, that's true. It produces very formal sequence of instructions, mimicing how real x86 processor (IIRC) works: putting values to registers, getting them back, calculating, putting result to register, getting it back once again for other use, etc, etc., and every action is processed separately. I think it does at least about 3-5 things more than needed per each primitive operation.

This "formal sequence" makes things like loops very very slow.

I have benchmarked this and I reckon the Lua plugin, which has an extra layer of abstraction to deal with, is about 5-10 times faster depending on what you're doing (loops mainly).

Also this thread: http://www.adventuregamestudio.co.uk/forums/index.php?topic=47320.0

So when talking specifically about the speed of AGS script, it is you that has no argument here, my friend.

When it comes to replacing AGS script, I am broadly for it but I understand that it's simply unlikely to happen due to the time involved.

Snarky

Quote from: Radiant on Tue 15/07/2014 20:46:05
Quote from: Calin Leafshade on Tue 15/07/2014 14:38:20
Speed and upstream support seem like the most obvious ones to me.
I've seen zero evidence that AGS has speed issues. In fact, I've done all kinds of crazy scripting within AGS (including writing first-person maze games and platformers) and based on this experience I'd suggest that AGS's speed is fine.

AGS clearly has speed issues. Whether the scripting language is the real bottleneck in most practical situations is admittedly more of an open question.

QuoteThe thing you're overlooking is that AGS's scripting language isn't some weird arbitrary custom design. Rather, AGS's language is, for the intent of learning it, pretty much identical to C++, one of the most commonly used languages in the world (granted, there are some C++ constructs that AGS doesn't support, but nothing that would be used by somebody learning how to program).

Like for loops? (At least until the upcoming version.) Switch/case statements? Classes? And on the other hand, how many "pObj->method()" expressions do you see in AGS Script? Yes, AGS Script roughly resembles C++, but there are plenty of differences, and anyone trying to use a C++ tutorial to learn AGS Script isn't going to get past "Hello World!"

You can make an argument for AGS Script being less intimidating to newbies than Lua (though it is very popular for game development), but that argument doesn't really stand up against the other alternatives mentioned.

QuoteAs far as I've seen, AGS ports just fine to Mac, Linux, and Android already, so I'm not sure what else you'd want to port it to.

http://unity3d.com/unity/multiplatform
The point wasn't about AGS supporting more platforms, but making it easier to move game logic between AGS game builds and builds in other engines.

QuoteAnd I'm not sure why anyone would want to use AGS as a prototyping tool; it's an engine for creating games, not a prototyping tool.

A game prototyping tool. I can see ways in which it would come in handy, but it's a pretty speculative use-case, so let's not argue about it.

QuoteNow I grant that it would be nice to use better debugging tools and be able to import standard libraries, but converting AGS to some other language won't actually accomplish that.

Well, switching to a standard language would allow us to use off-the-shelf tools and libraries, so it would accomplish that.

QuoteRather, I'd be curious to see what specific examples of debugging and libraries people would need. I'm willing to bet that simply creating these would be way easier than swapping out AGS's scripting language to something else.

Perhaps the handful of specific features or libraries I could mention off the top of my head would be easier to reimplement (for example, let's start off with a library for handling database connections, proper XML parsing, graph layout, a comprehensive physics engine, a more advanced parser, an agent-based AI library, and better gesture/drawing recognition...), but that building every feature and library that would be useful for an adventure game would be quicker than slotting in a different scripting language I find highly doubtful. That's sort of the crux of the argument: While switching to a standard language may be more work up front, trying to compete with popular, well-supported languages on features, tools, libraries and performance is an insurmountable task, and no matter how much work people put into hacking it, it's still not going to be as good. (No offense to the engine devs, but without a large, expert, dedicated team focusing solely on the scripting language, it's just not.)

The same way AGS shouldn't rely on its own home-brewed graphics library or audio library, it really shouldn't have its own home-brewed scripting language. The only reason it does is legacy, and while it will no doubt be with us for quite a while still, it would be better to start the planning for its replacement earlier rather than later, in my opinion.

Radiant

Quote from: Calin Leafshade on Tue 15/07/2014 22:52:44
I have benchmarked this and I reckon the Lua plugin, which has an extra layer of abstraction to deal with, is about 5-10 times faster depending on what you're doing (loops mainly).
That's really not the point. Just because certain things can theoretically be faster doesn't mean that AGS has speed issues. If you want pure speed, go program in assembly code. The question is whether AGS is fast enough for what it's meant to do (i.e. run adventure games, and possibly other games) and the answer is a resounding "yes".

Look, if you're going to make a case for fundamentally rebuilding AGS, start with practical arguments. As in, find some people who've been working on a large scale adventure game, and ask them what their actual issues were, and if they managed to create a game anyway despite the lack of a "switch" command. Ask if any of their players found that their low-resolution games were running inordinately slow on contemporary computers. Ask whether they found AGS hard to learn, and if they'd have had an easier time if its script had been based upon Turbo Pascal. Get a view of the real issues, not of hypotheticals.

Crimson Wizard

#96
I've mentioned this once: if there were a plugin interface for script interpreter / compiler, you could use any language in AGS.
Interpreters mainly do their internal job, their only connection to engine is registered engine API.

Calin Leafshade

Yes, that would be best. Thats essentially what the Lua plugin does. Most of the necessary API functions are there (except for registering engine events).

monkey0506

I have a question regarding the issue of pointer members of managed structs. CW has said that this can't (simply) be made to work. I'm trying to understand why. The engine already has reference counting and garbage collection for the built-in managed types, including DynamicSprites and dynamic arrays. These pointers can be included as members of user structs, which only exist as one instance, but are still cleaned up by the garbage collector. There are no known issues with pointer members of user structs. So where is the discrepancy when it comes to managed (dynamically allocated) user structs? Shouldn't the reference counts for pointer members still be updated, and the relevant objects freed if their reference counts reach zero? :-\

Crimson Wizard

#99
Quote from: monkey_05_06 on Sun 20/07/2014 04:08:31
I'm trying to understand why. The engine already has reference counting and garbage collection for the built-in managed types, including DynamicSprites and dynamic arrays. These pointers can be included as members of user structs, which only exist as one instance, but are still cleaned up by the garbage collector. There are no known issues with pointer members of user structs. So where is the discrepancy when it comes to managed (dynamically allocated) user structs? Shouldn't the reference counts for pointer members still be updated, and the relevant objects freed if their reference counts reach zero? :-
The problem is in changing reference counter.
The difference between two struct types - the ones "statically" and "dynamically" created - is that those statically created are "managed" by compiler. Compiler knows when to destroy them, and what to do with their data. IIRC when a struct instance is deallocated (deleted from local stack or global data), all reference's counts are decreased, but this is done by compiler (it writes certain script commands), because compiler knows the structure of data, it knows WHAT to decrease.
Dynamic arrays "know" that too, because they contain only one type of data (and specifically aware if this is pointers).
For dynamic user object implementation in the engine, there's no "structure", there's only a data chunk of given size. In order to know if this data contains references and where they are located in this chunk we need some kind of "type description".
Basically, this is it.

Here I explained how this may be done (for example):
http://www.adventuregamestudio.co.uk/forums/index.php?topic=49969.msg636492665

EDIT: Another possible way is to invent some kind of automatically generated destructor (created by compiler), which will do same work, but as a part of byte-code.
The compiler does not know WHEN to dispose object, so this will require engine to run a generated byte code by given address, or something.

EDIT2: I support the first solution. Furthermore, I'd even support changing how compiler/engine manage stack and global data, so that compiler would just tell engine which variable to dispose, and engine knew how to do that, instead of emulating x86 "pop stack" operations... (but this is another topic).

SMF spam blocked by CleanTalk