Topic: [DISCUSSION] Integrating Alternative Scripting Languages (was: Lua for AGS)

[Calin Leafshade]

Dualnames: To be merged with topic www.adventuregamestudio.co.uk/yabb/index.php?topic=46493

I think we should consider generalising it and making an interface similar to the the FontRenderer one. I don't think it would be *that* hard. AGS provides serialisation methods and stuff in the plugin API so it would just be a matter of taking what has been learnt from AGS-lua and applying it more generally. Most of the building blocks are already in place.

[Crimson Wizard]

I am actually wondering if there's a way to fully substitute AGSScript with Lua.
To make myself clear - I am not thinking Lua is the best choice (never used it actually) nor that AGSScript should be thrashed right away.
My thought is that since there's an alternate scripting language why not make it fully integrated without AGSScript as an intermediate.
There was a discussion about this earlier (http://www.adventuregamestudio.co.uk/yabb/index.php?topic=46157.msg622114#msg622114).
Not sure if that's easy task though. Just something to keep in mind.

[Joseph DiPerla]

I would think that maybe adding it as a seperate scripting system in AGS all together would be good, while keeping AGSScript would be nice. For instance, in the scripts editor, you have Global.ASH and Global.ASC which is the default language for AGS. However, when adding a script, perhaps you can now have an option of either adding AGSScript or LUA scripts (AGSLUA). You can make calls between the two types of scripts and you can also, possibly do externs and such. As far as events, such as repeatedly execute, it would only be allowed to be called within one of the scripts. You can not have one variation existing in ASC and the same one in LUA. Its either or. If it does exist in both, AGS will run the one found in ASC.

[Calin Leafshade]

Considering the (relative) ease that Denzil managed to implement Lua, I think it would be stupid not to go in this direction.

I am of the opinion that expanding the plugin interface is the way to go here, specifically giving plugins access to hook onto game scripting events. Implementing a defacto language or improving the current scripting language which is not fit for purpose considering the competition seems counter productive. I realise that i have somewhat changed my tune in this regard but on consideration and on seeing what this Lua plugin is capable of it seems to be an inescapable conclusion. When CJ implemented the compiled language CPU cycles were at a premium. Now that is not the case and things like strongly typed variables, pointers and c style string formatting just seem silly in a scripting environment like this.

I mean think about this. *All* our complaints about AGSScript (passing structs, delegates, resizeable collections, encapsulation, inheritence, easy number casting) are fixed here and this plugin sat with relatively little attention. This plugin is the *best* thing to happen to AGS since i've been here. We should capitalise on it and learn from it.

[Crimson Wizard]

I am of the opinion that expanding the plugin interface is the way to go here, specifically giving plugins access to hook onto game scripting events.

Do you mean registering callback that is called along/instead of default event handlers?
JJS mentioned he is to integrate this plugin in his portable engine. He may update plugin interface perhaps. I will check too if I get time.

EDIT:
There's RequestEventHook function in plugin API. But it allows hooking to very limited number of events, mostly room events (enter/leave) drawing and loading events.
Also, plugin callbacks are run explicitly in the engine code. Adding more hooks this way will require adding calls throughout the engine. That should not be difficult, but I don't like it this way. Perhaps there should be an extendable implemenation. Uh, but that's rather for Engine development board.

[SpeechCenter]

This plugin is certainly possible thanks to both AGS and Lua's flexibility and of course ingenuity.

I believe the remaining limitations here occur because both AGS and Lua compile to different intermediate languages. Meaning, if we were to make sure all languages can compile to the same intermediate language then we'll have total flexibility on this. It would make the engine agnostic to the actual language.

I see here 3 routes:

• Examine if Lua can be compiled to existing AGS IL. This would make it an editor change and not an engine change. I already studied some time ago the code that runs the engine and understand it quite well. The IL seems rather general so it's a possibility.
• Adjust the AGS IL to fit requirements by Lua and other scripting languages. This is an hybrid approach. Meaning we make some changes to the IL, but keep the basic core.
• Change the IL completely to something else, maybe Lua's IL, maybe a different one. This is the most risky approach, it requires verifying backwards compatibility can be maintained.

Another part is making sure the editor allows different languages, this is much easier to do once we have a documented IL that has been proven to work with more than one language. One possibility is to extend the plugin interface to support this and allowing the editor to have a standard interface to all its scripts regardless to their language.

Anyway, this requires technical analysis, I'll be happy to take part. My relevant background is C, C++, x86 assembly, text parsers and more recently, C#.

[Denzil Quixode]

(Edit: No longer relevant now the thread is split)

[Calin Leafshade]

I'm not sure that's the best way to approach it.

Compiling Lua to a bytecode IL in the editor removes some of the benefits of Lua (runtime reloading of scripts for instance).
It also raises the level of entry for new language implementations.

Also, surely keeping the language implementation as close to the official release is better since new versions can be dropped in on release.

I guess the question we need to ask is that is the overhead of a plugin interop worth the ease of making new languages available.
My thought is probably yes since speed is not of primary importance but others may feel differently

Edit: Also, yes perhaps a new thread might be useful. Can a mod split out the thread or something please?

[Crimson Wizard]

I feel it's better to focus on extending plugin interface for now.

Compiling from lua (or other language) to AGS IL is a totally separate task, and will take more time to discuss and implement. Besides, as Calin mentioned, it will add restraints to external language's features and efficiency. It is also a separate way of making scripts. In fact those two ways may co-exist.

[Denzil Quixode]

Lua requires various things that I'd think would be very difficult to make work in an IL that wasn't specially designed for it, even ignoring dynamic compilation:

• A hybrid hashtable/array "table" datatype. (Note that any datatype can be used for table keys, not just strings, unlike e.g. JavaScript objects.)
• Dynamically-typed local variables.
• A good garbage collector designed for handling dynamic objects (with special support for "weak" tables).
• Closures with upvalues.
• Functions as garbage-collectable, first-class values.
• Coroutines.
• Double-precision floating point number support. Like JavaScript, Lua has a single "number" type, which normally is a double. (Switching to float for the number type is not a good idea, since while you can store any int value as a double without loss of precision, the same is not true with floats - so you would lose the ability to reliably store int values.)
• Distinct boolean type: In AGS-Script (as in C), a boolean is really just an integer, where "false" is just another name for the number zero. Whereas in Lua, "true" and "false" are special values, and the number zero is unrelated - if you use the number zero in place of a boolean, it will be understood as "true", not "false".

[Denzil Quixode]

If AGS saves a screenshot regardless of whatever the game settings are, then it might be AGS.

I'm not sure that it does, but another possibility just occurred to me: do you ever use Room.GetDrawingSurfaceForBackground()? I think if you do that, at least while you stay in that same room, the edited background would need to be stored in any savegame data too.

[AJA]

Nope, there shouldn't be any dynamic sprites or surfaces since all I use them for is rendering dialog text, and they're all deleted before you can save. (Unless they're in a coroutine but that's a rare case.)

[SpeechCenter]

Compiling Lua to a bytecode IL in the editor removes some of the benefits of Lua (runtime reloading of scripts for instance).
It also raises the level of entry for new language implementations.

Also, surely keeping the language implementation as close to the official release is better since new versions can be dropped in on release.

It's true, but there's more than one option here (I mentioned 3 and there are probably more). Let's cover what are the requirements before jumping to the solutions (so sorry for doing that earlier)

Here are some features that we need to decide if they are required (not saying they are, just questions I have here and the list of feature requirements should be longer eventually):

• Clear interface to support additional languages vs. single language
• If multiple languages are supported, can one project easily host multiple languages or do you choose a language at the beginning
• If single language, what is the existing games migration path
• Easy to learn (newcomers learning curve)
• Integration within dialog scripts
• Backwards compatibility
• Double click on room event and reach the actual code (and not AGS script that calls another language)
• Debugger
• If done via plugin interface, is integration between different plugins required? (not necessarily plugins that introduce scripting language)

[Clarvalon]

De-coupling the scripting from the game engine via an interface would seem to be the option providing the most flexibility going forward.  I've done something similar with XAGE:  http://youtu.be/0Ke49lJ8R94

The argument could also be made that having AGS Script tightly integrated in the IDE is one of AGS Editor's strengths.  As it stands, AGS is a tight, standalone package.  Potentially, beginners may be confused or deterred if forced to maintain their game scripts outside of the main IDE.