Messages

[compile-time scripts]

This is a bit of a rambling post about potential future features of the plugin. I'd love to hear feedback, particularly from people who can see potential possibilities here, or would at least be interested in experimenting with them.



One idea that I had early on but never quite got around to sorting out is compile-time scripts.

The idea here is that, when you hit compile, the editor itself sets up a Lua state and runs a special script.

Any changes made to the Lua state by this script would be preserved and loaded when the game is run, in the same way that the Lua state is saved and loaded when save games are used.

As a trivial example, if the compile-time script was this:

Code: Lua

compiled_on = os.date('Compiled on %Y-%m-%d')

...then a string with the date that the game was compiled would be accessible as the global variable compiled_on in run-time scripts.

Again, that's a trivial example, I know that if having the compile date was such a useful thing it would just be added to AGS. But the more interesting possibilities are to do some heavyweight pre-processing - if there is a lot of set-up involved right at the start, you could do it at compile time rather than run-time, so the start-up time could be reduced, and you would not need to include a whole bunch of code that would only ever be run at the start, reducing the amount of Lua code left around bulking up the game data and save-game files.

You could even access files in the project source folder. For example, to get the total number of times the game has been compiled, you could store this number in a project file and update it each time. I'm sure there are lots of other possibilities to accessing files in the project folder from a compile-time script.



But maybe it shouldn't always be necessary to do that, and we can do better. Say that when the compile-time script gets run, there is a special project table available. This table is only available to compile-time scripts, not run-time scripts, and it is special in that its contents get preserved between compilations. So, another trivial example of a compile-time script:

Code: Lua

if type(project.compiled_count) == 'number' then
  project.compiled_count = project.compiled_count + 1
else
  project.compiled_count = 1
end

compiled_count = project.compiled_count

Now, the run-time scripts can access compiled_count.

Note that compiled_count is copied to a global variable, because the project table itself won't be available.

The reason for that is that, in real-world examples, I imagine the project table would mostly be used for storing cached partial precalculations and other things that don't need to be in the final game data.



Maybe it should be an option though, so if you do something like this:

Code: Lua

project.runtime_access = true

...the project table (or a copy of it made at compile-time, technically) will available at run-time.



Taking the idea of Lua scripts running in the Editor even further - how about using Lua to script lightweight editor plugins?

The idea is to have a new collection of Lua scripts, called something like editor action scripts. These are not project-specific but would be stored in a subfolder of the AGS editor home directory. (Actually there could be project-specific ones as well, if that's useful.) Unlike the compile-time scripts, what these scripts do to the Lua state is not preserved for the run-time scripts. They might be able to access the project table described earlier, though.

These scripts can be run from within the editor whenever you like, by right-clicking and selecting "Run" from the dropdown menu. (Possibly they could be optionally bound to hotkeys, too.)

These editor action scripts would have access to a special global object called editor. This is an interface to the state of the editor, able to read and change things in the current project in the same way a full-fledged .NET editor plugin can. It would also have a few special dialogs, so you can pop up a message box with custom text, ask a Yes/No question, get the user to enter a string, or get the user to select a certain game item from the current project (sprite, character, etc.) with a selector dialog.



There is a project called LuaInterface that allows arbitrary .NET objects to be scripted by Lua, e.g.:

Code: Lua

luanet.System.Windows.Forms.MessageBox.Show("hi")

The editor action scripts could use this too.

I've been thinking about making the Lua plugin even more powerful by allowing for scripts that get run in the editor, rather than in-game. I was going to post about it here but I think the plugin thread might be a better place.

Calin - It's not going to fast, definitely not something you want to be using a lot every frame, but there is a pure-Lua module for bitops you could use for now: http://files.luaforge.net/releases/bit/bit/luabitv0.4

[try]

The first group (artists) probably isn't interested in learning how to handle exceptions and all the stuff a full featured programming language comes with. My guess is they will be fine with the current language.

Well, Lua's exception handling is actually pretty weak. It doesn't have native try/catch blocks like most other exception-supporting languages. Instead you use a special function called pcall to call a function instead of running it normally - pcall(myFunction, parameter1, parameter2) instead of myFunction(parameter1, parameter2) - and what you get back if there is an error is just an error message string, not a full object with additional information. (Actually, if you throw your own errors using the error() function, you can use any value, it's the built-in errors are all only strings.)

Also, in some ways Lua is less of a "programmer's language" than AGS-Script. Lacking "mutation" statements like a++ and a += 2 has already been mentioned. Lua also does not have binary operator support, although this is possible through an extension module (that I should probably get built in to the plugin), but then it is functions rather than operators - e.g. bit.bor(a, b) instead of (a | b), bit.band(c, d) instead of (c & d) and so on. (Lua 5.2 does have its own version of this module built-in, but for various reasons I intend to stick to version 5.1 for the plugin for now.)

Also, like JavaScript, Lua only has one kind of number (double-precision floating point). The way these bit operations work is all arguments are converted to 32-bit ints, the operation is done, and the result is converted back to a double. (Again, the same is true in JavaScript.) So if you want to do something that really relies on using true 8-bit/16-bit integer values like char and short, you will find this a lot easier to do in AGS-Script than Lua.

compile-time errors (the latter being something that Lua, as an interpreted language "can't do").

This actually isn't quite right, Lua is not interpreted but compiled to a virtual machine bytecode, so any genuine syntax errors (as opposed to typos creating unintended new variables) can be caught and cause an error at compile-time.

I've thought about it, but I feel like it would only really be worth doing if both AGS APIs were more deliberately tailored to supporting custom third-party scripting languages in general, and then had custom hooks for seamlessly dealing with breakpoints etc. for those languages at run-time in particular.

[socket.lua]

One thing you could try is removing the C functions just before the game gets saved, and re-load them just after the game is restored.

I haven't had a chance to test this myself, but if you add something like this to the end of socket.lua:

Code: Lua

-----------------------------------------------------------------------------
-- Avoid having any C functions around while the game is being saved
-----------------------------------------------------------------------------

function base.ags_presave_socket()
    socket = nil
    base.package.loaded['socket.core'] = nil
end

function base.ags_postrestore_socket()
    socket = require 'socket.core'
end

...then make sure these functions are called on the pre-save and post-restore events, hopefully that will make the C functions unreachable during serialization but still there when you need them.

[are]

Also if you'd like to make it a bit more robust by sanity-checking that self-yielding functions like QuestionScreen:AskAsync() are called from the context of an async(function() ... end) "block", you could modify the async() system a bit like this:

Code: Lua

-- weak table for identifying async coroutines without leaking memory
local async_coros = setmetatable({}, {__mode='k'})

function async(fn)
  local co = coroutine.create(fn)
  async_coros[co] = true
  coroutine.resume(co)
end

-- if called in the context of an async block, return the associated coroutine
-- if not running in an async block, throw an error
function inasync()
  local co = coroutine.running()
  assert(co and async_coros[co], 'async function called from non-async context')
  return co
end

...and then, use inasync() instead of coroutine.running() in QuestionScreen:AskAsync().

Here's what I would try, to make it a little neater:-

Have a self-yielding version of QuestionScreen:Ask() that expects to be called in the context of the coroutine to be suspended:

Code: Lua

function QuestionScreen:AskAsync(...)
  self:Ask(coroutine.running(), ...)
  coroutine.yield()
  return self.Selected
end

Create a utility function for running a block of code in a coroutine:

Code: Lua

function async(fn)
  coroutine.resume( coroutine.create(fn) )
end

An example of using these things together:

Code: Lua

async(function()

  logmsg("run")
  
  if QuestionScreen:AskAsync("Yes", "No") == 1 then
    PlayerAvatar:Say("Lol")
  else
    PlayerAvatar:Say("Not Lol")
  end
  
end)

I've added Lua.ScriptCount and Lua.ScriptNames[]. Here is the dev version of the runtime plugin with this change: download

It's slightly complicated by the fact that the scripts can be put in folders (and really are stored in this folder hierarchy internally, not an array).

I've added the line numbers to git.

Here's a built version (just the editor plugin): download

Great work JJS! Thanks! I've merged the changes, it all looks reasonable and it compiles without a hitch. (I can't remember why I ever had that mutex...)

I have a quick question on compiling the source: Is the version of Lua used (5.1.4) critical or would the plugin also work with Lua 5.2.x?

Unfortunately the Pluto serialization library that this plugin's save/load system relies on is extensively tied to the internals of 5.1. If there is a 5.2-compatible version of Pluto I'm not aware of it.

[edit:]

A new version (edit: another new version) of the plugin is now available from http://lua-for-ags.wikispaces.com/Downloads

The main changes:

• You can now access game variables (ags.game.text_shadow_color, ags.game.total_score, etc.) and character fields (ags.player.on is probably the only useful one). Values that can only be 0 or 1 are exposed as Lua booleans, so make sure you do things like "if ags.player.on then..." instead of "if ags.player.on == 1 then..."
  
• You can queue up a call to a global script function that takes 0, 1 or 2 int parameters, to be executed as soon as the current script has finished, by using:

  Code: Lua

  ags.queue('MyGlobalFunc' [, intparam1 [, intparam2 ] ])

  ...or to a room script function, with

  Code: Lua

  ags.Room.queue('MyRoomFunc' [, intparam1 [, intparam2 ] ])

If AGS saves a screenshot regardless of whatever the game settings are, then it might be AGS.

I'm not sure that it does, but another possibility just occurred to me: do you ever use Room.GetDrawingSurfaceForBackground()? I think if you do that, at least while you stay in that same room, the edited background would need to be stored in any savegame data too.

Lua requires various things that I'd think would be very difficult to make work in an IL that wasn't specially designed for it, even ignoring dynamic compilation:

• A hybrid hashtable/array "table" datatype. (Note that any datatype can be used for table keys, not just strings, unlike e.g. JavaScript objects.)
  
• Dynamically-typed local variables.
  
• A good garbage collector designed for handling dynamic objects (with special support for "weak" tables).
  
• Closures with upvalues.
  
• Functions as garbage-collectable, first-class values.
  
• Coroutines.
  
• Double-precision floating point number support. Like JavaScript, Lua has a single "number" type, which normally is a double. (Switching to float for the number type is not a good idea, since while you can store any int value as a double without loss of precision, the same is not true with floats - so you would lose the ability to reliably store int values.)
  
• Distinct boolean type: In AGS-Script (as in C), a boolean is really just an integer, where "false" is just another name for the number zero. Whereas in Lua, "true" and "false" are special values, and the number zero is unrelated - if you use the number zero in place of a boolean, it will be understood as "true", not "false".

[Edit:]

(Edit: No longer relevant now the thread is split)

This looks really great! Huge kudos to all involved. (And I have to admit, it's very exciting to see the plugin being taken for a spin! )

If I am doing a lot of processing in an AGSE_SAVEGAME event handler, potentially enough to cause sound skipping, is it safe to call the PollSystem() method to avoid the sound skip?