Changes to the AGS Scripting language

[Crimson Wizard]

I'm looking into making the built-in instantiable types into autoptr types as a step toward deprecating the pseudo-pointer syntax.
<...>
Once I get it figured out I can submit a pull request for it.

Wait a minute... are you going to put all this to 3.3.1? We would need a compiler switch to retain backwards compatibility... or something...
Does it have a true benefit that make it worth such change? Remember, you will practically force people to edit tonns of scripts and maybe some modules.
Also! this may screw some plugins, if they create script declarations with pointers.
Maybe it is better to delay this until some "groundbreaking" version?

PS. I am really getting worried about this... not that I think that language will become worse without * sign, but we are making AGS for certain people, not for the pleasure of hacking the program.

Sticking the keyword in front of the struct definitions was the easy bit, but now I have to figure out how to differentiate between the autoptrs and the actual instances that are being imported (like cEgo, which is a Character instance, not a Character*). I presume at this point I need to be looking in the engine code. Any pointers appreciated.

I think its in "AGS.Editor.Tasks.AppendCharactersToHeader()".

[Gurok]

I agree with you, CW. I've felt a little paralysed by the discussion going on here. I really don't want to put any changes that would break scripts into 3.3.1. I want to go back to a *simpler* plan for now:
- new operator (done)
- keyword "builtin" to lock down any operations we don't want (creating new objects, extending, etc.)
- fix bugs (somewhat done, zero-sized structs should probably be tolerated)
- constructors (even this is iffy right now)

Save the overhaul for 3.4 or even 3.5. I think we shouldn't worry about needing to remove "builtin" in the future. The rule should be: anything that doesn't get a syntax highlight isn't officially supported, and just leave it at that.

[monkey0506]

I did, of course, intend to add an option to retain legacy pointer syntax, which would be on by default for any old projects being imported and off by default for new projects. It wasn't some careless or thoughtless hacking either, but a step toward implementing a meaningful class keyword, as per my prior suggestion. I don't think that optionally removing the pseudo-pointer syntax (while preserving full compatibility for its use) constitutes such a drastic change that we should be forced to put it off.

Sticking the keyword in front of the struct definitions was the easy bit, but now I have to figure out how to differentiate between the autoptrs and the actual instances that are being imported (like cEgo, which is a Character instance, not a Character*). I presume at this point I need to be looking in the engine code. Any pointers appreciated.

I think its in "AGS.Editor.Tasks.AppendCharactersToHeader()".

I wasn't referring so much to where the instances are included in the header, but the fact that they are importing the actual instances rather than importing pointers. If the types are made to be autoptr, then the imports are seen as pointers, not the actual instances.

[monkey0506]

Created an unrelated pull request, to allow dynamic arrays within structs:

Code: ags

struct MyStruct
{
  int Array[];
};

function game_start()
{
  MyStruct ms;
  ms.Array = new int[5];
  ms.Array[3] = 42;
}

Will definitely require further testing, but preliminary testing (including saving and loading tests) show that this should be working.

[Crimson Wizard]

I did, of course, intend to add an option to retain legacy pointer syntax, which would be on by default for any old projects being imported and off by default for new projects.

Probably it will be possible to define "new" autoptr as a macro like:

Code: ags

#ifdef NEW_COMPILER_OPTION
   #define __autoptr autoptr
#else
   #define __autoptr

[Calin Leafshade]

- new operator (done)
- keyword "builtin" to lock down any operations we don't want (creating new objects, extending, etc.)
- fix bugs (somewhat done, zero-sized structs should probably be tolerated)
- constructors (even this is iffy right now)

This looks like a good roadmap for the time being. Better not to add too much in one go. Adding user managed structs with constructors is plenty to be testing and digesting over the next few weeks.

[Crimson Wizard]

Well, yes, let's focus on smaller step at the moment to make an intermediate build and let people test.

[Gurok]

Crimson Wizard, I followed your instructions and fixed the branch (check the commit history now!):

https://github.com/gurok/ags/tree/331_new_user_struct

monkey_05_06, I think I fixed that autoptr managed struct example with a contained autoptr struct.

Am I right in assuming that allowing zero-byte dynamic user structs will be an engine change, CW? If so, do we just add another commit for the engine?

[monkey0506]

If the consensus is to push back the autoptr declarations then I won't keep pressing it for now, but I think the compiler will need a way to see the instances as not being pointers.

Gurok, do you mean you can now have nested user pointers within a user type with access modifiers within the same script in which the first struct is defined? I'll test this afternoon. Probably do some more extensive testing on my changes as well.

[Crimson Wizard]

Am I right in assuming that allowing zero-byte dynamic user structs will be an engine change, CW? If so, do we just add another commit for the engine?

This, and something else. There's one annoying problem that I skipped when making the patch. I just need some time to check which solutions are possible.

[Gurok]

If the consensus is to push back the autoptr declarations then I won't keep pressing it for now, but I think the compiler will need a way to see the instances as not being pointers.

Gurok, do you mean you can now have nested user pointers within a user type with access modifiers within the same script in which the first struct is defined? I'll test this afternoon. Probably do some more extensive testing on my changes as well.

Yes. That's exactly what I meant. Sorry, it was confusing even as I was typing it out.

I started looking into constructors tonight. I think the return after parsing a "new" statement should become conditional (e.g. if constructor, don't return, just keep running the rest of parse_sub_expr). We will also have to adjust the way functions are interpreted to allow them to see constructors. It's a little awkward, but I think safer than duplicating or refactoring that block of code.

CW, do you think we should do the locking down with "builtin" as a separate commit? I could see that hitting files many times, especially agsdefns.sh.

Also, CW, if you give me another patch, I guess I could rebase the branch. Getting good at this Git thing.

[Crimson Wizard]

CW, do you think we should do the locking down with "builtin" as a separate commit?

IMHO yes, because that is, actually, different feature (new keyword), although related to same kind of objects.

BTW, all I wanted to say about commits were these recommendations: http://www.adventuregamestudio.co.uk/forums/index.php?topic=50001.msg636481384#msg636481384

[monkey0506]

Yes. That's exactly what I meant. Sorry, it was confusing even as I was typing it out.

It's a rather specific and convoluted thing to try explaining in regular speech. Thanks for the clarification.

Also looking forward to constructors, thanks for your hard work. Delving into the compiler last night I realize it's pretty complex.

[RickJ]

Wow, all great stuff!  Are function/method pointers supported also?

[monkey0506]

Not yet Rick, but the engine already has the capability, we just have to come up with a way of adding the mechanic. I definitely think we should push for C#-style delegates, but that is a pretty major change, that should probably be saved for a future version.

[RickJ]

Well, if the engine already supports function pointers and pointers to objects are to be supported in the language then why not allow access to the object's methods via it's pointer reference?  If simply exposing the engine's current function pointer capabilities is achievable now then that ought to be the direction.  Then delegates could be implemented in the future whenever somebody gets around to it.  In the meantime the benefits of function pointers could be realized.

[Calin Leafshade]

uggnnnnhhhh no pointers.

Pointers are a low-level, direct memory referencing construct and are not suitable for a high-level scripting environment. Let's not cut corners.

[monkey0506]

I basically agree with Calin on this. Object "pointers" in AGS would be more aptly named references. This applies to their usage for built-in and user types alike.

C# delegates are more similar to a reference to a function than a C-style function pointer. This is why I suggested that style mechanic, because it most similarly matches the "pointer" system already in-place, and maintains a high-level approach to a lower-level problem.

[Gurok]

Are we going to keep going with the standards document? I know that we've settled on more immediate things for now, but it would be nice to get all of these ideas down and perhaps discuss a draft of how people want delegates or function pointers to work. Delegates would be keeping with C#, but strangely C#'s delegates always result in me checking MSDN to see if I've got the keyword voodoo in the right order.

I was speaking to Calin on IRC about how there are a few traps in AGS for new scripters that could be minimised by the language. For instance, it would be nice to have operator overloading for concatenating strings ("" + "") and maybe type coercion for ints/floats, or even a Number type.

[Crimson Wizard]

Internally, AGS does not have only one function pointer type, it has three:
1. Pointer to engine API function (safe parameter list)
2. Pointer to plugin function (unsafe parameter list, should eventually be deprecated)
3. Pointer to script function (combination of script id and address in byte-code array).

Therefore it is impossible to just export real memory address, because that will work only with API functions (and will be terribly unsafe), and won't work with script functions.
In any case this should be a complex object.

Keeping latter in mind, "pointer" variables could be done in two steps:
- a simple function pointer, which keeps only reference to one function;
- a real delegate; making ones will require limited update on top of work required to make the previous step; I think.


RE @Gurok, of course, design & planning should go first, solving implementation problems only after.