Changes to the AGS Scripting language

[monkey0506]

Double-post for an unrelated issue.

A thought just occurred to me here, but what should be the expected behavior if the user tries to create new instances of the built-in types? For example, the following code is now perfectly valid:

Code: ags

GUI *new_gui = new GUI;
Character *new_character = new Character;
Object *new_object = new Object;

Of course, while these are valid script objects, they're totally unusable. They all have invalid IDs, and other invalid read-only attributes. Attempting to display new_gui, for example, aborts with an error that an invalid GUI was specified. This makes sense of course, but should it really be deferred to a run-time crash to handle this?

Making matters even worse is the auto-pointer nature of the String class:

Code: ags

String s = new String;

May look perfectly fine, and it even compiles now! But when the run-time encounters this line, it crashes with an error message that a user object of 0 bytes was requested. This isn't very friendly.

We could just denote in the manual that the new keyword is not meant for any basic or built-in types, but nobody reads the manual anyway, so we'd end up with dozens of tech help threads due to run-time crashing.

It may actually prove useful to allow the dynamic creation of some of these built-in types, say, a temporary dummy character only needed in one room. However, this would require additional steps to ensure that these objects are created in a meaningful and useful state. Conversely, we may also need to have a way to disallow dynamic instanciation of certain types (for example, extra Game or Mouse objects wouldn't make sense).

[Calin Leafshade]

Game and Mouse should probably just be declared as static if they arent already.

[monkey0506]

Mouse.x and Mouse.y are non-static fields, only accessible through the mouse instance (hence my insistence that for the time being, the Mouse class not be used statically). Aside from that, the Game type only has static properties and methods, but therein is my point that creating instances of the type is useless.

[Crimson Wizard]

A thought just occurred to me here, but what should be the expected behavior if the user tries to create new instances of the built-in types?

This is indeed important.

We need to diverse classes that may be created by user (with new) and those that cannot be. It is hypothetically possible that some of the game objects (characters etc) will be dynamically created in future versions of AGS, but so far they cannot be.
And there are some classes that can be created dynamically, but this is performed by special methods like string creation and static functions (DateTime.Now). This is also possible that we may change this in future, but for now it should stay the same.

Trying to use "new" keyword with these types must cause compilation error.

I propose adding a keyword to define classes that cannot be created with "new". This could be "builtin", or "internal". Maybe someone has better name?

[Gurok]

Should we also use this keyword to prevent people from creating structs that extend built-in types? Right now (3.3.0) it's possible to do:

Code: ags

struct ExtendedCharacter extends Character
{
}

which gives you a struct where every instance maps to character[0].

If I had to choose one your suggestions, CW, I would go with "builtin". "internalstring" is already a modifier, so "internal" seems bad.

But...

I was thinking we could do this with two struct keywords, "abstract" and "sealed".

"abstract" could be used to prevent people from instantiating your struct directly (only through derived structs). The engine would be exempt from this rule, of course.

"sealed" would prevent the extends keyword from being used. Extender methods would still be allowed, making the term "sealed" a little shaky, but I think they're pretty good fits.

I know you don't typically see "abstract sealed" in the wild, but there's also typically no completely inaccessible object construction like this. MSDN says that abstract and sealed can't be combined in C# because they're contradictory. That's not entirely true when they're used as modifiers for a class. They would just result in relatively useless classes (normally).

[SpeechCenter]

Given the many additions to the language specification, I suggest you keep a list of those changes either in the original post or a separate wiki page so it's easier to go through the final result.
It's also important to review those final decisions to ensure future compatibility.

By the way, I would still consider changing the classes to static to indicate classes that cannot be instantiated, assuming backwards compatibility can be maintained. Hence it would allow using a static class modifier in the language.

[monkey0506]

By the way, I would still consider changing the classes to static to indicate classes that cannot be instantiated, assuming backwards compatibility can be maintained. Hence it would allow using a static class modifier in the language.

I definitely agree that since we already have a static keyword, that it makes sense to reuse it to allow pure static classes, like Game, Room, and System (and Mouse too, if we could get proper X and Y static attributes to replace the instance fields -- though this would break basically every active AGS project's scripts over a capitalization issue ).

[Crimson Wizard]

Given the many additions to the language specification, I suggest you keep a list of those changes either in the original post or a separate wiki page so it's easier to go through the final result.

We keep a list of changes in the form of Git history . I used that for preparing change list for 3.3.0.

"abstract" could be used to prevent people from instantiating your struct directly (only through derived structs). The engine would be exempt from this rule, of course.

I don't know... this will look weird. "Abstract" means something not real... its like using word "interface" for something what is not.

[Gurok]

Given the many additions to the language specification, I suggest you keep a list of those changes either in the original post or a separate wiki page so it's easier to go through the final result.

We keep a list of changes in the form of Git history . I used that for preparing change list for 3.3.0.

"abstract" could be used to prevent people from instantiating your struct directly (only through derived structs). The engine would be exempt from this rule, of course.

I don't know... this will look weird. "Abstract" means something not real... its like using word "interface" for something what is not.
E: On other hand, it is a class which is implemented in the engine, and not in script, which means that its implementation is separate. So... okay.

I was working from this http://msdn.microsoft.com/en-us/library/sf985hc5.aspx

I think "abstract" on structs would allow us to optionally have abstract methods down the track.

If you don't like "abstract", I think "builtin" is the next best suggestion. Static only works for things like Game. Character could not have a static modifier, because technically there are instances, just not user-creatable ones.

[Crimson Wizard]

I added this to previous post, but will re-post just in case:

E: (regarding using "abstract") On other hand, it is a class which is implemented in the engine, and not in script, which means that its implementation is separate. So... okay.

[Calin Leafshade]

I dont think you should use abstract because it has a real meaning in the paradigm and appropriating that for something unrelated would be bad.

Characters are *not* abstract classes.

I strongly suggest "builtin".

[Crimson Wizard]

Characters are *not* abstract classes.

I think they are, because they are not implemented in script.
We may say that there are Script character class (abstract, base) and its implementation in the engine (derived class).

E: On other hand, using "sealed" is ideologically incorrect, because Engine still derives an implementation from them...

Ok, I am still unsure . I will think more about this. Also let's see if there are more opinions.

[Calin Leafshade]

To me, an abstract class is a class that cannot have an instance of itself. There are clearly instances of characters even if the scripter cannot make one.

Abstracts are designed to be subclassed.

[Snarky]

If the engine does in fact use a subclass, then abstract might be appropriate. But then again it couldn't be sealed, as you say, CW... And if the class (err, or struct?) can in fact be instantiated by the engine, it's not abstract. What you need to make this work is a way to distinguish instantiation in engine code from attempts in user script.

It wouldn't be possible to simply limit the scope of the constructor (and make the class sealed/final), so it can be accessed by the engine but not the user script (cf. Java package scope)?

Otherwise, I think you'll have to make up a non-standard keyword. builtin, internal... it doesn't really matter as long as it doesn't conflict with anything else. Will the keyword even be seen by users, or only engine devs?

[Gurok]

Hrmm... abstract classes can't be instantiated directly (e.g. new Character()). There's no rule about never having an instance of one. The idea here is that the user sees the abstract class (Character) and the engine is providing a phony implementing class that it constructs.

[Gurok]

It wouldn't be possible to simply limit the scope of the constructor (and make the class sealed/final), so it can be accessed by the engine but not the user script (cf. Java package scope)?

Actually, I thought about having protected constructors for the built-in types. This doesn't solve the problem of another class using extender methods on a built-in class though.

Sorry for the double post.

Edit: "internal" might be used by C# so, semi-standard.

[Calin Leafshade]

internal is a good candidate.

In C# internal members/classes are only accessible from within an assembly. If one could consider the engine and script as separate "assemblies" then to say that a character is an internal class makes sense I think even if it's not exactly inline with the C# definition.

I think we will, at some point, make it possible to instantiate characters from within the script (we're nearly there anyway) so it's not a huge issue either way but i think internal is a very good fit.

[Snarky]

This doesn't solve the problem of another class extending a built-in class though.

It would if you made the class sealed/final.

Yeah, I'm more and more convinced that what's needed is a notion of horizontal scope, an access modifier that doesn't depend on class structure but on the location of the code, like "internal" in C# or the default/package scope in Java.

Edit: The alternative would be to not worry about it and just prepare to point and laugh at the idiots who try to instantiate internal classes.

[Gurok]

This doesn't solve the problem of another class extending a built-in class though.

It would if you made the class sealed/final.

Yeah, I'm more and more convinced that what's needed is a notion of horizontal scope, an access modifier that doesn't depend on class structure but on the location of the code, like "internal" in C# or the default/package scope in Java.

Yeah, sorry, Snarky. I edited that. I meant to say using extender methods. If you use an extender method, you have that protected scope and so could call the constructor.

I can code up "internal" or "builtin". It's no problem. I just thought the two-word combo had more utility.

[Snarky]

Ah, good point!

[Calin Leafshade]

A maybe tangential point, but does it make sense to have a "managed struct". Aren't structs generally considered to be value types on the stack?

Maybe renaming "managed struct" to "class" would be wise semantically?

Since we are essentially designing a language here should we have some kind of symposium or somethign and draw up a formal specification so we dont run into absurdities later?

[Crimson Wizard]

Since we are essentially designing a language here should we have some kind of symposium or somethign and draw up a formal specification so we dont run into absurdities later?

And I was suggesting this nearly two years ago. .


...two years... time's flying

[monkey0506]

Here's my thoughts on the discussion so far:

* static structs should be allowed where they make sense, like Game, Room, and System.
* The Mouse.x and Mouse.y instance fields should be replaced by static properties, and the struct made static.
* The compiler will also then need a fix for accessing non-static members of a static member, e.g., System.AudioChannels[3].PlayingClip
* The new keyword internal should be added for structs that cannot be instanciated using keyword new
* internal structs should have access to a protected constructor, and should be able to return managed instances of the class by invoking it (e.g., via a static Create method, where appropriate)
* If reasonably possible, every existing "managed struct" should be revised to be an "autoptr managed struct" (this seems to work for user types already!)
* "autoptr managed struct" should be replaced by the new keyword class, a nullable reference type

Expanding from my earlier sample code, this feels much nicer:

Code: ags

function game_start()
{
  DynamicAttribute da1 = DynamicAttribute.Create();
  da1.DisplayInfo();
  DynamicAttribute da2 = DynamicAttribute.Create(42, "3.14", "Hello World!");
  da2.DisplayInfo();
  da1.iProperty = 24;
  da1.fProperty = 4.13;
  da1.sProperty = "!dlroW olleH";
  da1.DisplayInfo();
  DynamicAttribute array[] = CreateDynamicAttributeArray(5);
  array[3].iProperty = 1234;
  array[3].DisplayInfo();
}

I avoided dealing with "pointers" by defining the type as "autoptr managed struct DynamicAttribute" (hence my extension of Calin's suggestion to rename it to "class").

This will break a lot of things in user scripts, but I think it's necessary moving forward to develop the language.

Edit: btw, inb4wut -- I just wanted to point out, before anyone gets all flustered and confused, that AGS already has the autoptr keyword, but the only case where it's used is the String type. I'm not sure if it's fit for general use, which is why I said, "If reasonably possible...". As I said though, it seems to run without causing any problems.

[Calin Leafshade]

I think I agree with everything monkey says except maybe the factory methods.

I think the factory methods were included as a stop gap because there was no new keyword.

So where we have:

Code: ags

DynamicSprite *d = DynamicSprite.Create(320,180);

This should be replaced by

Code: ags

DynamicSprite d = new DynamicSprite(320,180);

Of course the factory method can be left in place for compatibility but it should simply be a wrapper for the new syntax.

Any class marked as internal should simply not be able to be instantiated in the script domain. These would currently be Character, Object, Hotspot, AudioClip, AudioChannel and Region. (might be more)

-----

I think structs should be value types and classes should be ref types and the pointer syntax should simply be dropped if thats possible.


Edit: Thinking about this is fun, i'll draw up some kind of doc soonish with my thoughts en masse.

[Crimson Wizard]

* The new keyword internal should be added for structs that cannot be instanciated using keyword new
* internal structs should have access to a protected constructor, and should be able to return managed instances of the class by invoking it (e.g., via a static Create method, where appropriate)

AFAIK in C# (if we take one for analogies) "internal" structs are those that cannot be used by external module. Here we have a struct that may be used externally (in scripts) but cannot be created with "new" by scripts.
Also, as Gurok mentioned above, protected members should be (are?) accessible in extender functions.
Should not constructor be "internal" instead?

This will break a lot of things in user scripts, but I think it's necessary moving forward to develop the language.

We may add another backwards compatibility option to compiler. It will ignore single "*" sign after "autoptr" type, unless the type is string. Or, something like that.
This is important, because our next update will include other features too, and people will want to have them without having to fix tonns of scripts.

[Calin Leafshade]

Ive changed my mind about structs being value types. It would be confusing for newbies to pass an object and get a copy of it instead.

So what would the distinction between a struct and a class actually be?

Is a struct just an automatically instantiating class?

Code: ags

struct myStruct {
    int i;
};

class myClass {
    int i;
};

myStruct myStructInstance; // this is instantiated
myClass myClassInstance; // this is a null reference

What is the utility of this distinction? Why would someone, in practice, ever use a struct?

[Crimson Wizard]

What is the utility of this distinction? Why would someone, in practice, ever use a struct?

Some newbie scripter who needs a simple way to group variables and don't want to have annoying "null reference" problem .
+ backwards compatibility...

[SpeechCenter]

* The new keyword internal should be added for structs that cannot be instanciated using keyword new
* internal structs should have access to a protected constructor, and should be able to return managed instances of the class by invoking it (e.g., via a static Create method, where appropriate)

AFAIK in C# (if we take one for analogies) "internal" structs are those that cannot be used by external module. Here we have a struct that may be used externally (in scripts) but cannot be created with "new" by scripts.
Also, as Gurok mentioned above, protected members should be (are?) accessible in extender functions.
Should not constructor be "internal" instead?

That would be confusing indeed. In C# static classes mean they cannot be instantiated (http://msdn.microsoft.com/en-us/library/79b3xss3.aspx). So either support that or define private/protected constructor.

[Snarky]

No, just like "abstract sealed", static is wrong because the engine is in fact creating instances of the class (and providing those instances to the user). CW had it right that it's the constructor that should be declared internal; by C# analogy (considering the engine and the user scripts as different assemblies) that would have the desired effect.

[Gurok]

Hey! Abstract doesn't mean someone can't create an instance for you. It just means you can't directly construct it.

I believe monkey_05_06 was suggesting to use static only for classes it applies to (i.e. those with all-static members). This makes sense, I guess, but I don't think it's 100% necessary.

CW has it right. Internal is to keep a property within the assembly it was declared in. If we had a property that meant "you can't construct this and can't use the keyword extends on it", I would prefer "builtin".

Regarding structs and the point of them v/s managed structs, I think it's actually a good way to ease people into the concept of objects. As CW said, there are no nulls. It's a very safe place to play.

[Calin Leafshade]

The problem with declaring the constructor internal is that it means all classes require a constructor and allowing a default, empty constructor would be cool.

I think defining the whole class as internal would be better. i.e "this class cannot be instantiated in the script domain"

EDIT AFTER GUROK: On balance defining the whole class as "builtin" is probably better.

[Crimson Wizard]

BTW, an interesting thing is that currently user can implement any function from builtin classes.
For example, I can write in script:

Code: ags

function Character::ChangeRoom(int room, int x, int y)
{
  Display("bla");
}

Did anyone know that?
This works because script implementation overrides builtin one when registered.
Do we need to protect builtin classes from such atrocities?

[Calin Leafshade]

No, thats *awesome*

Can you override Say and have dialogs use it correctly?

EDIT: Speaking of overrides how difficult would overloading functions be?

EDIT EDIT: (Although we probably need to be able to provide access to the base function if overridden.

[Gurok]

If functions are implemented in such a way, AFAICT they're allowed but they just get ignored. The engine's implementation still gets called. Are you seeing different behaviour, CW?

[Crimson Wizard]

Can you override Say and have dialogs use it correctly?

No, because dialogs do not call "Say" via script API, obviously, they call program functions directly (and THAT is the problem).

If functions are implemented in such a way, AFAICT they're allowed but they just get ignored. The engine's implementation still gets called. Are you seeing different behaviour, CW?

Actually I do.
I typed "player.ChangeRoom(0)", and "bla" was displayed instead of room change.

[Gurok]

Can you override Say and have dialogs use it correctly?

No, because dialogs do not call "Say" via script API, obviously, they call program functions directly.

If functions are implemented in such a way, AFAICT they're allowed but they just get ignored. The engine's implementation still gets called. Are you seeing different behaviour, CW?

Actually I do.
I typed "player.ChangeRoom(0)", and "bla" was displayed instead of room change.

Oh right. I was testing Say, naturally.

Code: ags

function Character::Say(const string message, ...)
{
	Display("Here's the message: %s", message);
}

[Snarky]

Personally I say get rid of structs as non-nullable types, and lose the pointer syntax completely. A bit of work for devs updating scripts now is worth it for the simplification it brings.

Hey! Abstract doesn't mean someone can't create an instance for you. It just means you can't directly construct it.

I believe monkey_05_06 was suggesting to use static only for classes it applies to (i.e. those with all-static members). This makes sense, I guess, but I don't think it's 100% necessary.

CW has it right. Internal is to keep a property within the assembly it was declared in. If we had a property that meant "you can't construct this and can't use the keyword extends on it", I would prefer "builtin".

Regarding structs and the point of them v/s managed structs, I think it's actually a good way to ease people into the concept of objects. As CW said, there are no nulls. It's a very safe place to play.

Gurok, I was responding to SpeechCenter, who suggested using "static" as the "builtin" keyword. As for abstract, we already discussed this upthread.

Using "internal" on the constructor to make it uninstantiable by user code would be consistent with C#. Contra Calin, I'm not sure I see the problem with requiring the former managed classes to declared an explicit "internal" constructor, and I think it might be useful to have a keyword that could be used to make specific other methods engine-only as well, but if you prefer to tag the class rather than the constructor I agree something like "builtin" might be better.

Overriding methods is hella cool (and sort of obsoletes extender functions), but there's no super() yet, is there? That's probably necessary to make it super-useful.

[Crimson Wizard]

Overriding methods is hella cool (and sort of obsoletes extender functions), but there's no super() yet, is there? That's probably necessary to make it super-useful.

That was not overriding in the sense of type hierarchy, but in the sense of... I dunno, substitution (like with delegates?).
Therefore word "super" won't help here, because engine does not provide parent class for Character, but its implementation. We would need word... "internal"? Like "internal::ChangeRoom", unless that's too weird.

Overriding in the sense of polymorphism was already supported with extending structs (to some degree): http://www.adventuregamestudio.co.uk/wiki/Extender_Methods_mean_Polymorphism!#Polymorphism

[Snarky]

Hmmm... you mean that the override won't just affect instances of your class, but all instances of the original class throughout the code? Yes, that is a bit different from how inheritance override usually works. I'm starting to see what you mean about planning out the language design, because this is starting to sound a little kludgey.

I think you could still argue that conceptually super() (or base(), going off C#) is an easier, more familiar term, even if it works slightly different from the Java way. But keep going too far down that path of "close enough", and the whole thing will be an even stranger, more non-standard mess than it already is.

[Crimson Wizard]

Not every hidden feature of AGS script is worth to be kept and developed further, some work simply because it was coded this way. While it may be fun to uncover them, their usefullness may be dubious.

I am all for moving towards tidy language.
The backwards compatibility is not a biggest issue, because in the worst case we may make more compiler switches in addition to the ones we already have (non-OO functions, etc).

As for keywords and concepts, I believe we first need to have a list of which entities, technically, we have/want to have, because this list defines which keywords we need to use and control them.

1. Storage / scope.
We have following places to store objects
* global memory: exists until program exit
* local memory (stack): exists until function returns
* managed memory: exists until reference count reaches zero

2. Implementation.
* implemented in separate assembly (e.g. engine)
* implemented in this assembly

3. Place in hierarchy
* can be inherited
* cannot be inherited

4. Access to members.
etc.

[monkey0506]

internal constructors would probably work for what I was proposing. I still think that static classes make sense as an indication that a type is only used to organize related data (methods and properties), all of which are static themselves, of course.

[Calin Leafshade]

I think we should also discuss the scoping of AGS script as currently its a bit archaic.

Could we feasibly add a two pass compiler and scope modifiers?

so we could have:

Code: ags

public class MyGlobalClass { };

class myPrivateClass {};

public function MyPublicFunc() { }

function myPrivateFunc() {}

So the local scope would be limited to the script but stuff can be exported with a simple modifier?

[Calin Leafshade]

I have started a standards document.

https://docs.google.com/document/d/1oW7dqB_GjbmlgPVoYzvBOyQevBPBjzc_jJPY2D5sDH8/edit?usp=sharing

Anyone can comment but if someone wants to do some writing then let me know your gmail address and i'll add you to the contributors.

I think a more formal process like this will allow us to do this quicker and more concisely.

I think discussion should essentially begin with some kind of proposed standard in the doc, followed by a discussion on that section on the forums, followed by edits, followed by consensus.

How does this sound to everyone?

[Crimson Wizard]

First, may I ask of moving the related posts to another thread? It begins to be a separate discussion.

Second, I want to remind about what I said about implementing functions of builtin classes in script. I believe this should be denied. Class must have a modifier that would prevent game developer from "redefining" its member functions in script.

Third, there's one thing that bothers me. Please keep in mind, that AGS script is being used by newbie scripters who may have difficulties learning all those numerous keywords. Please, don't make AGS script require typing scope keyword before every function . Or think of compiler option that would allow to skip these.

[Calin Leafshade]

re: third point.

I think things would be assumed private first, like now.

What's simpler?

Code: ags

public function ThisIsGlobal() {}

or

Code: ags

//in header

import GlobalFunction;

// in script

function GlobalFunction() {}

The former is better because you can see the function is public immediately and its obvious how to make a function public.

the latter is bad because it's confusing and unclear.

My way is better for newbies i think.

[Crimson Wizard]

For some reason I thought you are talking about struct member functions.

Regarding global functions. I think you are confusing two concepts. Public/private is a concept of access. Import/export is a concept of location.
In AGS "import" does not mean "public" (in AGS everything is public by default). Its meaning is closer to C++ "extern": function/variable defined elsewhere.

[Calin Leafshade]

I will rephrase.

I think there should be a better way of accessing members from different scripts.

Ideally, imho, every member of a script should be scoped locally to that script (as is the case now) unless the member is prefixed by the modifier "public" or perhaps better "global".
If this is done then that member is accessible globally to all scripts.

Code: ags

//scriptA

int privateInt = 0;
global int PublicInt = 0;

//scriptB

function foo()
{
    Display("%d", privateInt); // COMPILE ERROR
    Display("%d", PublicInt); // compiles fine.
}

[Crimson Wizard]

At the moment the sole purpose of script header is to make things accessible from other places. You may say that everything put in header is automatically public.
In such case, we either do not need headers anymore, or we do not need access modifiers for global functions/structs.

[Calin Leafshade]

Given the choice, I would remove headers.

They are a complex idea for newbies. Ideally we should move the responsibility of the header to the compiler, not the user.
I think the headers are a stop gap for the lack of a two-pass compiler.

[monkey0506]

I agree that the headers can cause more confusion than they might be worth. I've seen numerous beginners putting actual definitions in the header files, not understanding the import/export keywords. I find them perfectly easy to work with, and as CW said, comparable to C++'s extern, but if we can lower the learning curve for newbies then I'm all for it. Calin's suggestion for a global keyword seems reasonable to me, and it pretty clearly explains what it does.

I also understand we shouldn't add new keywords without consideration first, but I support this, and the two-pass compiler suggestion.

[SpeechCenter]

Removing the headers is a good idea, but what about existing code? Will it continue to function or require conversion?

As for keywords, I would try to stick to C# as much as possible, as long as it makes sense. So perhaps 'public' can still be used.
(another reason to restrict the number of new keywords is that it increases the possibility someone uses this keyword in existing code and then the code won't compile)

[monkey0506]

Another minor issue with managed user objects: If you try to create a struct B which has a protected or writeprotected member (field) of type struct A in the same script where struct A has been defined, then the compiler throws a syntax error that struct A is not a type.

Code: ags

// Script.ash
autoptr managed struct A
{
};

autoptr managed struct B
{
  A a1; // works fine
  protected A a2; // throws compile error
  writeprotected A a3; // throws compile error
};

This can currently be dealt with by simply moving the second struct definition to another script:

Code: ags

// ScriptA.ash
autoptr managed struct A
{
};

// ScriptB.ash
autoptr managed struct B
{
  protected A a; // works fine
};

Also, while I have your attention, might I ask how difficult it would be to allow dynamic arrays as struct members? We can already have pointer members to built-in and user defined types, so simply storing the pointer to the array seems as though it wouldn't be terribly difficult to manage...?

[tzachs]

AFAIK in C# (if we take one for analogies) "internal" structs are those that cannot be used by external module. Here we have a struct that may be used externally (in scripts) but cannot be created with "new" by scripts.

Exactly, therefore Internal is not a good candidate in my opinion.

That would be confusing indeed. In C# static classes mean they cannot be instantiated (http://msdn.microsoft.com/en-us/library/79b3xss3.aspx). So either support that or define private/protected constructor.

But static implies you can have only one, this is not the case here, you do have instances.

Protected or internal constructors are an option but would not be friendly to the AGS scripters who has no access to the inners of the built-in types, and so will have to learn by compilation errors.

The "builtin" keyword might sound like a good option, but I think that having to invent a new keyword should ring an alarm that maybe we're doing something wrong here, maybe we're thinking we're dealing with a new beast that was not seen before, but that's not the case.
And I think the term that is missing here is interfaces.
Interfaces can be used but not instanciated. Replacing our built in types with built in interfaces (only in the API that's exposed to AGS script, the engine will use the same types that will just implement those interfaces) will give greater readability to the fact they can't be instanciated, and will also give some other extensibility benefits. A user will be able to provide a different implementation to the character interface, and it will work out of the box with modules that use the character interface, for example. And if we have sub interfaces like a position interface that is implemented by the character, object and gui interfaces, it will make the code much more flexible (the tween module could be much simplified, for example).

[Snarky]

Uh-oh! That's sounding even more complicated. Even though the logic is sound, let's try to avoid reinventing Java/C# if we can, eh?

I'd again like to repeat my proposal to not do anything, and just let people who try to construct new instances of the built-in types deal with the resulting errors.

[Crimson Wizard]

Yes, let's not overcomplicate things, I am sure there's a way to do this simply.

[Calin Leafshade]

I think interfaces would not be semantically appropriate and referencing Characters and Objects directly rather than by proxy is more user-friendly.

I think we need to remember that this keyword will not be available to the scripter so it's kind of irrelevant in that sense.

Just pick a word and throw an error if the user tries to instantiate a class with that keyword. I don't think it need be any more complicated than that.

Also, the goal would be, at some point, to drop this restriction anyway and allow the user to instantiate them.. so it's not like its a permanent thing anyway.

[Crimson Wizard]

Also, the goal would be, at some point, to drop this restriction anyway and allow the user to instantiate them.. so it's not like its a permanent thing anyway.

I agree with this.

[monkey0506]

Discovered another issue, though this might be one that we can reasonably write off. If a user struct only has imports, and does not actually define any members, then you cannot create a new instance of it ("requested user object of size 0"):

Code: ags

autoptr managed struct MyStruct
{
  readonly import attribute int MyProperty;
};

function game_start()
{
  MyStruct i = new MyStruct; // crashes
}

If there is a reasonable use case for instanciating a user struct like this (can't really think of one off the top of my head, the imports could all just be static) then it could be rectified by simply adding an unused char to the struct definition:

Code: ags

autoptr managed struct MyStruct
{
  protected char __unused; // $AUTOCOMPLETEIGNORE$
  readonly import attribute int MyProperty;
}

[Crimson Wizard]

Discovered another issue, though this might be one that we can reasonably write off. If a user struct only has imports, and does not actually define any members, then you cannot create a new instance of it ("requested user object of size 0"):

Hmm... we may allow to create one. There's a real object created in any case (in engine), the user data is stored in internal buffer, which may be zero.

[monkey0506]

I'm looking into making the built-in instantiable types into autoptr types as a step toward deprecating the pseudo-pointer syntax. Sticking the keyword in front of the struct definitions was the easy bit, but now I have to figure out how to differentiate between the autoptrs and the actual instances that are being imported (like cEgo, which is a Character instance, not a Character*). I presume at this point I need to be looking in the engine code. Any pointers appreciated. Once I get it figured out I can submit a pull request for it.

[Crimson Wizard]

I'm looking into making the built-in instantiable types into autoptr types as a step toward deprecating the pseudo-pointer syntax.
<...>
Once I get it figured out I can submit a pull request for it.

Wait a minute... are you going to put all this to 3.3.1? We would need a compiler switch to retain backwards compatibility... or something...
Does it have a true benefit that make it worth such change? Remember, you will practically force people to edit tonns of scripts and maybe some modules.
Also! this may screw some plugins, if they create script declarations with pointers.
Maybe it is better to delay this until some "groundbreaking" version?

PS. I am really getting worried about this... not that I think that language will become worse without * sign, but we are making AGS for certain people, not for the pleasure of hacking the program.

Sticking the keyword in front of the struct definitions was the easy bit, but now I have to figure out how to differentiate between the autoptrs and the actual instances that are being imported (like cEgo, which is a Character instance, not a Character*). I presume at this point I need to be looking in the engine code. Any pointers appreciated.

I think its in "AGS.Editor.Tasks.AppendCharactersToHeader()".

[Gurok]

I agree with you, CW. I've felt a little paralysed by the discussion going on here. I really don't want to put any changes that would break scripts into 3.3.1. I want to go back to a *simpler* plan for now:
- new operator (done)
- keyword "builtin" to lock down any operations we don't want (creating new objects, extending, etc.)
- fix bugs (somewhat done, zero-sized structs should probably be tolerated)
- constructors (even this is iffy right now)

Save the overhaul for 3.4 or even 3.5. I think we shouldn't worry about needing to remove "builtin" in the future. The rule should be: anything that doesn't get a syntax highlight isn't officially supported, and just leave it at that.

[monkey0506]

I did, of course, intend to add an option to retain legacy pointer syntax, which would be on by default for any old projects being imported and off by default for new projects. It wasn't some careless or thoughtless hacking either, but a step toward implementing a meaningful class keyword, as per my prior suggestion. I don't think that optionally removing the pseudo-pointer syntax (while preserving full compatibility for its use) constitutes such a drastic change that we should be forced to put it off.

Sticking the keyword in front of the struct definitions was the easy bit, but now I have to figure out how to differentiate between the autoptrs and the actual instances that are being imported (like cEgo, which is a Character instance, not a Character*). I presume at this point I need to be looking in the engine code. Any pointers appreciated.

I think its in "AGS.Editor.Tasks.AppendCharactersToHeader()".

I wasn't referring so much to where the instances are included in the header, but the fact that they are importing the actual instances rather than importing pointers. If the types are made to be autoptr, then the imports are seen as pointers, not the actual instances.

[monkey0506]

Created an unrelated pull request, to allow dynamic arrays within structs:

Code: ags

struct MyStruct
{
  int Array[];
};

function game_start()
{
  MyStruct ms;
  ms.Array = new int[5];
  ms.Array[3] = 42;
}

Will definitely require further testing, but preliminary testing (including saving and loading tests) show that this should be working.

[Crimson Wizard]

I did, of course, intend to add an option to retain legacy pointer syntax, which would be on by default for any old projects being imported and off by default for new projects.

Probably it will be possible to define "new" autoptr as a macro like:

Code: ags

#ifdef NEW_COMPILER_OPTION
   #define __autoptr autoptr
#else
   #define __autoptr

[Calin Leafshade]

- new operator (done)
- keyword "builtin" to lock down any operations we don't want (creating new objects, extending, etc.)
- fix bugs (somewhat done, zero-sized structs should probably be tolerated)
- constructors (even this is iffy right now)

This looks like a good roadmap for the time being. Better not to add too much in one go. Adding user managed structs with constructors is plenty to be testing and digesting over the next few weeks.

[Crimson Wizard]

Well, yes, let's focus on smaller step at the moment to make an intermediate build and let people test.

[Gurok]

Crimson Wizard, I followed your instructions and fixed the branch (check the commit history now!):

https://github.com/gurok/ags/tree/331_new_user_struct

monkey_05_06, I think I fixed that autoptr managed struct example with a contained autoptr struct.

Am I right in assuming that allowing zero-byte dynamic user structs will be an engine change, CW? If so, do we just add another commit for the engine?

[monkey0506]

If the consensus is to push back the autoptr declarations then I won't keep pressing it for now, but I think the compiler will need a way to see the instances as not being pointers.

Gurok, do you mean you can now have nested user pointers within a user type with access modifiers within the same script in which the first struct is defined? I'll test this afternoon. Probably do some more extensive testing on my changes as well.

[Crimson Wizard]

Am I right in assuming that allowing zero-byte dynamic user structs will be an engine change, CW? If so, do we just add another commit for the engine?

This, and something else. There's one annoying problem that I skipped when making the patch. I just need some time to check which solutions are possible.

[Gurok]

If the consensus is to push back the autoptr declarations then I won't keep pressing it for now, but I think the compiler will need a way to see the instances as not being pointers.

Gurok, do you mean you can now have nested user pointers within a user type with access modifiers within the same script in which the first struct is defined? I'll test this afternoon. Probably do some more extensive testing on my changes as well.

Yes. That's exactly what I meant. Sorry, it was confusing even as I was typing it out.

I started looking into constructors tonight. I think the return after parsing a "new" statement should become conditional (e.g. if constructor, don't return, just keep running the rest of parse_sub_expr). We will also have to adjust the way functions are interpreted to allow them to see constructors. It's a little awkward, but I think safer than duplicating or refactoring that block of code.

CW, do you think we should do the locking down with "builtin" as a separate commit? I could see that hitting files many times, especially agsdefns.sh.

Also, CW, if you give me another patch, I guess I could rebase the branch. Getting good at this Git thing.

[Crimson Wizard]

CW, do you think we should do the locking down with "builtin" as a separate commit?

IMHO yes, because that is, actually, different feature (new keyword), although related to same kind of objects.

BTW, all I wanted to say about commits were these recommendations: http://www.adventuregamestudio.co.uk/forums/index.php?topic=50001.msg636481384#msg636481384

[monkey0506]

Yes. That's exactly what I meant. Sorry, it was confusing even as I was typing it out.

It's a rather specific and convoluted thing to try explaining in regular speech. Thanks for the clarification.

Also looking forward to constructors, thanks for your hard work. Delving into the compiler last night I realize it's pretty complex.

[RickJ]

Wow, all great stuff!  Are function/method pointers supported also?

[monkey0506]

Not yet Rick, but the engine already has the capability, we just have to come up with a way of adding the mechanic. I definitely think we should push for C#-style delegates, but that is a pretty major change, that should probably be saved for a future version.

[RickJ]

Well, if the engine already supports function pointers and pointers to objects are to be supported in the language then why not allow access to the object's methods via it's pointer reference?  If simply exposing the engine's current function pointer capabilities is achievable now then that ought to be the direction.  Then delegates could be implemented in the future whenever somebody gets around to it.  In the meantime the benefits of function pointers could be realized.

[Calin Leafshade]

uggnnnnhhhh no pointers.

Pointers are a low-level, direct memory referencing construct and are not suitable for a high-level scripting environment. Let's not cut corners.

[monkey0506]

I basically agree with Calin on this. Object "pointers" in AGS would be more aptly named references. This applies to their usage for built-in and user types alike.

C# delegates are more similar to a reference to a function than a C-style function pointer. This is why I suggested that style mechanic, because it most similarly matches the "pointer" system already in-place, and maintains a high-level approach to a lower-level problem.

[Gurok]

Are we going to keep going with the standards document? I know that we've settled on more immediate things for now, but it would be nice to get all of these ideas down and perhaps discuss a draft of how people want delegates or function pointers to work. Delegates would be keeping with C#, but strangely C#'s delegates always result in me checking MSDN to see if I've got the keyword voodoo in the right order.

I was speaking to Calin on IRC about how there are a few traps in AGS for new scripters that could be minimised by the language. For instance, it would be nice to have operator overloading for concatenating strings ("" + "") and maybe type coercion for ints/floats, or even a Number type.

[Crimson Wizard]

Internally, AGS does not have only one function pointer type, it has three:
1. Pointer to engine API function (safe parameter list)
2. Pointer to plugin function (unsafe parameter list, should eventually be deprecated)
3. Pointer to script function (combination of script id and address in byte-code array).

Therefore it is impossible to just export real memory address, because that will work only with API functions (and will be terribly unsafe), and won't work with script functions.
In any case this should be a complex object.

Keeping latter in mind, "pointer" variables could be done in two steps:
- a simple function pointer, which keeps only reference to one function;
- a real delegate; making ones will require limited update on top of work required to make the previous step; I think.


RE @Gurok, of course, design & planning should go first, solving implementation problems only after.