Stack Overflow Error, Pointers and Writable Properties - help wanted!

[edmundito]

I didn't want to post this on the 2.71 thread because it really has to do with several ags versions.

My question is, how many recursive calls can AGS do? Someone tried to make a movie with ron movies, and he got a stack overflow error:

---------------------------
Adventure Game Studio
---------------------------
An internal error has occured. Please note down the following information.
If the problem persists, contact Chris Jones.
(ACI version 2.71.894)

Error: run_text_script1: error -6 running function 'btnRunScript_Click':
Error: call stack overflow, recursive call problem?
(and more...)


---------------------------
OKÃ,  Ã, 
---------------------------

It says to contact chris jones, so I am.

My script does load the RON Movies script file recursively. Let me know if there is a workaround to this problem or do I have to rewrite my script to do some iterative calls?

Scroll down for help question...


[edit by scorpiorus to change and expand the subject name]

[Gilbert]

I think CJ mentioned a 4KB (or something) size of the stack, so the number of levels of recursive calls might depend on how many local variable space those functions need. I'll recommend, if possible, move as many variable declarations with values shared across calls (unless those that you really need a fresh copy each time the function is called) as possible outside of function contents.

[edmundito]

I've been comparing movies for a few hours and as far as I can tell, my movie does close to 37 recursive calls on one specific function, where the movie that crashes does close to 41 calls on the same function. I think this is where the problem lies.

The reason why this is happening is because it saves everything on a parse tree. It's easier for me to evaluate the code that way. Hmm... I'd like to know more about the stack information, though.

[Gilbert]

Can you post the variable declaration lines of that function here ?
(Of course, apart from the variables, there're lots of other overheads which eat the stack I think).

I think some better possible solutions are:
1. Introduce some mechanics into the function which limit the number of recuring (like recording the level of recursing with a variable, which is zeroed in rep_ex every game loop, and everytime a function is run it's incremented by 1, and don't run another function anymore functions if the counter reaches a certain number), or
2. Try to optimise your functions like I mentioned in the last post, or
3. Bug CJ to increase the stack size, or
4. Don't use recursive calls at all.

[edmundito]

Yep, I choose #2, basically here's some pseudocode on what it's doing:

Tree* get_things
   Tree*root
   root.type = thing
   root.left = get_one_thing()
   if (there_are_more_things) then
       root.right = get_more_things() #stack problem starts here
   return root

The tree result is:

     thing
    /       \
  ...      thing
           /      \
         ...     thing
                 /      \
               ...      etc.

[Pumaman]

AGS currently allows the call stack to be 50 levels deep, so if you have a recursive function that calls itself more often than that you'll get the error. Additionally, as Gilbert says the stack size is set at 4 KB so if the recursive function declares a lot of local variables you could reach the limit that way, too.

[edmundito]

Got it. That's what happens when I assume that AGS has a stack as big as C or even Java programs.

So um, I tried correcting the problem last night, and it's a bit more difficult than I thought. If anyone wants to help out with a proposed solution, please post it. The pseudocode mentioned earlier is my original function, since my parser for RON Movies is based on recursive descent.

My seond try looked like this, I was half asleep when I did it, and It's been a while since I've messed with pointers and recursion.

function Tree* get_things()
   Tree*root, *temp
   root = newTree()
   temp = root;
   while(there_are_more_things)
      temp.left = get_one_thing()
      temp.type = thing
      if (there_are_more_things) then
          temp.right = newTree()
          temp = temp.right
   return root

With that approach, root and temp match (in address), root.left and temp.left match, and root.right and the temp.right match. But root.right.right and temp.right.right will not match.

[Kweepa]

The pseudocode looks reasonable for what you're trying to do (as in the diagram above), assuming that newTree() returns a properly initialized Tree.

What I don't understand is why this structure needs a tree at all. It seems like a mighty unbalanced tree. Perhaps you could show the stream of things that get_one_thing() returns and the tree structure that results and explain why you need it in a tree rather than just a list.

I also don't understand the last two lines of your post. Are you saying there's something wrong with the implementation? Or does an on-paper run-through given some input data result in an error? If so perhaps you could walk us through it.

[edmundito]

The pseudocode looks reasonable for what you're trying to do (as in the diagram above), assuming that newTree() returns a properly initialized Tree.

Ah yes, you would think so, but I think that there must be an error in my managed tree plugin, because basically root.right.right and tree.right.right will yield completely different memory addresses.

What I don't understand is why this structure needs a tree at all. It seems like a mighty unbalanced tree. Perhaps you could show the stream of things that get_one_thing() returns and the tree structure that results and explain why you need it in a tree rather than just a list.

Because it's using a recursive descent parser to read the code written for ron movies. it's quite easy to interpret the code, plus it's the only way I know how to parse for now, plus I already wrote the ags plugin for a tree. I could also use a list or stack to store some of these things, but that would mean that I would have to write a second plugin/add to the exsisting plugin, wouldn't it?

I also don't understand the last two lines of your post. Are you saying there's something wrong with the implementation? Or does an on-paper run-through given some input data result in an error? If so perhaps you could walk us through it.

Yep, as I mentioned, there must be something funky going on with my plugin code... I'm looking at it to see what's going on. I think this is the problem to all of this.

[edmundito]

Ok, I've been looking at this for a while, and I believe I figured out where the problem lies:

function Tree* get_things()
   Tree*root, *temp
   root = newTree()
   temp = root;
   while(there_are_more_things)
      temp.left = get_one_thing()
      temp.type = thing
      if (there_are_more_things) then
          temp.right = newTree()
          temp = temp.right
   return root

That's where all integrity is lost, and temp.right and root.right go their separate ways. But why is it not doing what it's really supposed to do? Well, it definitely has to do with the plugin. Let's look inside:

For AGS, the Tree is defined as:
managed struct Tree {
   import static Tree* create();
   import attribute int data;
   import attribute Tree *left;
   import attribute Tree *right;
};

create() allocates memory and registers on ags's managed objects, so that it can do the garbage collection. Every attribute must have a get_ and set_ function, depending on the action performed on the attribute. For right, the get_right function, looks like this:

Code: ags

Tree* Tree_get_right(Tree* obj) {
	return obj->right;
}

So, when we do temp = temp.right, it's actually doing something more like temp = Tree_get_right(temp); because tree.right is an attribute. The other way around, for tree.right = newTree() would be Tree_set_right(temp, newTree());*** This is somehow quite different from having two true (non-attribute) Tree objects, ie:

Tree* a;
Tree* b = Tree.create();
a = b;

This is where I step into the unknown, because I don't know the internals of AGS to determine why a Tree attribute and a Tree behave differently. Chris, could you provide any information that might be useful? Is there something I'm missing?


***

Code: ags

void Tree_set_right(Tree* obj, Tree* newright) {
    if(obj==0 || newright==0) return;
    obj->right = newright;
}

[Scorpiorus]

I've just come across this rather interesting topic...

I can not say for sure, as I haven't even tested pointers represented as writable properties, but I'd guess it's possible that an attribute pointer is a bit different from a script pointer.

While script pointers do support the object management, the attribute pointers probably don't. If that's true then the core of the problem may be the following line:

temp.right = newTree()

Here "newTree()" creates a new instance and returns its address so that "temp.right" could catch it. But if "temp.right" is an ordinary pointer then the reference count associated with that newly created object is not increased (it's 0) and therefore the object instance may be disposed of almost instantly resulting in "temp.right" pointing to some unused memory address space.


If all of the above is the case then I wonder how do properties such as Character.ActiveInventory and InvWindow.CharacterToUse work? Do they adjust reference count withing their set_ functions? Or, are those just "weak" pointers? (ie. no reference count is managed; I believe it should work for such instances since inventory items and characters are global static objects anyway and aren't allocated/deallocated at run-time)

[Pumaman]

The code you have looks reasonable, and the pointers should be fine, can you explain what exactly the problem is?

[Scorpiorus]

and the pointers should be fine

Ah, so they are already made to work like the script pointers, good to know.

So as I understand, it means that each time we are using an assignment operator with a pointer-attribute as l-value, AGS handles all the object management work automatically relying on the appropriate pointer get/set functions when necessary?

Therefore we should define these functions to clearly read and write address values so that the mechanism would work reliably?

Meaning, for instance, the following function...

Code: ags

void Tree_set_right(Tree* obj, Tree* newright) {
Ã,  Ã,  if(obj==0 || newright==0) return; // <--- may possibly cause problems?
Ã,  Ã,  obj->right = newright;
}

...should be changed in order to allow AGS reset the pointer to null (ie. when newright = 0) ?

I'm not however sure it's the one and only cause of the problem Edmundo seems to come across.

[Pumaman]

Sorry, I think I may have missed the point with my last post.

If you have a plugin that takes in a pointer from the script (as a parameter to a function, or as a writable property), the reference count will not be incremented if you hold onto the reference in your plugin ... therefore if you keep the reference but the script destroys its reference, the object could get deleted.

And there doesn't seem to be a plugin API method to change the reference count, so this should probbly be added.

[Scorpiorus]

Aha... ok, thanks for clarifying!

And there doesn't seem to be a plugin API method to change the reference count, so this should probbly be added.

Honestly, I would find it extremely handy to be able to add/release references from within the plugin code.

I understand it may probably would be safer to left this to AGS, but anyway since there is an issue with writable properties I would definately use them, were they exposed and available.


Oh, would it be possible to provide a way to retrieve the current value of reference count for purely testing purposes and to assist debugging? Maybe even as a separate "get" function with a note that it shouldn't be used and relied upon in decision-making situations throughout the code?