[SOLVED] Addressing managed objects in memory

Started by Monsieur OUXX, Mon 14/02/2022 09:49:40

Previous topic - Next topic

Monsieur OUXX

I remember CW saying that one of the obstacles of easily "managing" managed objects inside the AGS virtual machine is that they share the same address space as basic pointers. I don't remember exactly which kind of difficulties it was causing but I clearly remember that it was creating a technical glass ceiling -- because in the end everything is a pointer. So this limitation was indirectly impeding some glorious steps towards engine modernity and dynamic objects. 

This message is not to ask "what is the problem exactly" but to suggest a solution. Maybe what I'm about to write is dumb and has nothing to do with the issue, but here goes nothing. I've done that successfully in some pseudo-VM I was writing in AGS :

Split the pointers address range in two. I mean, a pair of completely artificial "fake" addresses ranges. For example :
0x0000 --> 0xFFFF = old-style pointers, nothing changed.
0x10000 --> 0x1FFFF = managed objects. You generate those exactly like you used to (values from 0x0000 to 0xFFFF), but you immediately add 0x10000 for as long as they're passed around as value from C++ function to C++ function-

With a simple binary mask you can tell which value represents which type. <= 0xFFFF = regular pointer. >=0x10000 = managed pointer.
Whenever you need to actually read/write to memory, just remove the bogus offset : For example, convert managed pointer 0x100AA to 00AA, then do whatever you would normally do (pointers arithmetic, real address in memory, etc.)

Food for thought.


 

Crimson Wizard

#1
Quote from: Monsieur OUXX on Mon 14/02/2022 09:49:40
I remember CW saying that one of the obstacles of easily "managing" managed objects inside the AGS virtual machine is that they share the same address space as basic pointers.

I honestly don't remember saying that, and frankly not sure what this even means.
Managed object addresses are not stored in a common script memory, instead they store 32-bit handles, which are simply IDs. These IDs are then rerouted to real memory address using a simple map.

EDIT: Could you may be refering to the script bytecode (and old VM) working with addresses as 32-bit values, while memory may be 64-bit? I'm not sure, but it's the only problem I know that has any connection to the address values.

Quote from: Monsieur OUXX on Mon 14/02/2022 09:49:40
I don't remember exactly which kind of difficulties it was causing but I clearly remember that it was creating a technical glass ceiling -- because in the end everything is a pointer. So this limitation was indirectly impeding some glorious steps towards engine modernity and dynamic objects.

It would be better to clarify which difficulty you are refering to, and what "steps" it impedes. Before working on solution we need to know which problem we are trying to solve. Not having a clear statement of a problem or a goal leads to trouble and wasted time.

Monsieur OUXX

OK I'll have to dig it out. you don't remember ever saying : "We can't do _________ because in AGS VM pointers for _________ are the same as pointers for ________ and the VM can't technically tell the difference"?
 

Monsieur OUXX

Github has removed all my ties to the AGS repo???? notifications, dicussions, forks, etc.
What the hell. Did the Microsoft acquisition reset everything???
Now it's impossible to search what I was referring to.
 

Crimson Wizard

#4
Quote from: Monsieur OUXX on Mon 14/02/2022 11:15:55"We can't do _________ because in AGS VM pointers for _________ are the same as pointers for ________ and the VM can't technically tell the difference"?

I may be begining to understand what you are refering to. Indeed the old VM could not tell what the pointer is refering to: is it a global variable, a value on stack, a managed object created by the engine, or object created by plugin etc. This is why I had to store additional info along with pointers (along with every script value in fact), which tells what it is. It's a simple "enum" meaning "value type".

I don't know what "we can't do" refers to though. I think it's best to clarify what is the overall goal is first, before searching whether and how we may (or not) do that, and find solutions for that.

As per your solution, it might work in certain enviroment, which either a) stores these addresses in a variable which is guaranteed to be larger than the pointer size and/or b) you use a virtual memory management that forces allocation to particular address range.
The former (a) will break the bytecode offset calculations again, returning the old problem of values being larger than 32-bit (as any offset calculation in bytecode currently relies on everything being 32-bit).
The latter (b) have been suggested and discussed before by several people, as a potential solution to simplify the current VM and possibly make it faster, if all script objects will be allocated within 32-bit space.

But the above only makes sense if we take full backward compatibility into account. Because as soon as compatibility with old scripts and games is not a concern, then bytecode and VM may be adjusted differently to suit our needs.

Monsieur OUXX

If you've already fixed that issue from a year ago (by attaching metadata to each pointer) then the discussion is useless. Problem solved! Well done and hats off to you  :cool:
 

SMF spam blocked by CleanTalk