Author Topic: Planning to fix AGS script file paths... again  (Read 578 times)

Crimson Wizard

  • Local Moderator
  • AGS Project Tracker Admins
    • Best Innovation Award Winner 2013, for spearheading the AGS 3.3.0 project
    •  
    • Lifetime Achievement Award Winner
    •  
    • Crimson Wizard worked on a game that was nominated for an AGS Award!
      Crimson Wizard worked on a game that won an AGS Award!
Planning to fix AGS script file paths... again
« on: 16 Feb 2018, 17:02 »
The file access rules - is something that still bothers me in AGS, and I'd really wish to patch it as one of my last ammendments to the engine.

It all started when Radiant asked me to save config file in the user documents instead of current folder, because it does not work when the game is installed in C:/Program Files (or any other non-writeable location). Instead of doing just that I also decided to change how file access works in game scripts too, but I have doubts that did that fully right.
I would like to make final fixes to that, and decided to post here to get any input from other people and make sure I don't screw that up again.


Summing up current state of the file system (as of 3.4.1):
1) When working with file system in scripts (opening files, loading sprites, and so on), following tokens may be used to define location:
* $INSTALLDIR$ - a directory where game was installed.
* $SAVEGAMEDIR$ - a directory for savedgames, one for each system user.
* $APPDATADIR$ - a directory for storing shared data, same for all system users.


2) The rules for working with files are following:
* Absolute paths are forbidden.
* When using $INSTALLDIR$ you cannot write anything, only read existing files.
* Game config lets you set up custom paths for $SAVEGAMEDIR$ and $APPDATADIR$. This is done primarily to let players change save game location according to their preference.
* When no location token is given, engine remaps the path as relative to $APPDATADIR$. Note, that this is different from how it worked before AGS 3.3.5, where path without token meant "relative to game's (installation) directory".
* Now, there is something not very explicit, which was made primarily for backward compatibility: when you are trying to read file using relative path without token, the engine will check two paths: first in $APPDATADIR$, then in $INSTALLDIR$. If a file is found in $APPDATADIR$, then it will be read, otherwise file in $INSTALLDIR$ (if it exists).
The reason for doing this was to silently remap relative paths in scripts in old games to $APPDATADIR$ when they are writing files, but still let them be able to read existing files from the current directory.
But this solution is something I am not very proud of...



The problems, as I see them, are:

1) Is it really good to completely deny writing to the current game directory? Initially I thought it is, since contemporary operating systems have stricter rules to where programs may write files. So this acts as a safety measure to prevent unexperienced game creators from making a mistake (as being said, game may fail if it tries to write to C:/Program Files/...).
On the other hand I myself found at least one case when that may be useful: if you are creating an edit mode inside your game, which creates data for the game itself.
Currently the only way to do that would be to setup custom path for $APPDATADIR$ for the development time and then do not forget to reset it to default before public release.
There is now a token with explicit meaning ($INSTALLDIR$), for which we may have a warning in the manual, telling not to write files there without serious reasons.

2) What should be the way to treat paths without a token? The idea behind changing from treating them as relative of game dir to relative of $APPDATADIR$ is that new users won't make their games impossible to be run from C:/Program Files by mistake. Is it a good reasoning? Because while it may make things safer it also may cause confusion
 when user will look for the written files in the game directory and won't find them.

3) Not much related to running new games, but in terms of backward compatibility, is there a better solution for their issue? I am speaking of a situation when the game writes files to game directory, which makes it impossible to be install in particular locations.



The possible solutions I am thinking about right now are:

1) Allow to write files in $INSTALLDIR$. There are options here: make it work always or only if game works in Debug mode, if that makes sense.

2) I don't have a decisive opinion about how to deal with paths without tags.

3) I would like to try implementing different solution for older games. To remind, the problem is this: consider there is a game made a while ago where game author has used paths without token to save files to the game directory. You won't be able to use such game if it's installed somewhere where you cannot write (restricted directories, read-only device). One solution in such case is to simply install it somewhere else. But if that's unwanted or impossible for any reason, may there be a simple fix for this situation?
Right now, as I said, engine silently remaps writing file to $APPDATADIR$, but when reading it has to check both $APPDATADIR$ and $INSTALLDIR$.
Of course this is not ideal, because user may not find files where he expected to. This may be particulary annoying with games which include level editor, like old "ColorWise" game, for instance, where original levels are stored in the game directory, and player will probably expect to find all levels in one place.
So, I was thinking, what if instead of silently changing the path, it would use another option in config. The meaning of option would be "how to treat paths without token", and has possible values corresponding to supported path tokens. Such option could be used like a casual hack: if you've found an old game (or a newer game where author did not consider possible issues) that writes files to the game directory, and that causes trouble for you, you just set this option to something else, like $APPDATADIR$. (Naturally, at same time you may also set custom path for $APPDATADIR$).

BTW, if such option is introduced that may also change the opinion on p. 2), and perhaps paths without tags could mean $INSTALLDIR$ again. Idk about that.



« Last Edit: 16 Feb 2018, 17:06 by Crimson Wizard »

Re: Planning to fix AGS script file paths... again
« Reply #1 on: 24 Feb 2018, 00:37 »
This is also complicated by built-in redirection in Windows, where writes to some un-writeable paths are redirected into the user profile without the user knowing. So potentially, when reading a relative path with no token it may actually be checking $APPDATADIR$, then any write which was redirected into the user profile when you tried to write to $INSTALLDIR$, and then $INSTALLDIR$. Personally I would say, leave what it is doing now as the default. There is an attempt to make old games work, you can never know in advance if the installation directory is writable, and people don't usually know about the redirection that is built into Windows. To be more flexible, you could just add one new game option with three file redirection options:
  • Redirection on (current rules apply)
  • Redirection off in debug mode only
  • Redirection off
In your case (editing game data while making the game), you pick the second one.
If you want to release the game like that and not have to remember to switch redirection options, you pick the third one.
« Last Edit: 24 Feb 2018, 00:39 by morganw »

Re: Planning to fix AGS script file paths... again
« Reply #2 on: 26 Feb 2018, 07:51 »
Here's my take:
1) Never write to $INSTALLDIR$.

2) For relative paths assume APPDATA for writing and reading, and a fallback to reading from INSTALLDIR if the file isn't found in APPDATA.
That way if the game has some level editing mechanics it will both work and preserve the original installation state.

3) Always keep #2 approach. Users gotta learn to revise their expectations.

- Alan

Radiant

  • AGS Baker
  • Mittens Knight
  • Return once more to the Two Kingdoms!
    • I can help with publishing
    •  
    • I can help with story design
    •  
    • Radiant worked on a game that was nominated for an AGS Award!
      Radiant worked on a game that won an AGS Award!
Re: Planning to fix AGS script file paths... again
« Reply #3 on: 04 Mar 2018, 15:26 »
On Windows 10 at least, AppData is also a user-specific folder. Also, given that these are official system paths, I question the wisdom of allowing the player to redefine them and point them someplace else. This appears to be a complicating the setup tool with options that the average player doesn't need or understand; the KISS principle applies.

Crimson Wizard

  • Local Moderator
  • AGS Project Tracker Admins
    • Best Innovation Award Winner 2013, for spearheading the AGS 3.3.0 project
    •  
    • Lifetime Achievement Award Winner
    •  
    • Crimson Wizard worked on a game that was nominated for an AGS Award!
      Crimson Wizard worked on a game that won an AGS Award!
Re: Planning to fix AGS script file paths... again
« Reply #4 on: 04 Mar 2018, 15:33 »
On Windows 10 at least, AppData is also a user-specific folder. Also, given that these are official system paths, I question the wisdom of allowing the player to redefine them and point them someplace else. This appears to be a complicating the setup tool with options that the average player doesn't need or understand; the KISS principle applies.

Maybe there is some misunderstanding, but this is not about redefining system paths. AGS does not change anything is system settings. This is about custom paths that engine and only engine uses. It may query a particular system path using system API and rely on that, or it may take literal custom path from config.

Regarding changing these, players, and even some game devs, have asked to make it possible to save games where they want, so I added that. The option resides under Advanced tab. From what I know this is mainly used to be able to save games in the game directory, since some people find that convenient for some reason.

Setting up $APPDATADIR$ is not in setup tool, and I was not planning to put it there. This may only be changed by manually editing config file.
Initially, the main purpose for this option was a convenience hack when running old games with the new engine.

I heard both options are also used to put saves and data written by game at runtime under Steam control.


From the answers above it looks like the general consensus is "don't change anything", so I guess this is what I do.
« Last Edit: 04 Mar 2018, 16:23 by Crimson Wizard »

Snarky

  • Global Moderator
  • Mittens Earl
  • Private Insultant
    • I can help with proof reading
    •  
    • I can help with translating
    •  
Re: Planning to fix AGS script file paths... again
« Reply #5 on: 04 Mar 2018, 20:41 »
What would really help would be to document where $APPDATADIR$ actually is by default. One time I had such a hard time finding it that I had to write a new file with a totally unique filename just so I could do a file search for it and thereby locate the directory. (IIRC, it turned out to be somewhere in C:\ProgramData\)

Crimson Wizard

  • Local Moderator
  • AGS Project Tracker Admins
    • Best Innovation Award Winner 2013, for spearheading the AGS 3.3.0 project
    •  
    • Lifetime Achievement Award Winner
    •  
    • Crimson Wizard worked on a game that was nominated for an AGS Award!
      Crimson Wizard worked on a game that won an AGS Award!
Re: Planning to fix AGS script file paths... again
« Reply #6 on: 04 Mar 2018, 21:35 »
What would really help would be to document where $APPDATADIR$ actually is by default. One time I had such a hard time finding it that I had to write a new file with a totally unique filename just so I could do a file search for it and thereby locate the directory. (IIRC, it turned out to be somewhere in C:\ProgramData\)

Yes that's true. I could not figure out the best place for it in the manual. Maybe add it on top of the "File functions and properties" section?

Re: Planning to fix AGS script file paths... again
« Reply #7 on: 04 Mar 2018, 22:07 »
On Windows 10 at least, AppData is also a user-specific folder.
These are not environment variables, so not the same location as %APPDATA%.