Improving user config situation

Started by Crimson Wizard, Thu 10/12/2020 08:07:56

Previous topic - Next topic

Crimson Wizard

The Problem

In the old times AGS games stored everything in the game folder. Around AGS 3.0 the saves were by default stored in user's docs ("Saved Games" on Windows), but config file remained strictly in game folder. In 3.3.5, I think, user config location was also moved to "saved games folder", by request, because in some circumstances it was not possible to write user config in game folder.
Numerous users did not like having saves in another folder, and around same time (3.3.5 or 3.4.0) an option was introduced to let player customize save game location, and just put saves where they like (even in same dir). Iirc this was primarily done at that time for the sake of Steam releases, as they could only sync saves which were present in steam game folder (maybe?).
However the user config is still written strictly in old save location and currently there's no way to change that. The reason was quite silly: savegame location was written in user config, so it's impossible to, for instance, learn user config location from save location without reading user config first... so at that time I could not figure out a way to resolve this.

This situation creates number of problems, starting simply with ags games cluttering obscure place on disk with their config files, which some users do not like, and followed by users being confused with the general rules of how it works. For example, user may try to edit acsetup.cfg in game folder, only to find out that nothing has changed. This may be especially annoying on Linux, where we still do not have a standard setup program.
I regularily received questions about this in the past, both on forums and in github issue tracker.

Goals

What I'd like to find is a nice way to support two "modes":
1. User config stored in default writable location defined by OS.
2. User config stored in custom location that could be defined by both game author and player. This custom location could be just the game dir itself.

At the same time the working of this system must be clear to players.

Questions and some thoughts

1. How to define the above "mode"? Supposedly this may be a setting in "default config" that comes with a game.
And what actually should be default setting for this mode (game dir or os-specific location)?

2. Is there a nice way to tell that user config is where saves are, thus merging two settings? OTOH someone may like to have user config in the game dir and saves somewhere else - is that also a valid case?

3. Speaking of user config inside the game dir. Suppose the game files could be overwritten, e.g. by installing updates, either by hand or by other system like Steam.
In such case even the config file could be overwritten. And if user config is written to same file as default config, it will likely be lost.

This is where I realized that we might need to make another change and separate default and user config by filenames. I.e. instead of calling everything "acsetup.cfg", call default config "default.cfg" and user config "user.cfg". Then we'll have a nice distinction even if user config is written to the same dir. Additionally, it will be more clear to players what "default config" is, because it will be named so. And it will be possible to "reset to defaults" by deleting user.cfg - even if it's in the game directory.
How will such change affect other problems? maybe it will make certain solutions easier?

One of the vague ideas I had in the past is to detect user.cfg in the game dir, and if it's present then use that, and if it's not then use writable OS-specific location. The problem with this is that the abscence of user.cfg in game dir could mean simply that no user.cfg was yet written, and therefore not really determinative.

selmiak

I like the idea of having a user.cfg.
Even better, would it be possible to save the default configuration of the game the game author sets into the game file itself? then there is one file less to confuse the player and it is clear when user.cfg gets deleted all settings get reset. and on top of that there is no chance a stupid user deletes the default.cfg.

about the default path, maybe start a poll?
I'd prefer settings and saves in the directory the game is in. Or maybe have a subfolder for gamesaves as these can pile up, but a subfolder in the gamedir and not somewhere completely different.
Quotesome people simply don't like it that the games write files elsewhere and pollute obscure locations on their disk (that's to paraphrase what users told me). There is also no automated cleanup as most "homemade" games don't have proper installers, so these files may stack up there for years (same goes to saves tbh).
This exactly (quote from github)

acsetup.cfg seems like a relic from the past, adventure creator setup. so renaming that to user.cfg is just cool in 2021 and the only downside I see is having to rename it in the manual and the wiki too. for compatibility the game itself can still accept acsetup.cgf but not generate it.
A user.cfg should be preferred by the game though, if available.

eri0o

Just make a decent cross platform GUI and all these issues will go away.

Selmiak suggestion of packing the default config IN the game package will break any non-winsetup config GUI, so please, don't.

Don't break games in read only locations.

selmiak

@eri0o: I mean, in case there is a acsetup.cfg or user.cfg the game should use these settings, if these files are not found it should use the few values stored in the game binaries as default value. BUt with "non-winsetup config GUI" you mean "custom" "winsetup" programs that write a user.cfg file or start the game with some parameters???


the thing is, if I'd install all AGS games I play, I wouldn't mind these extra locations for files, but then I'd expect the game to delete these files in extra locations when uninstalling the game (or ask me if I want to delete them or keep them).
But actually I like how easy it is to just unzip the gamefiles and then just play, no need for an installer, so this also includes no need for another directory for save files in my book.

can the game on the engine level try to save when starting the game and refuse to play with an error like "Please locate the gamefiles in a folder with write access or you cannot save your game or configuration" in case it does not work? Would this be possible? Every player will understand that and not unzip AGS games into protected system folders anymore...
QuoteIn 3.3.5, I think, user config location was also moved to "saved games folder", by request, because in some circumstances it was not possible to write user config in game folder.
What are these circumstances?  Are they still relevant?


But to give crimson something he can work with
1. How to define the above "mode"? Supposedly this may be a setting in "default config" that comes with a game.
And what actually should be default setting for this mode (game dir or os-specific location)?


I like GameDir as default for games I download, unzip and just play (and then delete sooner or later or keep it all together). For installing games that delete the savefiles with uninstall I don't care where they are.


2. Is there a nice way to tell that user config is where saves are, thus merging two settings? OTOH someone may like to have user config in the game dir and saves somewhere else - is that also a valid case?

Either have it in plain sight in the game folder.
Or maybe display the path in the winsetup like: "Game settings and Save-file path" with options like "open path" "change path" "use game folder" "delete saves and config"


3. Speaking of user config inside the game dir. Suppose the game files could be overwritten, e.g. by installing updates, either by hand or by other system like Steam.
In such case even the config file could be overwritten. And if user config is written to same file as default config, it will likely be lost.


I like the idea with 2 files, in that case the default.cfg might be overwritten but not the user.cfg, problem solved. if the gameauthor delivers a default.cfg and their own user.cfg via steam update, well, this is just crappy on the author's end and not AGS's problem anymore.

QuoteOne of the vague ideas I had in the past is to detect user.cfg in the game dir, and if it's present then use that, and if it's not then use writable OS-specific location. The problem with this is that the abscence of user.cfg in game dir could mean simply that no user.cfg was yet written, and therefore not really determinative.
Is this the check if the gamepath is writeable and the game can add files like user.cfg or is this the check to see where the user wants to have these config files?
for the check where the config files should be, what would the game do when both in the game directory and in the OS specific location no user.cfg, or default.cfg is present?

eri0o

#4
Selmiak, if you are in Linux, go into .local/share in your home and see literally every config and save file for every game and software in your machine. All ags games are under a single directory named ags. You will probably see directories for each publisher/porting house (feral-interactive, aspyr-media, daedelic, ...) , single games, other engines (Unity!), ... Edit: old Unity used .config, but newer moved to .local/share apparently.

Laura Hunt

From the simple user perspective of somebody with a very limited tech understanding of the underlying complexities of the situation, it seems like a graphical setup program that makes the process of updating the relevant files transparent to the user would be the best way to go, the same as it's done in Windows.

If this is not possible or needs more time to be developed, renaming the file in the game directory to "default.cfg" also sounds like an improvement, to which I would add the suggestion that AlanDrake made in the relevant github thread: add a HUGE comment at the top of this file to let the user know that editing this file will not change the game's settings after the game has been run once, and letting them know where to locate the file that they should edit (as, right now, players have no way to know why the changes they are making to the config file they have right in front of them are not being picked up, or where to find the right file).

eri0o

Considering that the Engine have it's own GUI toolkit (for in game GUIs) and that one could easily add one through a library (there are GUI toolkits with the components needed for a winsetup starting at 1000 lines of code) and that the engine has a software renderer that never fails and that it handles user input... I fail to understand what blocks making an interface to configure that would simply run anywhere AGS runs...

Crimson Wizard

#7
Quote from: eri0o on Mon 15/03/2021 11:13:01
Considering that the Engine have it's own GUI toolkit (for in game GUIs) and that one could easily add one through a library (there are GUI toolkits with the components needed for a winsetup starting at 1000 lines of code) and that the engine has a software renderer that never fails and that it handles user input... I fail to understand what blocks making an interface to configure that would simply run anywhere AGS runs...

Software renderer fails on many machines. It does not always work on my for example. Currently it depends on game color depth too. SDL renderer may work differently, but it's not in official release yet. In any case relying on particular renderer for setup application to work is not the best idea.

Nothing blocks anyone from experimenting though.

But I am dissapointed this turned into GUI discussion. GUI does not solve all the system's problems, it should be considered an option, not the foundation of the solution. First comes the system, the logic, when there's a working logic we may add GUI on top to make things easier.

E.g. we have GUI on Windows (winsetup), and yet users still have certain problems with config (some of the people who were nagging me about these issues are Windows users).

eri0o

Doing 2 will give in game script filesystem access outside of it's current boundaries - say, my downloads folder.

Maybe your problem on Windows is things that are written on disk without expectation? The reset savestate, is there a way to hold it on RAM? Also the config files that are/were written (like mouse acceleration) even if they didn't change.

Isn't it possible to give an option to clear the game files in the config menu/command line (delete the reset state, other savegames, and config files)? Also giving an option to open the save dir in the file explorer.

Spoiler
Here are the things I noticed when looking into configs

- every possible config should be possible through command line
- every possible config should be possible from in-game, with a fail-safe (never modify the display resolution, or crashes will make your Windows unusable) method, this would allow the gamedeveloper to make their own options menu.
- some configs should not exist (like allowing the user to modify how characters are rendered by the game, this should be only a gamedev time config).
- I believe the main usages of the config GUI are: configuring Translation, changing between windowed and fullscreen, and finding a graphics/sound configure that works. The resolution of the windowed would be better changed by allowing the user to resize the window, and if desired, snapping at the multiples of the game native resolution.
- notify in the setup GUI a "global config" exists since it will override things.
[close]

Crimson Wizard

Quote from: eri0o on Mon 15/03/2021 13:21:29
Doing 2 will give in game script filesystem access outside of it's current boundaries - say, my downloads folder.

Could you clarify what's "2"? Is it some point from my original post? There are two numbered lists there, I sort of guess which you may mean, but not fully certain.

eri0o

#10
Sure, I meant

Quote2. User config stored in custom location that could be defined by both game author and player. This custom location could be just the game dir itself.

I understand your idea of letting the player change this - I don't agree, but I understand.

But I don't like the possibility of the game developer setting to something like C:\Windows, I would make the config possible by the player but not by the game author in this instance.

Edit: maybe I don't understand what you mean actually, regarding the game developer. But I have been slowly trying to use more Windows and it's pretty magical to sync Saved Games directory on One Drive and get cloudsave across my Windows machines.

Crimson Wizard

#11
Replying to some of the points made above...

Quote from: selmiak on Sun 14/03/2021 22:31:39
Even better, would it be possible to save the default configuration of the game the game author sets into the game file itself? then there is one file less to confuse the player and it is clear when user.cfg gets deleted all settings get reset. and on top of that there is no chance a stupid user deletes the default.cfg.

I would not see this other than an option, not a rule. Engine will be able to read this if told where to look in. But this will make it more difficult to work with the file by external tools, as eri0o pointed out. And probably less convenient for game developer to quickly change default game setup by hand; unless we also change how the game package is compiled (i.e. have a directory to put all source files in for compiler to gather and put into pack).
This opens whole new list of problems, and that's why I would not consider this a part of base behavior.


Quote from: selmiak on Mon 15/03/2021 00:44:42
I mean, in case there is a acsetup.cfg or user.cfg the game should use these settings, if these files are not found it should use the few values stored in the game binaries as default value.

Yes, this is how engine works with all the game files: it first checks in the directory, then in the package.


Quote from: selmiak on Mon 15/03/2021 00:44:42
can the game on the engine level try to save when starting the game and refuse to play with an error like "Please locate the gamefiles in a folder with write access or you cannot save your game or configuration" in case it does not work? Would this be possible? Every player will understand that and not unzip AGS games into protected system folders anymore...

It may (and does), but it's not at all correct to refuse to run from writeprotected directories. It's a common thing for binaries to be installed in a protected place, such as C:/Program Files on Windows for example. And that is why AGS has saves (since 3.0 or 3.1 version) and config (since 3.3.5) in user documents by default instead.

What may be done instead is testing if it can save here by default, and if not then save elsewhere (in these user docs etc). There are two issues following this:
1) Morgan Willcock mentioned that there's a Windows service that can automatically remap files written to protected location into other place. I did not investigate this yet, but if there's literally no way to find out, then engine will think that it writes to game dir, while these files will appear in another place instead. Don't know if Windows shows them in same place for the user, if it does then this may not be the biggest problem.
2) Notifying users where the saves/user config is in such case. This is a complicated question, depending on what we think, do users expect them to be written there at all or not. I think it's also related to question how to tell these file locations to users in general (regardless of what intention was and why they need these). Since 3.5.0 there is "--tell" command line option that prints various information, including these locations (More on this here). In theory this may be expanded into something more convenient, but I don't know what. Maybe good solution would be to have setup program able to open these locations in explorer by a button press for example.


Quote from: selmiak on Mon 15/03/2021 00:44:42
I like GameDir as default for games I download, unzip and just play (and then delete sooner or later or keep it all together). For installing games that delete the savefiles with uninstall I don't care where they are.

This would mean that Editor should configure game for writing to game dir by default, and provided options (e.g. in General Setting) for changing that for games with real installers.

What I don't know is, how is all this seen from modern OS perspective. Windows started as not caring about where the files are saved, then it shifted towards separating user files and program files - and that's something that unix systems already had years before that.

In regards to polls, polls will give an insight on user expectations but they don't determine the correct program behavior as presumed by the system.


Quote from: selmiak on Mon 15/03/2021 00:44:42
QuoteOne of the vague ideas I had in the past is to detect user.cfg in the game dir, and if it's present then use that, and if it's not then use writable OS-specific location. The problem with this is that the abscence of user.cfg in game dir could mean simply that no user.cfg was yet written, and therefore not really determinative.
Is this the check if the gamepath is writeable and the game can add files like user.cfg or is this the check to see where the user wants to have these config files?
for the check where the config files should be, what would the game do when both in the game directory and in the OS specific location no user.cfg, or default.cfg is present?

This is a check where the user wants the config files. Your second question is practically repeating my own question you quoted, and I don't know the answer. This is one of the missing pieces in the idea.



Crimson Wizard

#12
Quote from: eri0o on Mon 15/03/2021 13:21:29
Doing 2 will give in game script filesystem access outside of it's current boundaries - say, my downloads folder.

Quote2. User config stored in custom location that could be defined by both game author and player. This custom location could be just the game dir itself.

The rule of the script filesystem is to not determine which physical locations to read or write itself, but to let engine (defaults) and user to setup locations which it may use. Script filesystem recognizes these locations only by their tags (like $SAVEGAMEDIR$). The actual physical location is set either using default system paths or user options, and in the latter case it's user's problem how that will work.
User here - is anyone who uses the game.

Quote from: eri0o on Mon 15/03/2021 13:21:29
But I don't like the possibility of the game developer setting to something like C:\Windows, I would make the config possible by the player but not by the game author in this instance.

If they do that would likely not work for writing. I am not even sure how to make config possible to player but not author as author can add same files to the game, and edit them, and play it as player too.
But this is a general concern about anything configurable and question of responsibility and trust.
For example, someone can write a plugin that breaks your system. E.g. look into the Shell plugin, it literally allows to run any shell command on Windows, and you don't know which one game runs unless you extract script data from the game and research it.


Quote from: eri0o on Mon 15/03/2021 13:21:29
Maybe your problem on Windows is things that are written on disk without expectation? <...> Also the config files that are/were written (like mouse acceleration) even if they didn't change.

The main problem is the place they are written. How or when may be a lesser issue on its own, it becomes serious in combination with a previous problem.

Quote from: eri0o on Mon 15/03/2021 13:21:29
The reset savestate, is there a way to hold it on RAM?

Hm, reset savestate is a one of many possibilities that may be in game. Reset point could resaved anytime later (there's a function for that). Game script may use any custom slot as an autosave too.
If you refer only to the default reset point created at the game start, then reloading the game from disk will be simplier equivalent.

Quote from: eri0o on Mon 15/03/2021 13:21:29
Isn't it possible to give an option to clear the game files in the config menu/command line (delete the reset state, other savegames, and config files)? Also giving an option to open the save dir in the file explorer.

Some of this could be added, yes.
There's one potential problem: engine knows only a) default system locations b) user-set location. If user first set custom location A, saved some games, then switched to custom location B, then naturally engine won't know about location A anymore.


Crimson Wizard

#13
This is what I'd like to try out.

1. Config is split onto default.cfg and user.cfg for the purpose of:
- distinguishing their meaning;
- distinguishing them if they end up in the same dir;

2. Engine may be told to look for user.cfg in the game dir, and there will be 2 methods to do so:
- user.cfg is found in game dir;
- explicit option. I'll omit how this option is defined for now, but let's assume there's just a way to do so.

3. When engine told that user.cfg's location is in the game dir it will make a write test to find out if it has access perms there. Write test does not change anything, only tests if it's possible to open user.cfg for writing.
- if it is writeable, then it will just use that as been told;
- if it is not writeable, and there was no user.cfg there in the first place, then it will fallback to standard behavior and refer to the user documents folder (as it does now);
- if it is not writeable, and there was a user.cfg in the game dir (but it can't write there), then it will read both user.cfg in the game dir and one in the user documents, merging them together (latter will have option priority). But it will be writing only to the user documents though, obviously.
This last point may be bit confusing, but the purpose of such behavior is to make certain that user.cfg in the game dir is used anyway, or there will be raised a reasonable question why it is not.




Now, how do we explicitly tell the engine to use game dir for user.cfg?
There are two ways here: command-line option and config option. Well, strictly speaking there are more ways, like, enviromental variable, but this is not something a lot of people would use (or know how to), so we leave it as a backup plan.

1. command-line option is plain and simple, except it means you can't just ran game.exe by double clicking, you would have to either:
- call it as "game.exe --usercfgingamedir" by hand each time (this is a made-up option name for now, not final);
- create a startup link or script to run it like that.
So, this is something more suitable for technical nuts, or if you are using a custom AGS game launcher that runs any game using particular engine version and its own settings...

2. config option would mean it should be in config file. But it's likely most games would be coming without one in default config, so how do we avoid having to add one into each game you're playing?
Here's where global AGS config come into play. Idk how many people are aware, but AGS supports a global config file which is applied to every game you run (user config still overrides its options). The global config is currently disabled on Windows because some developers were afraid that players could create one by mistake and have troubles... but the more I think about this the more it does not seem like something highly probable.
The only issue would be to figure out where to store it on Windows. It should perhaps be placed somewhere around user's AppData folder, or similar.
Then players who would want all ags games they run have user cfg in the game dir would need to create such file once, write only couple of option there to force config and saves in a game dir, and it should work from now on.

I guess in theory a person distributing game may also add such option to the default.cfg. This may work for e.g. Steam releases or if game is released with a custom frontend (launcher), and authors would like to have all game files together in same dir there. But that should not be recommended for a regular case, because that may complicate situation for some players.

In my opinion that's the only acceptable way if we want to keep AGS engine compatible with modern system requirements.

The way to make these things more convenient is what eri0o has been suggesting: make a setup program that is aware of all the above and let you configure the global settings for AGS games.

Even in the worst case, it's no secret where AGS writes config and game saves, on Windows that is a very standard "Saved Games" folder, which may be easily found in explorer, and no different from many other contemporary games. If this information is missing in the manual then it should be added there to be visible for everyone (or moved in a more visible topic).






That all said, I actually wonder if it may be better to postpone split between default.cfg/user.cfg until AGS 4. Having this kind of change at once in a minor updates to 3.5.* we have now may not be good. In addition to that, if we implement this in AGS 4 then we won't have to deal with backward compatibility of configs and may completely ignore old acsetup.cfg.

In case this split is postponed then above option may still be implemented. It's just that it would be only explicit option, as there will be no way to distinguish default/user config in game dir.

Crimson Wizard

#14
EDITED: I wrote a post regarding giving game authors opportunity to read and write "user config", so that they won't have to rely on assumptions of where the config file may be...

But then I began thinking, what's the point of inventing something rushed; how many config settings may be actually changed within a game?
* Translation - that's the obvious one;
* Windowed/Fullscreen;
* Mouse speed for fullscreen mode;
and what else?

So, on a second thought this is not that necessary at the moment, as with above limited list of 2-3 options people could live with a custom config file too.
Also, if someone used an option to assign user config to game dir, they may as well assign savegame location there too, then even the older scripts that read config file from savedir will work.


The old post is hidden under spoiler if someone wants to read it, but I guess this may be left for future versions where there will be more to change at runtime (like graphic mode).

Spoiler

While I am at that, I realized that it would be a good idea to add just anything in script to let game developers read and write real config. Idk how many games actually attempted that; I've seen maybe couple of them doing this but it could be more. This works so far because user config is coincidently inside $SAVEGAMEDIR$, but if there are any options that change its location, then things become uncertain.

Plus it's a very old feature request... too bad this is happening in a rush again.

I've been looking at options here. The most dumb method would be to add something like File.Open("$CONFIGFILE$") for reading & writing. The problem is that... the file is INI format, which means you can't just write what you want there, and it's possible to break your own config by mistake. Sure you may use IniFile module by WyZ, but even that has certain restrictions. Also it's kind of weird to add a script command that's practically unusable without a good script module.

Then there's an idea to reduce amount of work for game developer and let them only fill items they'd like to write (or get config content in a form of list of items). Luckily we already have Dictionary type that may represent key/value pairs. INI format is more than that: it also has got sections, so one Dictionary is not enough. But following API may work:
Code: ags

/// Reads a list of existing config sections into the given Set
File.ReadConfigSections(Set* sections);
/// Reads all items from the section and stores into the given Dictionary
File.ReadConfigItems(const string section, Dictionary* items);
/// Write items from the given Dictionary into the user config file
File.WriteConfigItems(const string section, Dictionary* items);

The above frees game author from having to parse/write INI format, and it is probably even safe for any changes of config format. Engine may provide merged config from all the config files it reads. Engine will also merge options back filling them in the same order, and maintaining any additional contents that could be there.
In fact, it is not even dependant on reading particular file at all, and may work if some system port would suddenly not support conventional config files on disk.

There's still one downside to this: you got to match current engine's config options (names and value format). If you do it wrong, the config will end up with mistakes.
Also, if config options change in any way in the future engine version - you'll have to adjust the game script that reads and writes them.
But if people won't go over the top with this and only write those few options that are currently supported to be changed by the engine at runtime (like Translation, for instance), they might be fine.

If we also add "filename" parameter - we will basically have functions to work with custom INI file for your game. We'll just need a special keyword for "user config", similar to the first solution mentioned above.


This is as much as I could come up with without inventing something complicated.
I guess for more safety there could be an API that creates a kind of "Property Set" with value type checking, automatic formatting etc. But that is something that has to be properly thought through and designed first.
[close]

SMF spam blocked by CleanTalk