Changing the way Editor references original files

[RickJ]

I think the confusion comes from looking at this from two different perspectives.  I wasn't thinking much about how sprite import currently works but I think ChamberOfFear was thinking in these terms.

If there is a Sprite Path property then one could specify a number of locations where sprites can be found which can be specified relative to the project folder or be an absolute specification. As selmiak says settle on a good set of defaults so that most people won't notice.

Sprites would be imported relative to the path which could include both absolute and relative specifications.  There are a couple of situations that must be handled properly though. 

1) Suppose the same file name is used in more than one folder.  The path property is essentially a search list, so to re-import the file one would search the folders specified by the path property until  a match is found.  Now if the original file was imported from a folder appearing latter in the list then the wrong file would be found.  This could be solved in a number of different ways.  For example, each folder in the path specification could be assigned a logical or symbolic name, and that name would be stored with the sprite. Alternatively one could ask the user to select from a myriad of options if such a situation occurs.

2. Navigation when the import dialog opens would need to be aware of the path specification and probably select the first folder in the list by default. The user would then be able to select sub-folders and eventually a file.  When the sprite is imported it is save with a path that is relative to selected path folder. No need to look everywhere for sprites when the path specification tells where they are.  However, there should be provision to import a sprite from an arbitrary location as well.

There are a couple of use scenarios where this would come in handy.

Multiple Games with Common Assets (RON)
In a situations such as "Reality on the Norm" the same game assets are used on multiple projects. So one could easily imagine a repository external to the game folder that contains these sprites.  Of course there are always game specific sprites such as room objecfts, inventory items, etc that are game specific.;  These sprites could be kept in a repository residing in the game folder where they are used. 

Multiple Artists
If a project had multiple artists creating sprites e then each one could have their own repository.  If re-import mechanism had an option to select folder from the path specification then it would be possible to re-import everything from this artist but not that other artist. There are of course other kinds of groupings possible.

Sprite Override
Suppose there is a template for a specific kind of GUI.  The Module author provides a default set of button, panel, and other sprites used in the GUI.  The default sprites could be kept in an absolute location (i.e. the AGS template repository or AGS install folder).  Anyone or all of these sprites could be replaced simply by making s copy and modifications in the project's sprite repository.

As is usually the case there are likely more detail to iron out but hopefully the above explains my thoughts more fully.

[Crimson Wizard]

I can see that the simplistic fix, initially proposed by Calin, has grown to a serious design change in this thread. This is all terribly confusing. I guess its what happens when the design is not properly discussed before changing something.


Let me point out that the AGS never had a full-scaled "re-import from source" mechanism. It was not long ago when "Replace from source" command appeared for sprites in AGS 3.3.0. It was made for simple convenience, without having big changes in mind.
Similarly, in respect for audio files: the connection between source file and Audio cache was introduced only in AGS 3.2.

There are a lot of questions rising when you start to think of this thing in whole. For example: why is there this bind between source and cached versions? What is the purpose of this duplication? Currently it appears that the cache (at least sprite cache) is actually a main version, because re importing sprites could be nearly as complicated as importing sprites anew. So maybe it will be better to let Editor work with original source instead, and use cache only if source is not found?
What to do with tiled imports, or imports from clipboard? Should these be copied to some "source" folder too first?
Should modified source be copied to sprite cache automatically? because now it does not.

Take absolute paths, why anyone would need them in a collaborative project at all? Maybe introduce enviroment variables, like "PATH_TO_DEFAULT_SPRITES", and redirect sprite search there?

Etc, etc, etc.




This is something you need to have in mind: AGS simply does not have the complete feature of binding "source" material and project "cache". Therefore, it is not correct to propose modification to existing design, because there is practically none; if one wants this work differently and being organized, then a full design plan should be written from scratch, and discussed.
All this is beyond the scope of the initial idea of a fix.

If you are interested in discussing this, I have a suggestion to actually separate the fix issue and design issue, because the latter will take more time to flesh out and implement.
The fix we were originally speaking of is very much wanted: I was noticing this was mentioned several times on forums. The purpose of it is mainly to let people keep source material inside project folder and reference that with relative paths.
We may go with a more simple decision for current release (AGS 3.3.5), something like cat's proposal; and spend a proper amount of time designing the actual re-import mechanics for the next big release.


Maybe we should first agree on what we are discussing here, before continuing?

[Snarky]

I think CW has it right. All this duplication is bad, and to really get a satisfactory solution a redesign from the ground up is needed.

In the long run, I think AGS should move to having the source files as part of the project, so they can be updated simply by updating the file (how to deal with animated gifs, tiled sprites, and transparency settings is a bit of an open question, as is the question of how resource indexing would work if a file goes missing), which would also improve source control support. AGS should also use the file system folder structure to organize sprites, scripts, audio files, etc., rather than forcing users to duplicate this organization themselves. All file references should be internal to the project, and relative to the root folder path, so that a project could be shared without breaking any references.

In the short run, I think adding another configuration setting so that everyone gets to have it however they like is not a good idea. One definition of design is to impose constraints on a problem, or in other words, to make decisions. But since Pumaman, CW and the other AGS developers have always been bombarded with different opinions and ideas, all too often the "decision" has been to have it both ways: to make it optional, configurable. In the long run, all these configuration options makes the system hard to learn and understand, difficult to support and debug (if someone has a problem, you first need to figure out exactly which of a million possible configurations they're using), and increasingly impossible to maintain.

So if we need a short-term solution before moving on to a (possible) future redesign, I would urge the devs to just consider the different arguments carefully, and then making a decision, not try to cover every individual preference as a separate setting. In this case, I think the simplest solution would be to store the relative path if the file is within the project folder (or a subfolder), and the absolute path otherwise. Users can address the other use-cases RickJ brings up by linking sub-folders of the project to a repository however they want; AGS shouldn't have to be involved in that. But that's just my opinion.

[cat]

In this case, I think the simplest solution would be to store the relative path if the file is within the project folder (or a subfolder), and the absolute path otherwise.

This.

Please don't make it overly complicated.

[ChamberOfFear]

I agree that this has grown out of proportions, way beyond the scope of the original fix. I'd be happy to just apply what was originally going to be applied if we agree that the design-stuff will be re-visited later, more specifically I will be applying this:

The user adds a new resource.
Scenario 1: It is stored in a subfolder of the game folder -> it is stored as .\sounds\bang.ogg
Scenario 2: It is stored somewhere else -> it is stored as C:\Users\cat\Pictures\logo.bmp

A final question to Crimson Wizard, on GitHub you requested this of me

Another thing, would it be possible to run an automatic fixup for all the existing references when project being opened and apply the relative path rule? This will help when importing existing projects with new Editor.

Should I still look into this? Or is it better to drop it for now?

[Crimson Wizard]

A final question to Crimson Wizard, on GitHub you requested this of me

Another thing, would it be possible to run an automatic fixup for all the existing references when project being opened and apply the relative path rule? This will help when importing existing projects with new Editor.

Should I still look into this? Or is it better to drop it for now?

I think this won't go out of scope, because it does not change the design itself, but enforces these changes you apply.
Yes, I would ask you try to implement this, because otherwise the game project may appear to have one half of paths following old rule, and another half following new rule.

[Monsieur OUXX]

Regardless of the final implementation and decisions taken here, I think that if the "path" property was made editable in the editor, that would be really cool.


PS: what we do in our project is that every person working on the project can put the sprites wherever they want but then they have to mount a network drive so that the (absolute) sprite import path is always something like Y:\sprites\...

[tzachs]

I support cat's proposal for the "short term" solution, I think it's simple and intuitive.
I also like Monsieur OUXX suggestion of making the path editable in the editor.

Now for the long term, regarding the cache/source folders duplication:
I think we need to separate this into two different duplications:

1. Duplication of organization: I agree with Snarky, this is completely unnecessary imo. I already have everything organized in my folders, the sprite window should either reflect that (i.e adding a folder in the sprites window will add a folder in windows explorer) or be scratched altogether by just having all the assets accessed from the tree.
The only disadvantage that I see for this is losing the ability to bind to absolute paths, as all assets would need to be inside the project (but this could be solved if it's really needed).

2. Duplication of content (the actual need for both using the source folders and the cache):
- The source folders are needed for development convenience (easily replacing assets with less work to bind them in the editor).
- The cache is needed for deployment- you want all assets embedded in the game file and not out in the open for the users to play with (or at least, we want the developer to have the option to embed the resources in the game file).
I don't see a need for the cache during actual development, though. So I think a possible solution would have the game to take the assets directly from the folders when developing, and adding a deploy button which will embed the resources (or have a checkbox of whether to embed the resources). We'll also be able to add more features for the deployment in the future (like an optional installer, choosing the platforms, etc).

[Monsieur OUXX]

1. Duplication of organization: I agree with Snarky, this is completely unnecessary imo.

Yeah, "virtual" folders are always a source of confusion in every IDE.

I don't see a need for the cache during actual development, though.

Yeah but sometimes you want to share your game project with other developers, or simply back it up, without worrying that you forgot some of the other "external" folders where you keep your pictures, or without worrying that your backup is in a consistent state. There are countless examples of such features in other fields (for example Adobe software, that lets you zip in all the font files you use in your text editor project alongside it, not just strictly your project file). That's what the cache file does.

HOWEVER a possibility would be to simply tweak the cache file code, that (if I'm not mistaken) already has mechanisms to check if the file is outdated, to simply display a warning in the editor at compile time (in the Errors pane), that would tell you if the file is 1) missing or 2) outdated .