The State of Revision/Source Control in AGS

[edmundito]

Hey guys, I just ran into my old Back Door Man production notes where Grundislav and I used Subversion to share the project between us, and so I wrote some notes down about using Subversion with AGS. However, throughout the project it seemed that we had to put out a few fires with SVN, and so I'd like to start this thread as what people have done to share projects with others and what could be improved to make future versions of AGS more revision control friendly.


So on that note, with back door man, here's what we did, and what happened:
- We used AGS 3.1.2
- We didn't use the integrated SCC stuff that comes with AGS. I don't think Subversion supports it (though I think I paid plugin exists that might)
- I checked it the project in svn as /trunk/hustler (instead of the project itself being the root of the trunk). That way we could add documentation and stuff on the side as well, and when the game compiles, it was hustler.exe.
- We added a rule to request a lock acsprset.spr, backup_sprset.spr, and sprindex.dat such that when someone needed to add/change a sprite they needed to get a lock on those files, save them and commit them. That seemed to work fine to let someone know that they couldn't change sprites at the time, but still, it was 3 different files we had to be concerned about.
- We did not add the debug/built executables into the repository
- The one big problem we had  is that we started working on things at the same time, and then realized that we couldn't. Eventually what we did was one of us would work on the game while the other would wait, then one person committed all the changes, and the other updated to star working on it. Otherwise we would start getting conflicts on files that couldn't be merged.


As a techie guy, I realized some of the problems that would cause headaches:
- .crm files are a weird source/compiled hybrid. This was a huge issue because if I was working on the script and Grundislav was spicing up the room, we couldn't do it at the same time. It would be nice to split between what's source and what's compiled.
- Global and Module script changes (I think it was the script headers?) would seem to require that all the room recompile.
- To lock/manage the sprites, you needed to handle three files (as mentioned above)
- I wasn't too sure about what seemed to be two project files for the project (game.agf and game.agf.bak) I'm guessing that bak is the backup, but I don't know what would happen in case it would merge.
- We couldn't both add rooms at the same time, obviously because of the numbering... it would cause a conflict.
- It would have been nice to have the characters, guis, and objects mirror a similar system to the rooms where each has their own script, perhaps their own data file as well. And of course the compiled data would be split also.
- No technical details in the help file on what each file is about, and what's generated or not generated and what's important.

Honestly, I think improving some of those things above would make things much smoother!


Has anyone else worked on an AGS game under revision control? What was your experience like? What would you like to improve?

[Pumaman]

- .crm files are a weird source/compiled hybrid. This was a huge issue because if I was working on the script and Grundislav was spicing up the room, we couldn't do it at the same time. It would be nice to split between what's source and what's compiled.

Yes, this is true and is something I definitely want to change for a future version.

- Global and Module script changes (I think it was the script headers?) would seem to require that all the room recompile.

This is deliberate, since any room script could be referncing something from a global header, the only safe way to make sure the game doesn't screw up as a result is to rebuild all the room scripts.

- To lock/manage the sprites, you needed to handle three files (as mentioned above)

You don't need to add the backup_sprset.spr to your source control, there's not really any point having that checked in as it's just a backup copy.

- I wasn't too sure about what seemed to be two project files for the project (game.agf and game.agf.bak) I'm guessing that bak is the backup, but I don't know what would happen in case it would merge.

Yes, again game.agf.bak is a backup file so there's no need to add that to source control.

- We couldn't both add rooms at the same time, obviously because of the numbering... it would cause a conflict.

You can manually change the new room number in the Add Room dialog, which can help with this.

- It would have been nice to have the characters, guis, and objects mirror a similar system to the rooms where each has their own script, perhaps their own data file as well.

Perhaps, though then you'd end up with hundreds or even thousands of files in the game folder for all these different things. Not sure if it's worth it.

[CShelton]

Perhaps, though then you'd end up with hundreds or even thousands of files in the game folder for all these different things. Not sure if it's worth it.

I've wished for this myself. Update the sprite source files and have the IDE update to reflect the changes. I've noticed the new sound system references sound files in this way, but sprites tend to be trickier.

In my case, I'm rendering out 1024x768 character sprites (for 100% proper scale in relation to my scenes) and I always use the sprite clipper to cut away the fat. The processing that is done in the IDE is valuable, and not something I want to do myself externally. Thus the conundrum of how the IDE handles updated images when they have been processed, perhaps re-ordered, etc.

Back on topic though, it is certainly tricky to handle simultaneous edits to the project file, but even in commercial game development, we have to wait our turn to do some things. Setting up guidelines on what can be done during an edit session would probably get you most of the way there.

I'm managing multi-edit situations by creating my project within a Dropbox folder. Using that, I get effortless, nearly instant updates of file changes. If you haven't heard of Dropbox, check it out, there's a free version, and it's very useful.

[Calin Leafshade]

Sorry to highjack the thread but can someone explain source control to me.. how does it work and what are the advantages?

[Snarky]

You store the code on a server, and when you want to work on it you update your local copy to the latest version, make your changes, and check it back in.

There are three main advantages:

1) If your computer dies, you don't lose your work (as long as the server lives; source control servers usually make frequent backups)
2) If you screw something up, it's easy to go back to a previous version; you can go back to any earlier checked-in version (kind of like Wikipedia).
3) Most importantly: it makes it much easier for multiple developers to collaborate, because each programmer can make changes independently, check them in, and the next time the others update their copies, they'll have them. The source control system is smart enough to merge separate changes made at the same time to the same file, as long as they don't overlap ("conflict"). If you do have conflicting changes, it offers easy ways to compare the changes and choose which ones to keep.

In professional environments there will often also be a process to automatically compile the code from source control every night (the "nightly build") and run a set of scripted tests on it to make sure that it works. That wouldn't be possible if the code wasn't stored in one central repository.

[edmundito]

Yeah snarky gives a pretty good explanation of what the basics are. In art for example, it can track all the different changes every time you "commit" a change to the server it means that you can save all the versions of the file you've worked on automatically instead of saving it as a different file, and automatically everyone in the team has access to them. It also allows some other artist to come in and add other things. For example, it's common for one person to perhaps do a base background drawing and another person to paint it. So artist 1 checks in/commits their work and tells artist 2 is ready, and they start painting the background. Meanwhile programmer 1 can add that work in progress background into the game and can see the background change when they update the game 2 since artist 2 checks in every time there is progress with the painting.

CJ, you bring some valid points there. It seems that definitely it would be nice to have a section in the manual about some of the files in AGS and how to properly put a project under revision control. It would definitely be great to split the room compiles into their own files, and perhaps have the room data be xml-based for easier merging, and that will solve a lot of the problems.

As for splitting everything into its own script, I do kind of agree that it will be a mess. How about something like... a Characters event script, a GUIs event script, and an Inventory event script? That way the global script file doesn't get so crowded like it is now.

[CShelton]

How about something like... a Characters event script, a GUIs event script, and an Inventory event script? That way the global script file doesn't get so crowded like it is now.

You can do this now by adding a new script to the scripts section and by adding imports to the GlobalScript.ash file. I've broken my project out into:

Game
UI
Utility
Globalscript

The only thing I let GlobalScript do now is the stuff that's sort of hardcoded:

Game Start, Keyboard and Mouse input, On Event, and it's RepEx. I may still move Keyboard and Mouse out of GlobalScript, but it seems a little tricky. In the end, I'd like for GlobalScript to do four things:

1. Start the game.
2. Run a global Loop.
3. Run World Events.
4. Be the header for imported functions.

[edmundito]

Oh of course you could do it manually, but say that if you want to handle a mouse click in a gui, the event handler method would still have to go in the global script... I'm suggesting when you do that it automatically adds the event handler to the GUIs.asc file instead.

[Calin Leafshade]

You can use the ClaimEvent() function to stop the global script catching gui input.

[monkey0506]

The problem isn't stopping the global script catching input, the problem is allowing these events to be defined in a script other than the global script.

For example, if you put:

Code: ags

// NewScript.ash

import function btnIconSave_OnClick(GUIControl *control, MouseButton button);

// NewScript.asc

function btnIconSave_OnClick(GUIControl *control, MouseButton button) {
  gSave.Visible = true;
}

Where the OnClick event for btnIconSave is set to btnIconSave_OnClick, the game will compile. It will even run without error. But no matter how many times you click on btnIconSave, the event will never be called. This is because currently AGS will only call these events if the are defined inside of the global script. It doesn't matter if they exist elsewhere and a proper link has been set to allow the global script to call the function. They won't be called.

This also presents an issue for me in that GUIs and GUIControls are the only items shown on-screen that there is no way to generically simulate a click on. You could define a function to call the event handlers appropriately, but said function could only ever be used in the global, room, and dialog scripts. Thereby it is impossible for a user-defined script to simulate this type of click. It's been suggested before that a generic Click() method be added for this purpose but it's clear how high priority that's been rated.

Another thing I'd like to be able to simulate is key-presses, but at least I can move my on_key_press function to another script if I want. GUI, GUIControl, Character, and Inventory event handlers cannot be moved.

It can be worked around by having the actual event handlers call a separate function but it's still frustrating..not to mention the added overhead (albeit that it may be negligible in most cases it's still there) of the additional function call.

P.S. [/offtopic]

[Dave Gilbert]

I've made 3 games in AGS under source control, although it was much easier for me since I am usually the only one doing the programming.  There have been times during Puzzle Bots when my wife had to take over programming duties (she's much better at that stuff than me), and the only problem with that is that I am forced to stop working myself.  We've done our best to modularize everything so things don't overlap, but inevitably it's just easier if one person works at a time.

Typically, I use the SVN so all the artists/composers have an easy way to upload their work, with the bonus of being able to to check out a new build whenever they want.  It's much more convienient than everyone emailing assets to each other.

[xenogia]

How do I go about setting up Revision Control? This sounds great.

[Dualnames]

I've worked with Grim Reaper on that and I really had some troubles with the GUI's not importing their functions of their controls when imported. It would be nice if we could export a GUI with the option to include global script functions.

[GarageGothic]

Yeah, it would be brilliant if you could link a GUI to call it's interaction functions from a module rather than the global script. Would also be a great help for module developers who want to distribute a debug GUI along with the script itself.

[CShelton]

We have diverging topics here, but I'll address this:

How do I go about setting up Revision Control? This sounds great.

There are many options for freeware version control packages. All of them involve a server, and a client (or server interface). You can of course make a single computer your server and your client.

I'm currently running a free package called Mercurial. It's very fast and light, supports multiple users, has tons and tons of options, but is command line based.

Fortunately someone has written a GUI for it TortoiseHg. Tortoise adds right click functionality to standard windows file menus and such, allowing you to commit changes, revert files to an earlier version, browse the repository.

I am very happy with Mercurial in that you can just move the file repository around with ease. Copy and paste the folder to another drive, and that folder is a new repository (branch). Mercurial doesn't seem to have a "check out - check in" structure, you can edit anything you want and you get to decide later what you check in. It presents you with a list of files that have changed.

Another option for those looking for a professional tool, Perforce offers a free, two client version of their client/server software. If you have a two man team, this might be a great product as it is very complete, though much more difficult to get set up.

I use p4 professionally at my day job (commerical games), but I find that Mercurial is good enough for my night job (non-commercial games).

[xenogia]

Cheers for that CShelton, got it up and running