Messages

[*bump*]

And, *bump*.

New version is up.

Hey Gord, this sounds pretty cool, but if there's functions like GetFrequency and SetFrequency, it might make more sense to just implement (expose) this as a single "Frequency" property (attribute) with the getter and setter being hidden inside the engine code itself.

Also, the usual way of implementing these changes into the current code base is to use a pull request against the current development branch (develop-3.4.0) from your personal fork. It's pretty simple to set up, and this lets you make sure that there won't be any conflicts when we try to merge in your changes (make your changes, pull in the latest changes from the main repo's development branch, correct any conflicts, then submit a pull request).

[.Name]

As for the Deployer interface, it sounds a lot like what I am currently developing as the IBuildTarget interface in my personal fork, with the exception that I'm still relying on an Enum for the Settings pane to select which targets should get built. If I could rework to read the .Name of the IBuildTargets returned by BuildTargetInfo.GetRegisteredBuildTargets()*, then it would essentially be as generic as you've described. I've actually been trying to work out how to make it more generic, and I've been using a FlagsUIEditor to provide a drop-down box based on the values of the Enum (BuildTargetPlatform). I'll have to look and see how I can possibly manipulate that to simply concatenate the string values instead.

*Just as an additional note, I've added the BuildTargetInfo class so that the IBuildTargets can be added from the AGS.Editor assembly, but still referenced from AGS.Types.Settings. If there's a better way of doing that, then I'm all ears.

[Edit:]

I was actually reading into this and it seems that the assemblies aren't loaded at run-time, but rather just-in-time when a call is made, and actually throws a FileNotFoundException if the assembly is missing. If that's true (and I haven't horribly mistaken what I was reading) then it would be sufficient for me to have a try/catch block inside the constructor for a BuildTargetAndroid class. I could work up a quick proof-of-concept for this to test it.

Edit: I realize I mucked up that subject line a bit. Fixed it.

Drawing so close to a working build process for Linux has got me thinking forward to building for Android as well. I actually started on a port of the ADT's jobb tool to C#. I couldn't find any existing libraries for creating/formatting disk space as a FAT partition though, so I'm working on porting the library that the jobb tool uses to C# as well.

This would add at least a few extra assemblies to the editor though. How does everyone feel about that? Is there a way that the editor could reference those extra assemblies only if they exist and Android is targeted for building?

Once that's out of the way, the game resources could then be packed into a single OBB file and the engine packed into the APK. Then the APK could have a prebuilt script to mount the OBB file as a drive and read the game from there. It's how I've gotten other AGS games (including large projects like Al Emmo) to run from a single APK.

Thoughts?

The .user file was not, in fact, my idea. It was implemented by CJ around AGS 3.0.2:

Edit: BTW Chris, what is this Game.agf.user file that seems to have cropped up?

It's for storing any settings that are specific to you (the user of the AGS Editor) and not game settings.
At the moment it's only used to store source control information, but in future it might also contain which windows you had open when you closed the editor, etc.

Based on this, I would say it makes sense to keep this type of info in that file (since it's not really project data).

I actually did try looking into that, but I had no idea what I was doing mucking around in the compiler code.

My personal preference would be to add a Length member to dynamic array objects (or even static arrays too), but that might be more complicated, so I'll leave that for whoever has the ability to actually implement it.

[Edit:]

Just a quick update on this particular discussion, but I've finally gotten around to splitting up the build process for the game data files and the actual Windows EXE. It's still kind of a mess because of the way the whole process was structured, but those interested can check out the code here. A lot of things need to be moved around, renamed, and just generally cleaned up, but this should allow other platforms (like Linux) to build their files without having to worry about what other platforms have or haven't been built (data file is always built first, beyond that there shouldn't be a need for any other co-dependency), and adding new platforms is much, much simpler with this new structuring.



Edit: I need to test that everything is properly working and also do a full rebuild of the engine for Linux, but I should now have a fully functioning build of this for Windows and Linux. Will post a full build for testing later.

I think I sorted out the particular file that was causing problems, but here's a modified version of the function. This seems to be working as expected, and should disallow any "bad" file names.

Code: csharp

        public static bool CreateHardLink(string destFileName, string sourceFileName, bool overwrite)
        {
            if (File.Exists(destFileName))
            {
                if (overwrite) File.Delete(destFileName);
                else return false;
            }
            List<char> invalidFileNameChars = new List<char>(Path.GetInvalidFileNameChars());
            if (Path.GetFileName(destFileName).IndexOfAny(invalidFileNameChars.ToArray()) != -1)
            {
                throw new ArgumentException("Cannot create hard link! Invalid destination file name.");
            }
            if (Path.GetFileName(sourceFileName).IndexOfAny(invalidFileNameChars.ToArray()) != -1)
            {
                throw new ArgumentException("Cannot create hard link! Invalid source file name.");
            }
            if (!File.Exists(sourceFileName))
            {
                throw new FileNotFoundException("Cannot create hard link! Source file does not exist.");
            }
            ProcessStartInfo si = new ProcessStartInfo("cmd.exe");
            si.RedirectStandardInput = false;
            si.RedirectStandardOutput = false;
            si.RedirectStandardError = false;
            si.UseShellExecute = false;
            si.Arguments = string.Format("/c mklink /h \"{0}\" \"{1}\"", destFileName, sourceFileName);
            si.CreateNoWindow = true;
            si.WindowStyle = ProcessWindowStyle.Hidden;
            if (IsMonoRunning())
            {
                si.FileName = "ln";
                si.Arguments = string.Format("\"{0}\" \"{1}\"", sourceFileName, destFileName);
            }
            Process process = Process.Start(si);
            bool result = (process != null);
            if (result)
            {
                process.EnableRaisingEvents = true;
                process.WaitForExit();
                if (process.ExitCode != 0) return false;
                process.Close();
                // by default the new hard link will be accessible to the current user only
                // instead, we'll change it to be accessible to the entire "Users" group
                FileSecurity fsec = File.GetAccessControl(destFileName);
                fsec.AddAccessRule
                (
                    new FileSystemAccessRule
                    (
                        new SecurityIdentifier(WellKnownSidType.BuiltinUsersSid, null),
                        FileSystemRights.Modify,
                        AccessControlType.Allow
                    )
                );
                File.SetAccessControl(destFileName, fsec);
            }
            return result;
        }

Well even calling Process.WaitForExit isn't solving it, which is why I think that bit probably isn't related.

And I can agree that avoiding command-line processing would be ideal, but the .NET Framework doesn't have a method for creating hard links, and I still hold that a sub-optimal method is preferable over copying and duplicating huge amounts of data for every single platform. If you can devise a better method that would be framework-agnostic (runs on MS .NET and Mono), then I'm all ears.

Yeah, and I was thinking about that as well. I've been having some other trouble with it (which I don't think is related to this function itself) with files not being copied in time for me to set the permissions, and so I have quoted the parameters now, but it would probably make sense to strip or disallow any double-quotes from those parameters as well.

[copying]

This is an extension of a previous discussion brought up in my thread about building for Linux.

Through research as well as trial-and-error, I've attempted to come up with the most thorough, complete, and yes, even portable solution to create hard links from within the editor code.

Why do we need this? Particularly now that the build process is being split up to allow targeting multiple platforms from directly within the editor, the issue of copying files becomes particularly relevant. If your game's data files amount to several hundred megabytes or even reach into the gigabytes, then copying all of those data files into the appropriate build folders for Windows, Linux, etc. would become utterly obscene. Using hard links means that the actual data files can exist in exactly one place on your hard drive, but when you zip up your Windows or Linux folder, it grabs those files from wherever they're at instead of forcing you to have multiple copies of the files (think of it like a shortcut on your desktop, except you can also treat it like a normal file for zipping, etc.).

I wanted to post this here just to draw particular attention to it. Here's what I've come up with:

Code: csharp

        public static bool CreateHardLink(string destFileName, string sourceFileName, bool overwrite)
        {
            if (File.Exists(destFileName))
            {
                if (overwrite) File.Delete(destFileName);
                else return false;
            }
            ProcessStartInfo si = new ProcessStartInfo("cmd.exe");
            si.RedirectStandardInput = false;
            si.RedirectStandardOutput = false;
            si.UseShellExecute = false;
            si.Arguments = string.Format("/c mklink /h {0} {1}", destFileName, sourceFileName);
            si.CreateNoWindow = true;
            si.WindowStyle = ProcessWindowStyle.Hidden;
            if (IsMonoRunning())
            {
                si.FileName = "ln";
                si.Arguments = string.Format("{0} {1}", sourceFileName, destFileName);
            }
            bool result = (Process.Start(si) != null);
            if (result)
            {
                // by default the new hard link will be accessible to the current user only
                // instead, we'll change it to be accessible to the entire "Users" group
                FileSecurity fsec = File.GetAccessControl(destFileName);
                fsec.AddAccessRule
                (
                    new FileSystemAccessRule
                    (
                        new SecurityIdentifier(WellKnownSidType.BuiltinUsersSid, null),
                        FileSystemRights.Modify,
                        AccessControlType.Allow
                    )
                );
                File.SetAccessControl(destFileName, fsec);
            }
            return result;
        }

This supports relative or absolute paths, and should work with Mono (although I don't think there's a build of the editor that fully works with Mono to be able to test this on, it doesn't use any native code and specifies the appropriate linker file for Linux). It also attempts to make sure that appropriate security permissions are set on the hard link itself.

If I've overlooked anything else please let me know.

AGS will automatically close one of your text files when you close the editor. Not sure why this happens. I believe it's an editor issue and I'm looking into it

I've actually experienced this ever since the docking tabs were introduced. Whenever I close the editor the right-most tab disappears (while the others stay) prior to the prompt to save and then the editor actually closing.

Other than that, this looks like a pretty cool thing to have. Might be worth implementing in the main code base if it's a popular feature.

That's perfectly fine, too. This is slightly modified from my editor_make_dat branch, to clear out junk data and write two data files (native and managed).

I would like to have at least one or two people verify that it works for them (or even just send me the DTA files and I'll check them) before submitting the pull request.

For a real timer you would need to use DateTime class.

AGS can't do anything faster than 1 game frame (game logic loop and drawing loop are tied together ATM), so actually GetGameSpeed is the most accurate for cases like this. You could approximate milliseconds by doing:

Code: ags

int GetApproxMSInGameLoops(int targetMS, RoundDirection dir) // dir default is eRoundNearest
{
  float fresult = (IntToFloat(targetMS) / IntToFloat(GetGameSpeed())) / 1000.0; // (ms*40)/1000 for default game speed
  return FloatToInt(fresult, dir);
}

Of course a larger increment of time can be exactly precise (seconds = X * GetGameSpeed(), etc.).

[NOTE: This is NOT a stable build intended for making or releasing games! You probably should not even attempt to run games created with this build.]

NOTE: This is NOT a stable build intended for making or releasing games! You probably should not even attempt to run games created with this build.

This build is designed as a diagnostic to help me track down any lingering issues with building the game data file from the editor's managed C# code, as a step toward building multiple target platforms from within the editor (building for Linux is on its way soon!). While this produces a Windows EXE, let me reiterate that you should NOT use or distribute this file, as it's not stable and could cause major runtime issues.

I need your help!

What I need is for several people to build your projects with this editor (MAKE A COPY FIRST!) and then verify that the game28.main.dta and the game28.dta files (in your project folder) are identical files. These are binary files, so you can't just open them in a text editor, and the bare minimum size is about 92 KB. Huge games (such as The Cat Lady) produce DTA files that are around 3-4 MB. There are several ways to verify that the binary files are identical - there are tools that can do this automatically, you can use a tool to check the SHA1 hash, or you can do what I've been doing and feed the output from a hex editor into a text diff tool (slightly tedious, but it lets me see exactly where the divergence took place).

I would like to go ahead and submit a pull request to merge this process (sans the messy debug stuff, of course) into the AGS 3.4.0 codebase, but I'd feel more comfortable if other people test it out first since it is such a huge change. I have tested it against several projects myself and all features seem to be properly accounted for in the newly output data files (as compared to the existing file format, meaning the engine should have no problem reading them).


Thanks!

Download

[Edit:]

It's true I am writing the padding manually, but my goal was to first make an exact work-alike to be sure all the data was in a consistent format to what the engine currently expects (which, I suspect will be maintained for backwards compatibility) and then the deserialization methods in the engine could be updated to not require those padding bits (if game data format is the new version of course). After all, you're the one who keeps saying to keep separate tasks as separate commits. Removing the padding bits now would also require rewriting the deserialization methods, which I haven't taken on yet.


Edit: Reading back over this, it seems like you might be assuming that I'm changing how the game data file is read by the engine. I just want to make it perfectly clear that, for now, I am not making changes to how the data file is read, just from where (code-wise and language-wise) it is written.

Well, ultimately, yes. But the compiler is nowhere near being a standalone assembly ATM, so this is a step toward that.

When the C# code will fully work, the code that is not used for saved games could be removed from engine all at once.

That's mainly why I was asking. I didn't want to create a bunch of duplicated code if it wasn't useful, but the reason I decided to move it to C# in the first place was that 1) the code was a mess, and 2) it makes more sense in the main editor code base anyway instead of a separate assembly (given that as far as building the data file it is only run at compile-time). As I'm sure you're aware, there were a lot of hard-coded paths and methods that would be significantly more difficult to update across the editor and native assemblies which hopefully will now (once I'm finished, that is) be able to be updated and maintained in a single assembly (the editor).

Major changes don't have to happen right away, but I'm trying to keep my changes focused in the right direction, hence this discussion.

I've been meaning to look into this more than I've actually had an opportunity to, but it seems that the engine is currently reusing some functions in writing the game data file and in writing save game files. As most of you probably know, I've been working on porting the process of writing the game data file to the editor's managed C# code in order to make it easier for multiple target platforms to be built from within the editor. This has led me to the following questions:

* What data do the game data file and save game files currently have in common?
* What data do each of these (game data file and save game files) actually use/require?
* Aside from code reuse, what benefits are there behind keeping a similar/shared format between these files?

To be more specific, I'm pretty sure that GUI data (size, location, graphics, controls, etc.) is written the same way when building the game as it is when saving the game. While I appreciate that this removes duplicated code, some of the members used at run-time have no meaning or bearing whatsoever at design-time or compile-time while creating a bloated data file. These run-time members are written statically into the game data file as null/empty/junk values that are never used by the engine, but exist in the data file only as a form of padding to maintain similar offsets.

My current fork is still in the debugging phase, and when building larger projects there are several inconsistencies in the output game files (as compared to the pure native C++ build process), so I'm curious whether it is worth debugging if large portions may simply be replaced with calls back into the native code to maintain the existing format. Personally I think it would be better to split the format of the game data files and save game files, which could potentially result in drastically smaller game sizes (especially for larger projects I've helped out with like Gemini Rue or The Cat Lady), but I'd like to hear what others have to say before I make any final decisions (in the mean time, I'll keep debugging my work-alike port).