Topic: ENGINE PLUGIN: AGSteam v3.4 (Windows; Linux; Mac OS X)

[monkey0506]

The long and short of the reply I was about to post is that upon installing Heroine's Quest, running it in and outside of Steam, running it with and without Steam running, and running it directly via the AGS 3.3.0 "acwin.exe", I cannot reproduce a crash of the game using this build of the plugin. I have reuploaded the build to make the process for the user simpler. Have them extract this file on top of the Heroine's Quest files. They must replace "AGSteam.dll" and "steam_api.dll" with the copies enclosed. Then, when they launch the game, a file "agsteam_debug.log" will be created in the game folder. Please have them report back the contents of that file after the game crash (though, honestly, I really suspect that the game won't crash at all).

[Radiant]

Thank you, I've invited him to this thread.

[eMTe]

I've replaced the files.

Strange thing is the debug.log HAS NOT been created. The error message I get is very similar thoufg, only with different numbers.

Here's the crash file.

https://bitbucket.org/Maciek_Tr/sosnowy-las/downloads/CrashInfo.3.3.0.1162.dmp

One clue about my problems may be(?) that I am using an old system - XP. Morover, due to registration issues (XP is no longer supported) I have to reinstall the system every 4 months. So, could it be that after last reinstallation I am missing some system files which prevent me from running the game? Yesterday, Ive tried to run another game (not an AGS game) and I got the message that I'm missing d3dx9.dll. Didn't have time to download it yet. The question is, could that be the problem, like an outdated version of DirectX, missing dll files or something?

Also, I'm not the techy guy, but all AGS games are built, like, in the same manner, right? Plugins, system requirements needed, etc. So how come this is the only game giving me headaches? This is my first AGS game (and I've played over a hundred of them) that I can't run at all. What's so specific about Heroine's Quest, is it more advanced in certain technical areas?

[Radiant]

The question is, could that be the problem, like an outdated version of DirectX, missing dll files or something?

Yes, absolutely.

Also, I'm not the techy guy, but all AGS games are built, like, in the same manner, right?

Actually no. For example, it depends on what version of AGS they're using. Among numerous other things, older AGS games tend to run in 16-bit color mode under DirectX 5, whereas newer games (such as Heroine's Quest) run in 32-bit color mode under DirectX 9.

[eMTe]

I have directX installed, NET framework installed, seriously I can't say what may be the source of the problem on my part.

HQ is the only AGS game I have on hd with AGSteam and steam dlls, so I still believe this is the problem.

Btw, as I mentioned to you in pm or in the other topic, I was able to run HQ right after it was released, without any problems. Didnt have time to play it though and later I deleted it. Is there any chance the first version of the game, without plugins (I believe it was without them, right?) like it was released first, is still downloadable from some source? Maybe it'll work?

[monkey0506]

This plugin is being built with Visual Studio 2017 with static linkage to the C++ runtime, so it shouldn't require anything that AGS or Steam don't already depend on. Even if it did, then it should crash much sooner, during the phase where the engine attempts to load the DLL (the plugin). What's happening is that the engine is succeeding in loading the plugin, but then for some reason it's accessing some memory address incorrectly. Which is weird, because the only two pointers being dereferenced are a pointer to a static object and a pointer to an object that the Steamworks API has guaranteed (by the successful return of SteamAPI_Init()) to be valid.

I am already telling the log to flush after every write, but perhaps I'll try closing it after every write operation. The log should be created, no matter what. Curiouser and curiouser.

[monkey0506]

Please try this version. This will create "agsteam_debug.log" as soon as the "GetClient" function is called, before the AGSteam::Startup function is invoked. The debug log will now print the address of the AGSteamPlugin object and the ISteamUserStats object, which at the very least should verify if one of the pointers is, in fact, a null pointer. This also opens and closes the log file with every write operation, just to fully ensure that the writes are flushed to disk before any crash.

[Crimson Wizard]

Just in case, I made a quick try, and launched the game exe Radiant sent me earlier with you plugin. Game did not run, because some data is missing, but your plugin managed to create some log anyway:

AGSteam::GetClient called, returning address '0x0F9946FC'
Steam not initialized, calling SteamAPI_Init()
SteamAPI_Init() failed. Is Steam running and logged in?

that's to confirm that the log is generally working.

BTW, that's strange that it failed to init, because I had steam running on background. Maybe I miss something else.

EDIT: I just realized I have this game in my steam library, so I could download it...

[monkey0506]

BTW, that's strange that it failed to init, because I had steam running on background. Maybe I miss something else.

If you just launch the EXE Radiant uploaded above then Steam won't detect it as being a Steam game. You can force this by adding a text file called "steam_appid.txt" with only the text "283880" (no quotes) to the same directory as the EXE and plugin. This alerts the Steam API as to which "app" (game) the EXE is when it's being launched from outside of Steam. Or, as you said, you could just install the game via Steam.

Thanks for letting me know that it's working up to that point on your end CW.

[monkey0506]

I've replaced the files.

Strange thing is the debug.log HAS NOT been created. The error message I get is very similar thoufg, only with different numbers.

Here's the crash file.

https://bitbucket.org/Maciek_Tr/sosnowy-las/downloads/CrashInfo.3.3.0.1162.dmp

Okay, I actually got this dump to load properly! And, this is totally bizarre, but it says that engine is the NULL pointer.

main.cpp:L117


I have no idea what circumstances might lead AGS_EngineStartup to be invoked with a null pointer. If it's helpful, I can make the debug log start as soon as AGS_EngineStartup is entered, but this seems pretty definitively to indicate where the problem is coming from, which has to do with the engine's invocation of the plugin, and not the actual plugin code itself.

Edit: AGSteam-debug-2.zip will create the log immediately after engine has been assigned in AGS_EngineStartup, and will print the value of engine and lpEngine before continuing.

[eMTe]

Regarding your first link:

The log has not been created again, but here's the dump.

https://bitbucket.org/Maciek_Tr/sosnowy-las/downloads/CrashInfo.3.3.0.1162.dmp

Regarding your second link:

No log again. Dump:

https://bitbucket.org/Maciek_Tr/sosnowy-las/downloads/CrashInfo.3.3.0.1162a.dmp

General question, could it be that XP as a system is the problem itself? After all, the plugins were created rather recently of what I can read, so they were built specifically for supported OSes. Also, not many players use XP anymore. So maybe they have never been tested for old OS like XP.

Does anybody have actual means of testing any of the AGS games using these plugins, under XP?

[Crimson Wizard]

Okay, I actually got this dump to load properly! And, this is totally bizarre, but it says that engine is the NULL pointer.

Monkey, the problem with these mini-dumps is that they do not have full memory contents. Which means that every variable can be displayed as zero. You'd need to make AGS write a full dump (but it will be many megs large, or even hungreds of megs).

Does anybody have actual means of testing any of the AGS games using these plugins, under XP?

I have a Windows XP on a virtual machine, I'll give it a try.

[monkey0506]

No log again. Dump:

https://bitbucket.org/Maciek_Tr/sosnowy-las/downloads/CrashInfo.3.3.0.1162a.dmp

This latest dump is crashing when the std::ofstream is being constructed, citing engine being NULL, but engine isn't even used in that constructor:

Code: C++

1.     static std::ofstream ofstream{ "agsteam_debug.log", std::ios::trunc };

AGSteam-debug-3.zip doesn't even use engine in AGS_EngineStartup. Instead, it operates directly on the parameter lpEngine (which should be the same value, regardless). It attempts to create the debug log as soon as AGS_EngineStartup is entered, before any parameters are inspected or touched.

If this crashes, there is one other approach you could try, which is simply running Heroine's Quest with the AGS 3.4.0 engine. To do that, you would need to install (zip archive or installer) AGS 3.4.0, then copy the "acwin.exe" from the AGS folder into the Heroine's Quest folder. Drag and drop "Heroine's Quest.exe" onto "acwin.exe", and it will launch the game using the AGS 3.4.0 engine. The Steam version of Heroine's Quest does run on Windows 7 with the AGS 3.4.0-P4 engine, but I don't have Windows XP installed to test that environment. I'll see if I can get an XP VM running.

[monkey0506]

Monkey, the problem with these mini-dumps is that they do not have full memory contents. Which means that every variable can be displayed as zero. You'd need to make AGS write a full dump (but it will be many megs large, or even hungreds of megs).

Well in that case, are the dumps even useful at this point?

[Crimson Wizard]

Well in that case, are the dumps even useful at this point?

They were mainly useful to know the exact point and callstack of crash, which could help reproducing it (sometimes).
I think Chris Jones did not want to surprise users with 100-300 megs large dump (which would also be difficult to send through the internet in the early years of AGS).

You could make another version of the engine which writes full dump, but first make sure that's okay for eMTe to upload huge files .

[monkey0506]

You could make another version of the engine which writes full dump, but first make sure that's okay for eMTe to upload huge files .

Assuming he'll read this, any hint on how to go about "mak[ing] another version of the engine which writes full dump"? I clearly don't know anything about this, so I don't even have any idea where in the engine code (or solution settings, maybe?) that I would go about doing this. Thanks for the explanation.

They were mainly useful to know the exact point and callstack of crash, which could help reproducing it (sometimes).

Oh, and in this case, the dump from AGSteam-debug-2.zip's plugin indicates that the crash occurred at the point that the log file is supposed to be created by the std::ofstream constructor. If the ofstream constructor is actually failing, then presumably I should be able to catch an std::exception, wouldn't you think? I can try and decouple opening the file from the constructor (just initialize the ofstream as an empty object) then put the ofstream.open() call in a try/catch block. I was working on the assumption that the error must be coming from before the ofstream constructor begins, but maybe that's an asinine assumption?

[Crimson Wizard]

I just reproduced same crash on Windows XP. Dump is created, but log was not.
So this is system-related actually. I wonder if this may be related to standard library configuration, or something like that.
EDIT: also tested with the most recent engine (3.4.1), similar crash at startup, only EIP is different (because engine code has changed).
EDIT3: I just thought I've already seen standard C++ library functions crashing before. That happened when one component of program was linking to debug version of std library and another to release version.


Regarding crash dump, it is created using "MiniDumpWriteDump" function (you can search for it in the engine). If I remember correctly, it is all about flags you pass in. More info may be found here:
https://msdn.microsoft.com/en-us/library/windows/desktop/ms680360(v=vs.85).aspx
http://ntcoder.com/bab/2014/10/14/how-to-create-full-memory-dumps-using-minidumpwritedump/

EDIT2: I am actually thinking we could add a config option to the engine, switching dump type, to avoid the need of different engine builds.

[monkey0506]

EDIT3: I just thought I've already seen standard C++ library functions crashing before. That happened when one component of program was linking to debug version of std library and another to release version.

Well there are three assemblies to consider here:  acwin.exe, AGSteam.dll, and steam_api.dll. acwin.exe and AGSteam.dll both have static linkage to the STL, though they are different versions of the STL. steam_api.dll... well, it could have dynamic or static linkage to the STL, I'm not really certain that it's documented anywhere. Given that Steam is meant to work on a wide range of systems, it seems probable that they would have static linkage, and the file size of steam_api.dll seemingly supports this, but none of that is a guarantee. Regardless of whether steam_api uses static or dynamic linkage though, would different assemblies using their own version of the STL be problematic? If that were the case then I would think steam_api.dll would be unusable. AGSteam is a single assembly which links against steam_api, so I don't see how any part of AGSteam could have the wrong linkage to the STL, if that's what you were suggesting.

[Crimson Wizard]

There is something I did not research fully yet, but when I began using VS2015 I learnt that you may need a specific configuration if you want to target Windows XP. May this be related?
https://msdn.microsoft.com/en-us/library/jj851139.aspx

The Windows XP platform toolset that's included in Visual Studio is a version of the Windows 7 SDK that was included in Visual Studio 2010, but it uses the current C++ compiler. It also configures project properties to appropriate default values—for example, the specification of a compatible linker for down-level targeting. Only Windows desktop apps that are created by using the Windows XP platform toolset run on Windows XP and Windows Server 2003, but those apps can also run on more recent Windows operating systems.

[monkey0506]

Interesting find, CW. I had done some high-level skimming of MSDN and hadn't come across that particular article (or the enclosed topic). I'll see if I can get something working with that, now that I also have XP installed in a VM.


Edit: Unfortunately, still no dice. Even with the "v141_xp" toolset selected, I get the exact same crash before the plugin is able to create the debug log.

Edit 2: Just to test that the AGSteam assembly itself is working, I've created an executable version of AGSteam. This is meant to test only whether the AGSteam assembly can run successfully on your system. Extract the zip somewhere you have write permissions (e.g., "Desktop/AGSteam-debug/"), then run "AGSteam.exe" from the Command Prompt. You should get exactly this output (the number after "Caching Steam ID:" may actually differ, this is printed by Steam so I'm not sure) in the console window:

calling engine startup
ofstream.is_open()? true
writing to debug log...
finished writing to debug log
Calling AGSteam::Startup()
Setting breakpad minidump AppID = 283880
Steam_SetMinidumpSteamID:  Caching Steam ID:  76561198046544449 [API loaded no]
AGSteam::Startup complete, calling AGSteam::Shutdown()
AGSteam::Shutdown complete, exiting AGS_EngineStartup
done!

And you should have the much sought after "agsteam_debug.log":

AGS_EngineStartup called, engine = '0x00000000' (not yet initialized) and lpEngine = '0x00000000'
AGSteam::GetClient called, returning address '0x005738CC'
Steam not initialized, calling SteamAPI_Init()
SteamAPI_Init() succeeded, creating UserStatsReceivedListener
UserStatsReceivedListener created, requesting current stats. SteamUserStats = '0x00CF07B0'
User stats requested, AGSteamPlugin_Initialize() complete

In this case, the engine and lpEngine pointers are meant to be NULL, as the AGS engine isn't even running.

I've tested this on Windows XP, using the "v141_xp" toolset in Visual Studio, and everything appears to be working exactly as expected here. If this also works for you, then my suspicion falls back to the AGS engine failing to load the plugin properly, despite the fact that the crash dump says the DLLs are both loaded.