Author Topic: Porting engine to ScummVM  (Read 246 times)

Porting engine to ScummVM
« on: 19 Nov 2020, 18:42 »
Hey all. I'm a long-time ScummVM developer, and am currently (hopefully :) ) in the finishing stages of the current engine I'm working on. So I've been starting to cast around for a project to work on over the Christmas holidays, and thought it might be time to revisit the question of integrating AGS support into ScummVM. The previous threads in years past got somewhat contentious at times, but my understanding was that there was certainly overall support from the AGS community for older games to be supported at the very least. Though technically there are two options, with both the community codebase and fuzzie's earlier attempt, it seems like the community codebase is the way to go in that it covers a greater range of historical versions.

As such my question before I start such an effort, is whether there would be any issues with doing a conversion now. Particularly, I'm wondering what codebase point would be best to be used. I see that there's an ags3 branch, but it seems to be out of date compared to master, which has later 3.5 releases. Would the most 3.5 release tag be a good point to start for a ScummVM conversion? Though I'm also wondering if there's a near-term plan for a 4.0 release? Particularly if that ends up significantly changing the AGS format, would it be better to forget about it for now, and wait for the last 3.x release prior to working on a conversion? Something like that could end up being ideal, since there'd be less issues with a ScummVM implementation trying to play catch up with further commits and sub-releases on the main AGS repository.

Any guidance would be appreciated.

Re: Porting engine to ScummVM
« Reply #1 on: 20 Nov 2020, 00:51 »
Hello, there's a close to complete (or so I believed) port to ScummVM here: https://github.com/adventuregamestudio/scummvm/tree/ags

It was started by Alyssa Milburn (aka fuzzie) as you know, then amended by several people in different time, including some of our community. We decided to bring a fork to our organization account hoping to finish it maybe and provide an alternative for running old games, but it was never done 100%.

I am unaware of it's exact status, IIRC the current goal was to cover AGS 3.2.1 API, but some things are still missing. I recall that the biggest missing feature was save/restore function.

Though technically there are two options, with both the community codebase and fuzzie's earlier attempt, it seems like the community codebase is the way to go in that it covers a greater range of historical versions.

I believe it might be possible to copy necessary changes over to existing scummvm port, because every game logic in current ags code is under version check, so it's not hard to find parts related to this or that API. But a research may be required first to know what to do with some recent additions such as custom viewport/camera features and how to reimplement them in ScummVM (guess you would know better since you are a ScummVM developer).

As such my question before I start such an effort, is whether there would be any issues with doing a conversion now. Particularly, I'm wondering what codebase point would be best to be used. I see that there's an ags3 branch, but it seems to be out of date compared to master, which has later 3.5 releases. Would the most 3.5 release tag be a good point to start for a ScummVM conversion? Though I'm also wondering if there's a near-term plan for a 4.0 release? Particularly if that ends up significantly changing the AGS format, would it be better to forget about it for now, and wait for the last 3.x release prior to working on a conversion?

This is a very good question... Trying to explain in short:
- "ags3" is a forgotten branch, as in, I forgot to delete it apparently (we had number of hickups with planning). Conceptually, current master is ags3, as it continues full backward compatible support.
- There were plans to have small update in there, like 3.5.1, with certain API additions, but recently I leaned towards dropping that. The situation is explained here: https://github.com/adventuregamestudio/ags/issues/1121 in the ticket and comment(s).
- In shorter words, what we would like to do is this: 3.5.0 is the last "ags3" version (as in - version with full backward support). From now on this branch will only have fixes and perfomance updates, but no API, data format and game logic updates whatsoever.
- "ags4" becomes master, we make breaking changes there (cutting old stuff, redesigning some script commands etc) and this is where we do most work from now on. note "ags4" is just a meme at this point, as version number is currently not decided (imho differences are not enough to call it 4.0).
- we are currently in a big process of replacing backend from Allegro 4 to SDL2 in the master, if successful this will be AGS 3.6 with full backward support, but because this is only backend & library changes it's not needed for ScummVM port.
(We will also use SDL2 in the future "ags4" branch, ofcourse)

I was just planning to make a forum post about all above, which might still do in a few days.
« Last Edit: 20 Nov 2020, 01:04 by Crimson Wizard »

Re: Porting engine to ScummVM
« Reply #2 on: 20 Nov 2020, 01:48 »
Thanks for the informative reply. It sounds like it'll be more effective for me to work on the AGS3 "final version", after the switchover. Whenever that happens. I'm also familiar with SDL, and not at all with Allegro, so it would certainly make it much easier to make a port for ScummVM afterwards. Guess I'll hunt up something else to work on over Christmas.

Re: Porting engine to ScummVM
« Reply #3 on: 20 Nov 2020, 02:56 »
SDL2 port did not change the game code alot (less than I even expected), as most system actions are kept behind interfaces, and we are still having parts of the stripped allegro 4 library embedded into the engine: primarily memory bitmaps and software drawing/blending functions, because SDL2 does not have a good alternative for these.
The biggest changes were to the window management and audio backend, latter was completely rewritten, but again the game logic code remained untouched.
« Last Edit: 20 Nov 2020, 03:03 by Crimson Wizard »

Snarky

  • Global Moderator
  • Global Moderator
  • Mittens Lord
  • Private Insultant
    • Best Innovation Award Winner 2018, for his numerous additions to the AGS open source ecosystem including the new Awards Ceremony client and modules
    • Snarky worked on one or more games that won an AGS Award!
    •  
    • Snarky worked on one or more games that was nominated for an AGS Award!
Re: Porting engine to ScummVM
« Reply #4 on: 20 Nov 2020, 07:05 »
Thanks for the informative reply. It sounds like it'll be more effective for me to work on the AGS3 "final version", after the switchover. Whenever that happens.

Hmm, AGS 3.5.0 is already the current released version, and as Crimson Wizard said that is the "final" AGS3 version (according to plans), except for bugfixes and performance improvements and such.

Maybe you could save a little bit of time/effort by waiting for the SDL port, but given that there already is a mostly-complete ScummVM port by fuzzie, and that the bits that are missing are not (as I understand it) related to Allegro/SDL, I would encourage you to just go ahead and do a port of AGS 3.5.0 with that as a basis.

It's great to see some renewed interest in a ScummVM port, and it would be a shame to let the opportunity pass by.

Re: Porting engine to ScummVM
« Reply #5 on: 21 Nov 2020, 15:40 »
I played around with the fork of fuzzie's work on adventuregamestudio/scummvm, and the results are just too poor for my liking. Of the games that had detection entries, one crashed trying to load data, one threw a wobbly about slider value, and the final one did manage to start showing the intro sequence but then crashed with "FollowCharacter unimplemented". It's just unfeasible for someone like me, with no prior experience with the AGS engine codebase, to try and debug and update a semi-restructured/different project. It'll result in endless frustrations for me. Far better to have an almost identical codebase based on the current community engine, where the backend Allegro or SDL calls have a ScummVM stub layer. That way if something goes wrong, I'll be able to compare execution between the two if something goes wrong in the ScummVM version. That kind of thing has been helpful in the past. For example, I once hacked DosBox to do output of port values written to the sound card in one of the Tsunami Media games, and doing likewise in my sound driver code in ScummVM, I was able to do a diff, and easily track down where discrepancies in my code started happening. But I digress. That there aren't any more major changes planned for 3.5.. it may indeed be the case that there's not much to be gained waiting for any SDL changes. I'll have a play around with the codebase, familiarize myself, and see how I go.

Re: Porting engine to ScummVM
« Reply #6 on: 21 Nov 2020, 16:01 »
The biggest problem of the current codebase in ags engine is that it's very dirty, and fuzzie did a big job refactoring it into cleaner code. I think I must mention this. Over time we were gradually replacing old code with new one, especially when adding more features, but there are still places that look just terrible.

Re: Porting engine to ScummVM
« Reply #7 on: Today at 17:43 »
The biggest problem of the current codebase in ags engine is that it's very dirty, and fuzzie did a big job refactoring it into cleaner code. I think I must mention this. Over time we were gradually replacing old code with new one, especially when adding more features, but there are still places that look just terrible.

That makes sense. It's a trap I've fallen for myself in the past - back when I was converting a previous project to work on ScummVM I started making too many changes, then when the time came to debug it when I got it up and running, it became difficult to track down issues. I guess it was even harder if things were actively being changed in the source codebase. These days when I'm converting projects I initially tend to try and keep the original code as unchanged as possible, and simply create a thunk layer to emulate the original libraries. Then only worry about any cleanup once I've properly gotten everything to at least run. Over the last few days I've created a ScummVM skeleton engine, and seen how much code I code import and get to compile. It's been fun, although frustrating in that I still need to put it aside to finish my current project. But it seems like once that's done, I likely will experiment over the Christmas holidays to see if I can get it working for ScummVM.

Snarky

  • Global Moderator
  • Global Moderator
  • Mittens Lord
  • Private Insultant
    • Best Innovation Award Winner 2018, for his numerous additions to the AGS open source ecosystem including the new Awards Ceremony client and modules
    • Snarky worked on one or more games that won an AGS Award!
    •  
    • Snarky worked on one or more games that was nominated for an AGS Award!
Re: Porting engine to ScummVM
« Reply #8 on: Today at 19:41 »
Hey, that's great to hear! Really hope it works out. Interesting to hear about your approach to converting projects; that makes a lot of sense to me.

I don't know whether it was intentional or a typo, but I also learned a new term.

Re: Porting engine to ScummVM
« Reply #9 on: Today at 20:01 »
I don't know whether it was intentional or a typo, but I also learned a new term.

Yep, that was intentional. Call it a thunk, bodge, or shim layer. Heh, at times previously at work we called some of our nastier interface code the "Wibble" layer, from the Rowan Atikinson Black Adder series where he pretended to be insane by sticking pencils up his nose. :)