Test game

Started by morganw, Sun 18/03/2018 16:39:06

Previous topic - Next topic

morganw

Have there every been any test games, which just implement and test all the scripting functions? The reason I ask, I've probably spent more time in AGS writing scripts to test than in ScummVM actually testing them. I imagine test scripts could also be useful for the normal branch, the cleanup branch, and for XAGE. I'd like to suggest that the GitHub repository hold a test game along with the demo game (or potentially just make the demo game be a function showcase/tester, rather than an actual game).

Clarvalon

This would be incredibly useful for XAGE, a test AGS game that used all existing engine functionality.  It wouldn't necessarily need to be split into individual tests and assertions, it could instead be long cutscenes that performs various actions and then outputs the results to file.  This could then be compared side-by-side with the equivalent XAGE port output to more easily identify missing (or broken) scripting functionality.
XAGE - Cross-Platform Adventure Game Engine (alpha)

Alan v.Drake

Some time ago I made a plea to the community for making one. No luck so far.
At the moment I'm not working on AGS so I haven't been following the latest development closely. :(

- Alan



morganw

Has tzachs got any test framework for MonoAGS?

Crimson Wizard

#5
Quote from: morganw on Mon 19/03/2018 20:48:55
Has tzachs got any test framework for MonoAGS?

He has autotests, based on NUnit Framework.
I am not sure that is relevant, because MonoAGS does not support AGS scripts or its syntax.

morganw

It was more for an idea of what he is testing. I imagine there is enough script functions for the mouse that you could probably make a game play itself, but since the ScummVM main event loop doesn't match the original, just checking the timing of functions and blocking actions are similar between the two would be useful. If XAGE also supports script modules, perhaps those sort of tests could be implemented in separate modules, or also used as examples in the manual? The majority of the audio functions should be working, so I'll try and make something to test those to start with.

tzachs

Quote from: morganw on Tue 20/03/2018 18:34:52
It was more for an idea of what he is testing.
I'm only doing unit tests, I don't currently have any game-testing frameworks, though I would definitely want to have something like that eventually. What I would like to have in my fantasy game testing framework:
- Very stupid bot: it would randomly left/right click in various locations in the screen, trying to make weird things happen.
- Stupid bot: it would randomly trigger interactions with objects in the screen, trying to make weird things happen.
- Scripted tests: either by triggering interactions directly or using programmatic mouse clicks, and assertions will be made against pre-made screenshots (no clue on how to test audio).
- Benchmark tests: will make sure there are no performance regressions.
- Tests should be able to run in a docker container in some kind of headless OpenGL window, so it can run from the build server.
- Tests should record a video so we'll be able to see what went wrong.
- Dumb & Dumber bots should record all clicks and allow playback to reproduce errors (which can't be perfect because of timing, but might be useful on occasions)
- Dumb & Dumber bots should be generic and configurable so that they could be used by game developers to test their game.
- If we have the money, then we'd like to run the tests on xamarin test cloud which runs the tests on a lot of different real mobile devices, which will help us find issues with specific devices.

SMF spam blocked by CleanTalk