tap version 0.1.0 (https://ci.appveyor.com/api/projects/status/2qakyfoixd50r78g/branch/main?svg=true) (https://ci.appveyor.com/project/ericoporto/tap)
Get Latest Release
tap.scm (https://github.com/ericoporto/tap/releases/download/0.1.0/tap.scm) | GitHub Repo (https://github.com/ericoporto/tap) | Demo Windows (https://github.com/ericoporto/tap/releases/download/0.1.0/tap_demo_windows.zip) | Demo Linux (https://github.com/ericoporto/tap/releases/download/0.1.0/tap_demo_linux.tar.gz) | Download project .zip (https://github.com/ericoporto/tap/archive/0.1.0.zip)
Test Anything Protocol - AGS ModuleThis Module implements Test Anything Protocol for Adventure Game Studio. It's useful for unit testing! You can read more about it here: testanything.org (https://testanything.org/)An example test file output by this libary may look like:1..4
ok 1 - Input file opened
not ok 2 - First line of the input valid
ok 3 - Read the rest of the file
not ok 4 - Summarized correctly # TODO Not written yet
The demo game will execute and exit, leaving a
agstest.log file, with the test results, under this game savegame directory (ex:
C:\Users\%USERNAME%\SAVEDG~1\tap_demo). This module is more useful to other people building modules and plugins, who wish to unit test in AGS Engine itself.
MotivationA long time ago, Morgan talked to me about tests and unit testing, and showed me this magical thing that is TAP. Time passed and after experimenting with a bunch of different continuous integration systems I discovered they were able to magically pickup the specific output from tap and understand them when they are output to the CI shell... A bit of fiddling around later I got it working with AGS! This has been laying around in my repository for quite sometime and I noticed I never released it as a module. So here it is!
The planThe plan tells how many tests will be run, or how many tests have run. It’s a check that the test file hasn’t stopped prematurely. It must appear once, whether at the beginning or end of the output. The plan is usually the first line of TAP output (although in future there may be a version line before it) and it specifies how many test points are to follow. For example,
1..10
means you plan on running 10 tests. This is a safeguard in case your test file dies silently in the middle of its run. The plan is optional but if there is a plan before the test points it must be the first non-diagnostic line output by the test file. In certain instances a test file may not know how many test points it will ultimately be running. In this case the plan can be the last non-diagnostic line in the output. The plan cannot appear in the middle of the output, nor can it appear more than once.
The test lineThe core of TAP is the test line. A test file prints one test line per test point executed. There must be at least one test line in TAP output. Each test line comprises the following elements:
ok or not ok
This tells whether the test point passed or failed. It must be at the beginning of the line. /^not ok/ indicates a failed test point. /^ok/ is a successful test point. This is the only mandatory part of the line. Note that unlike the Directives below, ok and not ok are case-sensitive.
Test numberTAP expects the ok or not ok to be followed by a test point number.
DescriptionAny text after the test number but before a # is the description of the test point.
ok 42 this is the description of the test
DirectiveThe test point may include a directive, after a hash on the test line. There are currently two directives allowed: TODO and SKIP. Hashes at the beginning of a line are considered comments.
Executing tap with AGSBefore running any test, remove any leftover files and from a previous run, by running a clean test.
tap.clean_test()
By default, test is printed to a file named
agstest.log on the saved games directory for the game under test.
If you have a plan, state it before starting out.
tap.plan(3);
Then run the needed tests.
String null_string;
tap.ok(String.IsNullOrEmpty(null_string), "String.IsNullOrEmpty test null string");
tap.ok(String.IsNullOrEmpty(""), "String.IsNullOrEmpty test empty string");
tap.ok(!String.IsNullOrEmpty("with text"), "String.IsNullOrEmpty test non-empty string");
After you are done testing, state it.
tap.done_testing();
TAP AGS Script APISpoiler
tap.clean_test
void tap.clean_test()
This command removes previously outputed test log files if they exist.
tap.plan
void tap.plan(int n, String skip_reason = 0)
This allows setting a plan, so we know how many tests will be run. Just use it once for the group of tests following. If you want to start a new test, after a plan has been set, remember to call tap.done_testing() before.
Alternatively, you can pass NO_PLAN if you have no plan.
If the following tests are to be skipped, pass [SKIP_ALL, so all tests until next plan are skipped. They will still be executed, they just won't output log or be considered whether they fail or succeed.
tap.ok
void tap.ok(bool condition, String test_name = 0)
This test will succeed if the condition is true, and fail if the condition is false.
tap.nok
void tap.nok(bool condition, String test_name = 0)
This test will succeed if the condition is false, and fail if the condition is true.
tap.is
void tap.is(String got, String expected, String test_name = 0)
This test will succeed if got and expected matches. If this test fails, we can have some more diagnostic information since we know both inputs and the premise they are supposed to match.
tap.isnt
void tap.isnt(String got, String expected, String test_name = 0)
This test will succeed if got and expected does not matches. If this test fails, we can have some more diagnostic information since we know both inputs and the premise they are not supposed to match.
tap.is_int
void tap.is_int(int got, int expected, String test_name = 0)
This test will succeed if got and expected matches. If this test fails, we can have some more diagnostic information since we know both inputs and the premise they are supposed to match. This is for int values.
tap.isnt_int
void tap.isnt_int(int got, int expected, String test_name = 0)
This test will succeed if got and expected does not matches. If this test fails, we can have some more diagnostic information since we know both inputs and the premise they are not supposed to match. This is for int values.
tap.is_float
void tap.is_float(float got, float expected, float epsilon, String test_name = 0)
This test will succeed if got and expected matches. If this test fails, we can have some more diagnostic information since we know both inputs and the premise they are supposed to match. This is for float values.
tap.isnt_float
void tap.isnt_float(float got, float expected, float epsilon, String test_name = 0)
This test will succeed if got and expected does not matches. If this test fails, we can have some more diagnostic information since we know both inputs and the premise they are not supposed to match. This is for float values.
tap.done_testing
void tap.done_testing()
After you are done testing, before calling the next plan, state you are done testing.
tap.Harness
void tap.Harness()
This outputs a small summary of currently tests and failures. Don't depend on this output format.
AuthorMade by eri0o
LicenseDistributed under MIT license. See
LICENSE (https://github.com/ericoporto/tap/blob/main/LICENSE) for more information.
Pretty cool. Some stretch goals would be to build an editor plugin that gives a test explorer similar to the one in IDEs like Visual Studio, and getting test coverage from the test results.
Ha! That's a cool goal! Parsing tap output with C# is super easy - and if I am truly lazy someone has probably made a tap consumer in c# already.
So the test panel would basically be picking up the output file and making a pretty panel with green and red things to show what ran and what failed, this should be easy.
For coverage, I think I would do in a boring way, which would be in the tap comment (things after #), verify that the function is mentioned and require the test file (for coverage usage) be named like (modulename)_test.asc, then it's possible to use the editor to pickup the functions in modulename.asc so we can see what's covered and what's not - and I would do a little checkbox to disable coverage in the panel (laugh)
Code coverage comes in different standard formats which CI systems need to present the data, Cobertura, Lcov, etc.
I think I would aim at only Function coverage, since things like Line coverage would (at least in my mind) require some static analysis tooling - to be able to check the paths. Function coverage is easy to add, line coverage feels harder.
I can't seem to find a report that CIs expect, the implementations I found kinda depend on the good will of the CI to support your language :(
Ah, not exactly statically analysis btw, but if you do AGS Script using Atom, with Edmundito's AGS Script package for it, you can install an Atom linter and code AGS with linting and some quick fix options which makes it pretty cool!