Testing & QA

Five layers of checks, from two seconds of ctest to a two-hour campaign playthrough, and which of them to run for which change (as of 2026-11-10).

Compiling is not evidence of visual parity: look at the result.

ctest

ctest --test-dir build-cna-opengl3 -N                     # list the tests
ctest --test-dir build-cna-opengl3 -j4 --output-on-failure
ctest --test-dir build-cna-opengl3 -R VerifyShippedWorlds  # one test

CMakeLists.txt registers 32 tool tests (23 with add_test, nine through ge_add_editor_verify) and the GoogleTest target GalaxyEggbertWorldsTests, whose 83 TEST cases gtest_discover_tests turns into one ctest entry each. The easy-gl dependency adds five tests of its own. A native build directory listed 120 tests on 2026-11-10; CURRENT.md's last full verification line records 117/117 on 2026-11-09, before the latest additions. VerifyDemoReplays is registered only when CMake finds xvfb-run.

GoogleTest: GalaxyEggbertWorldsTests

Sources in tests/GalaxyEggbert/: Block, BitPacking, Chunk, World (serialization round trips, rejection of bad files), BlockMetadata, MoveObjectRecord, BigDecorRecord, HintZoneRecord, HubDoors. They link only the engine-independent model, no CNA.

Tool tests (tools/Verify*.cpp)

Plain main() programs that print PASS:/FAIL: lines and return non-zero on failure.

Adding a test

1. Write tools/VerifyYourThing.cpp in the check(cond, "what") pattern of tools/VerifyCameraShake.cpp. Keep it deterministic, without audio or windows.
2. In CMakeLists.txt, add an add_executable with the sources it needs (plus ${GE_SHARED_SOURCES} for the world model), the include directories include and src, and an add_test inside if(BUILD_TESTING), with WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} when it opens repository paths. Tests that touch CNA types copy the link pattern of VerifyInteractionSystem; editor tests use ge_add_editor_verify.
3. For a world-model unit test, add a TEST under tests/GalaxyEggbert/ and list the file in GalaxyEggbertWorldsTests.

Golden frames and trace

tests/golden/ holds three screenshots of the showcase world (golden_frame_0060.png, _0120, _0180) and a movement trace (golden_trace.txt); tests/golden/effects-medium/ and effects-high/ hold the same three frames with the 3D effects at those levels. The comparison is byte for byte.

xvfb-run -a tools/verify_golden_frames.sh build-cna-opengl3
xvfb-run -a tools/verify_golden_trace.sh build-cna-opengl3

verify_golden_frames.sh runs the game with --golden-capture three times (GALAXY_EFFECTS=0, medium, high) and compares each set with its references. verify_golden_trace.sh runs --golden-capture-trace, which also walks and jumps Blupi through a fixed script and records position, vertical speed, ground state and animation per tick. Both need a real or virtual display; neither is in ctest or CI.

Autoplay playthroughs

A script plays a mission tick for tick against the real game (--mission N --autoplay <script>) and checks conditions; see Autoplay.

tools/autoplay.sh build-cna-opengl3 tools/autoplay/world012.txt 12     # one mission
tools/autoplay_all.sh build-cna-opengl3                                # every worldNNN.txt
tools/autoplay_all.sh build-cna-opengl3 world011 world121              # chosen ones
tools/play_pass.sh build-cna-opengl3 <out-dir>                        # all, with contact sheets

autoplay.sh runs the game under its own Xvfb display from a run directory (default build-probe/autoplay-run/, or AUTOPLAY_RUN=<dir>) whose asset directories link to the repository's, with a fresh save and GALAXY_USER_DIR set to the run directory. It prints autoplay_trace.txt and exits with the game's code: 0 when the script passed. Silent (SDL_AUDIODRIVER=dummy) unless QA_SOUND=1; AUTOPLAY_WORLDS=<dir> substitutes worlds3d/; AUTOPLAY_REALTIME=1 paces at 60 Hz instead of flat out; AUTOPLAY_SHOTS=N saves a frame every N ticks. Extra game arguments follow the mission number. autoplay_all.sh skips a script whose first line says BLOCKED.

tools/autoplay/ has one script per campaign world (world001.txt to world199.txt); a theme hub's script (world010.txt...) includes its missions and plays the hub as a chain to its golden key. campaign.txt (written by python3 tools/autoplay_campaign.py) plays a new game through all twelve worlds and world 199. probes/ holds one-off scripts, spare/ alternate routes.

The script language

The authoritative text is the header comment of src/GalaxyEggbert/Game/Autoplay.hpp. One command per line, # comments, raw grid coordinates (a cell's centre is its integer value), yaw in degrees (0 faces -Z, 90 faces +X).

A condition is FIELD OP VALUE with = != < <= > >=. Fields: x y z yaw tick mtick mission treasures eggs lives keys dynamite vehicle ground dead ghost anim exit blitz temp switches saws crush phase (vehicle is accepted by the parser but missing from the header comment). Use mtick (ticks since the current mission started), never tick, in anything that can run inside a chain. A command that runs out of ticks, or Blupi dying before mortal, fails the run. tools/autoplay_search.py tries a wait {D} over a range and tools/autoplay_chains.py re-times synced chains after a timing change.

Last recorded results: the handbook records a full pass of all 78 campaign scripts on 2026-11-07 (CURRENT.md still names 2026-11-06 as its baseline); the whole-campaign run (campaign.txt) was last recorded on 2026-10-27.

Demo verification

The main-menu demos (Content/demos/demoNNN.gdemo, see Demos & Recordings) are recorded input, so a simulation change can make them replay differently. VerifyDemoFiles (VerifyDemoReplay --shipped) checks that every shipped demo parses with the current simulation revision and names its world's current bytes; tools/verify_demos.sh <build-dir> (ctest VerifyDemoReplays) replays each one in the game under Xvfb with --verify-demo, compares its end state, and checks that the stand-in player files were left untouched. When a demo reports "different", bump kDemoSimulationRevision (DemoReplay.hpp) and record the demos again with tools/record_demo.sh.

Live QA

tools/qa_live.sh <build-dir> <out-dir> <steps-file> [game args] runs the game under its own Xvfb display (needs xvfb-run, xdotool and ImageMagick's import) and executes a step file: sleep S, shot NAME, key KEYSYM..., keydown/keyup, click X Y, mousedown/mouseup, move X Y, focus, type WORD, resize W H. QA_LIVE_RUN, QA_LIVE_AVATARS, QA_LIVE_WORLDS and QA_LIVE_SCREEN choose the run directory, substitute asset directories (to look at a re-exported model or a probe world before it replaces the committed one) and size the display. Editor step files live in tools/qa_steps/ (editor_playtest.txt, editor_copy_paste.txt, editor_lift.txt, editor_backups.txt...).

• tools/qa_sweep.sh <build-dir> [out-dir] loads every shipped world, walks two seconds, takes a screenshot and writes one line per world to result.txt.
• tools/qa_original.sh <port-dir> <out-dir> <steps> drives a copy of the original game's Linux build with the same steps, to see what the original does where its source leaves it open.
• python3 tools/verify_lava_depth.py <build-dir> is a real-render regression (BD-0114: lava drawn over a translucent ghost Blupi).

Python world checkers

Run from the repository root; all are read-only.

CI

.github/workflows/cna-ci.yml runs one job, "Build + ctest" on ubuntu-latest: it checks out this repository and the siblings (cna, cna-engine, easy-gl, sharp-runtime, meta-gl) at pinned commits, plus a sparse checkout of mobile-eggbert's worlds/ as test data, installs the system packages, configures with -DGALAXY_EGGBERT_BUILD_CNA=ON and ccache, builds, and runs ctest --output-on-failure. Every registered test runs. Golden files, playthroughs, live QA and the web build are local, manual steps. .github/workflows/pages.yml deploys this website.

Which checks for which change