Skip to content

Testing and contributing ​

Test suites ​

CommandWhat it provesTime
scripts/determinism-test.shEvery case gives identical video, audio and GPU-upload hashes on wasmtime JIT, wasmtime AOT and V8 (Node), equal to the golden hashes in tests/golden/determinism.txt~3.5 min
scripts/net-test.shFull online sumo matches through gasm-relay for native↔native, native↔Node, Node↔native, and native↔Node over TLS (wss://, throwaway CA) end in the identical game state, with no desync~7 s
make testBoth of the above
make parity ROM=… FRAMES=…The NES game built natively (Rust, stub host) matches the wasm build~1 min first build
scripts/asset-test.shFile-backed assets and folders: --asset-dir == --asset list == Node (lazy and in-memory); prefix form, hidden files, precedence; a 200 MB asset streamed at random offsets has the in-memory hash, with max RSS measured against a tiny asset (LARGE_MB=0 skips)~10 s
node scripts/opfs-test.mjs [MB]Headless Chrome: csfs imports a data set into OPFS, assetcheck runs from it in Worker mode with lazy OPFS reads; hash == Node --asset-dir; renderer memory sampled while streaming 10 GB of random reads~60 s
node scripts/gfx-model-test.mjsgasm:gfx validation on the JS runner: the cases in tests/gfx-cases.json, which cargo test --lib gfx replays natively (both must accept and reject the same calls)< 1 s
scripts/gl-native-test.sh [--gl-software]gasm:gl on ANGLE in gasm-run (fetches ANGLE): gltest's hashes equal the null GL's golden ones, and the frame isn't blank~5 s
node scripts/web-smoke.mjs <url> <out.png> [secs]The browser runner loads and runs in headless Chrome (including WebGPU); prints status, fps and console, saves a screenshot~10 s

Linux locally (Apple container) ​

On Apple silicon (macOS 26+), the whole suite also runs on Linux arm64 in a container. It uses your read-only source; builds go to a named volume, so your macOS target/ dirs are untouched.

sh
scripts/linux-container.sh            # build + all tests (NET_REPEAT=5 to stress the network test)
scripts/linux-container.sh build      # just build; binaries land in dist/linux-arm64/
scripts/linux-container.sh shell      # interactive shell in the synced tree
scripts/linux-container.sh clean      # remove the volumes, image and builder (several GB)
CPUS=8 MEMORY=12g scripts/linux-container.sh

The image (container/linux.Containerfile: Debian + Node 22 + rustup) holds only toolchains. It also works with Docker/Podman (docker build -f container/linux.Containerfile container/). Headless tests need no display or GPU.

CI ​

ci.yml runs every suite above (plus gen-abi.mjs --check, the gasm-host unit tests and a build without the window feature) on ubuntu-24.04. The job platforms then takes the games built there and runs the determinism, network and asset tests on windows-2022, macos-14 and ubuntu-24.04-arm. Every platform must match the same golden hashes, so the runners on Linux (x86_64, arm64), macOS and Windows are verified bit-identical. Every download the scripts make (toolchains, engine sources, ROMs) is pinned by SHA-256 (scripts/lib.sh).

Browser checks by hand ​

Worker and main-thread modes against the Node runner: index.html?game=nes.wasm&rom=cpu_instr_test.nes&worker&hashframes=600&autostart prints frames=… video_fnv32=… audio_fnv32=…, which must equal node runners/web/headless.mjs build/nes.wasm --rom roms/cpu_instr_test.nes --headless 600. Verified in Chrome. Safari and Firefox haven't been run through the automated suites yet: Safari needs safaridriver --enable, and Firefox isn't set up here. Open the same URLs there to check by hand.

Determinism test ​

26 cases:

  • Test guests: the C test pattern (pad, keys, pointer); textured with texture uploads, dynamic offsets, a storage buffer and scripted text input; inputtest with keys and modifiers, pointer, wheel, motion and two gamepads.
  • Own main loops: the Rust (loopdemo) and C (loopdemo-c) loop helpers, playing until START is held, then returning from main.
  • SDL 3: snake and woodeneye-008 from SDL, a callbacks app with audio and gamepad events, a classic main() loop with text input and a save file.
  • GPU guests: triangle (one buffer, one pipeline) and sumo vs. the bot with scripted input (GPU uploads are hashed, covering the whole scene).
  • NES: the blargg CPU instruction, CPU timing and APU tests, spritecans, Quantum Disco Brothers, and 2,400 frames of Blade Buster gameplay.
  • DOOM: the attract-mode demos of shareware DOOM and Freedoom (video, sound effects and OPL music), Freedoom on the raw keyboard and mouse, and a scripted Freedoom game that saves to gasm:storage, plays on and loads the save.
  • ScummVM: Beneath a Steel Sky (intro skipped, a click to walk, a save through ScummVM's menu with a typed name, then a restore), the SCUMM engine on the Day of the Tentacle demo, and Drascula's opening with its CD music as Ogg Vorbis, MP3 and FLAC.

It fetches ROMs and WADs if missing and AOT-compiles the games first. Each case must also match its line in tests/golden/determinism.txt, the same file on every platform. After a change that is meant to alter output, record new hashes with UPDATE_GOLDEN=1 scripts/determinism-test.sh and commit the file with the change.

sh
check <name> <game> <frames> [runner args...]      # in scripts/determinism-test.sh
check my_game nes 1800 --rom roms/my_game.nes --input "60-65:START,120-900:RIGHT"

A mismatch means one of:

  1. the game reads nondeterministic state (clock, randomness, HashMap order),
  2. a runner bug (wrong copy stride, hashing padding, mis-ordered input), or
  3. an engine bug (rare; reduce to a minimal module and report upstream).

Network test ​

It starts gasm-relay on port 9123 and, for each runner pair, launches two headless sumo peers in a fresh room with different held inputs, running as fast as lockstep allows. Each peer logs frame N state=… score=a:b at quit_at and exits. The test requires identical lines and no DESYNC. Each peer has a hard time limit (LIMIT, default 60 s), so a stall fails the test instead of hanging it.

sh
QUIT=3600 LIMIT=40 PORT=9200 scripts/net-test.sh

Browser cross-play is checked by hand with web-smoke.mjs against a native peer:

sh
gasm-relay 127.0.0.1:9125 &
gasm-run build/sumo.wasm --headless 100000000 --allow-net \
  --param relay=ws://127.0.0.1:9125 --param room=x --param quit_at=1200 --input "0-100000000:RIGHT+A" &
node scripts/web-smoke.mjs "http://localhost:8765/runners/web/?game=sumo.wasm&autostart&relay=ws://127.0.0.1:9125&room=x&quit_at=1200" out.png 24
# both print: [sumo] frame 1200 state=<same hash>

Parity test ​

guests/parity runs the NES game as a normal native binary against the gasm crate's stub host and prints the same hash lines as the runners. If wasm and native builds disagree, suspect undefined behaviour, pointer-size assumptions (usize is 4 bytes in wasm32), or transcendental float functions (native libm vs the libm compiled into the module).

Visual checks ​

Headless runs can take --screenshot out.png. gasm-run renders GPU games' last frame on a real GPU offscreen. The Node runner has no GPU: it only captures video_present frames, and warns for gasm:gfx games. Useful ROMs from make roms:

ROMExpected screen
cpu_instr_test.nes (≈3000 frames)All 16 tests passed
cpu_timing_test.nes (≈1200 frames)PASSED
apu_test.nes (≈1500 frames)All 8 tests passed
bladebuster.nesTitle screen; with the input script from the test, gameplay

Benchmarks ​

sh
R=runners/native/target/release/gasm-run
$R build/nes.wasm --compile build/nes.cwasm
(cd guests && cargo build --release -p parity)
time guests/target/release/parity roms/bladebuster.nes 6000 --no-hash
time $R build/nes.cwasm --allow-precompiled --rom roms/bladebuster.nes --headless 6000 --no-hash
time node runners/web/headless.mjs build/nes.wasm --rom roms/bladebuster.nes --headless 6000 --no-hash

Always use --no-hash for speed measurements: hashing 245 KB per frame costs a lot of time.

Repository conventions ​

  • ABI changes start in spec/abi.json (see ABI below) and update spec/ABI.md, the gasm crate, both runners and the site docs in the same change. Breaking changes bump GASM_ABI_VERSION.
  • Hash format (frames=… / video_fnv32=… audio_fnv32=…) is an interface between runners and scripts; keep it stable.
  • Game logic that must be deterministic lives apart from rendering (sumo/src/sim.rs).
  • Dependencies: wasi-sdk version in scripts/fetch-wasi-sdk.sh, runner crates in runners/native/Cargo.toml, game crates in guests/*/Cargo.toml.
  • Generated or downloaded content (build/, tools/, roms/, target/, site/node_modules, site/.vitepress/dist, site/public/play) is git-ignored. Never commit ROMs.
  • Releases: push a version tag without a v prefix (git tag 0.2.0 && git push origin 0.2.0). See Releases and trusted publishing. Note what changed in CHANGELOG.md.
  • ABI: edit spec/abi.json, run node scripts/gen-abi.mjs (regenerates spec/gasm.h and guests/gasm/src/sys.rs), then implement it in both runners and the native stub. CI runs --check.
  • Agents: AGENTS.md lists build traps, invariants and conventions for coding agents.
  • Docs live in site/ (VitePress) and are published to gasm.emdzej.pl by GitHub Actions. spec/ABI.md is included into the site, not copied.

Releases and trusted publishing ​

release.yml runs on a version tag (0.2.0). It runs CI first, builds everything, and checks that doom.wasm and scummvm.wasm rebuild byte for byte from the source archives it ships (GPL: the source must produce the binary). The games carry their license notices (THIRD-PARTY.txt, from scripts/third-party-notices.sh). Then it creates the GitHub Release and publishes gasm-sdk + gasm-host to crates.io, @emdzej/gasm-host to npm, and ghcr.io/emdzej/gasm-relay. Versions come from the tag, and versions that are already published are skipped, so re-running a release is safe.

crates.io and npm use trusted publishing: the workflow gets a short-lived token from the registry via GitHub OIDC (id-token: write, environment release), and nothing secret is stored in the repository. The relay image is pushed with the workflow's own GITHUB_TOKEN.

One-time setup, because both registries attach trusted publishers to a package, which must exist first:

  1. First publish by hand (once per package):
    sh
    cargo login                                   # crates.io API token
    (cd guests && cargo publish -p gasm-sdk)
    (cd runners/native && cargo publish)
    npm login                                     # pnpm publishes with npm's credentials
    (cd runners/web && pnpm publish --access public --no-git-checks --otp <code>)
  2. crates.io, for gasm-sdk and for gasm-host: Settings → Trusted Publishing → Add → GitHub: owner emdzej, repository gasm, workflow release.yml, environment release.
  3. npm, for @emdzej/gasm-host: npm trust github @emdzej/gasm-host --repo emdzej/gasm --file release.yml --env release --allow-publish (or Settings → Trusted Publisher on npmjs.com). Then, under Publishing access, disallow tokens. In CI, pnpm packs and npm publish uploads the tarball, because npm performs the OIDC exchange and adds provenance.
  4. GitHub (optional): Settings → Environments → release, and add required reviewers or restrict it to tags.

Updating dependencies ​

  • wasi-sdk: WASI_SDK_VERSION=NN WASI_SDK_SHA256=<archive checksum> scripts/fetch-wasi-sdk.sh, then make clean test. To make it the default, update PINNED and the per-platform checksums in the script.
  • tetanes-core: bump in guests/nes/Cargo.toml. Hashes may change if the core changed behaviour: check that the runners still agree, then record the new golden hashes (UPDATE_GOLDEN=1). Re-check the test ROM screens.
  • wasmtime, wgpu, winit: bump in runners/native/Cargo.toml. The runners implement WASI themselves (runners/native/src/wasi.rs, runners/web/lib/wasi.js), so there's no WASI crate to keep in step. Old .cwasm files become invalid; the test script regenerates them.

MIT licensed. The NES demo core is tetanes-core (MIT/Apache-2.0).