# E2E scenarios: the contract With coding agents writing most of the production code, e2e tests are the document humans actually read and review. A scenario must state an intent, a compose model, and a sequence of steps `(command → expected observables)` — and nothing else. Everything operational (project naming, cleanup, failure diagnostics) belongs to the framework, not to the test. The DSL lives in [`scenario.go`](scenario.go) (execution, actions, requirements) and [`checks.go`](checks.go) (the vocabulary of observables). ## Writing a scenario The project files live in `testdata//` — standalone compose files, directly runnable with `docker compose -f testdata/TestRestart/compose.yaml`: ```yaml # testdata/TestRestart/compose.yaml services: app: image: alpine init: true command: ash -c "if [[ -f /tmp/restart.lock ]] ; then sleep infinity; else touch /tmp/restart.lock; fi" ``` ```go func TestRestart(t *testing.T) { NewScenario(t, "restart must bring an exited service back up, restarting the same container"). Step("up starts the service, whose first run exits at once", ComposeCmd("up", "-d"), Eventually(ServiceState("app", "exited"), 10*time.Second)). Step("restart brings the service back up, reusing the container", ComposeCmd("restart"), Eventually(ServiceState("app", "running"), 10*time.Second), NotRecreated("app")) } ``` Rules: - **New e2e tests use `NewScenario`.** The legacy `NewCLI` style remains for existing tests, converted opportunistically; don't add to it. - **One intent = one invariant.** The intent is a one-line statement of the behavior being locked, phrased as an obligation ("X must Y"). If you need two intents, write two scenarios. - **Step names are behavior sentences**, not command echoes: "an unchanged create is a no-op", not "run create again". The transcript of step names should read as the specification. - **The project files live in `testdata//`.** The scenario resolves that directory by convention (subtests map to nested directories, following `t.Name()`) and copies it to a temporary directory, so the committed files are never mutated. The directory holds a `compose.yaml` plus whatever the project needs (Dockerfile, env or config files) in their native format, directly runnable outside the test. Ownership is strictly one test per directory — no shared fixtures — and a check fails the suite on any `testdata` directory no test owns. Interpolate runtime values via `Env`. - **Regression tests link the issue** in a comment above the test, with a sentence on the failure mode being locked. ## Checks: observe real state Checks are the shared vocabulary between scenarios; their discipline is what keeps the contract meaningful. - **Prefer state-based checks** (`ServiceState`, `NotRecreated`, `LabelSet`, `RunsOnPlatform`, …): they observe containers, labels and image manifests — what the user actually gets — not what the CLI printed. - **`OutputContains` is a last resort**, legitimate only when the CLI's reported decision is itself the observable (e.g. "Skipped" vs "Pulled"). - **Never poll by hand**: wrap a state check in `Eventually(check, timeout)`. No `time.Sleep` in scenarios. - **A new check must be generic** — no test-specific logic — **and named after the observable it asserts**, not after the test that needed it. It goes in `checks.go`, where the whole vocabulary is reviewed as one file. Before adding one, verify the observable isn't already expressible. - A check should also fail loudly on a broken precondition (e.g. `NotRecreated` errors if the service had no container before the step) rather than pass vacuously. ## When a scenario fails The report opens with everything needed to diagnose without re-running: - **`artifacts: `** — a stable per-project directory holding the untruncated material: `compose.yaml`, `failure.txt`, each step's full command and output (`step-NN-*.txt`), `containers.txt`, `events.txt`, full container logs (`logs-*.txt`) and the per-step state snapshots (`snapshots.json`). Read these before re-running anything. - **`E2E_KEEP_FAILED=1`** — rerun with this set to skip teardown of failed scenarios: containers, volumes and networks stay alive for `docker inspect`/`exec`. Clean up afterwards with `docker compose --project-name down -v --remove-orphans`. - The inline report shows the transcript (every step, exit code, duration), the failing step's output, project containers, engine events since the scenario started, and container log tails — truncated for readability; the artifacts have the full versions. Run a single scenario with: ```sh go test -tags e2e ./pkg/e2e/ -run TestRestart -v ```