AGENTS.md documents 'Test unit: go test ./pkg/...' and 'E2E tests:
go test -tags e2e ./pkg/e2e/'. Both statements were false: no file in
pkg/e2e carried an e2e build constraint, so the -tags flag was a no-op
and the 'unit' command actually launched the full e2e suite — requiring
a Docker daemon and the locally built binary, creating and destroying
containers, for up to 20 minutes. CI only avoided this by grepping e2e
out of the package list (Dockerfile).
Every *_test.go in pkg/e2e now carries //go:build e2e (merged with the
existing !windows constraints), and the Makefile e2e targets pass the
tag. The documented commands become true:
- go test ./pkg/... -> pkg/e2e reports [no test files]
- go test -tags e2e ./pkg/e2e -> runs the suite
AGENTS.md now also states the daemon/binary prerequisites explicitly.
Part of #14074 (A: the code misdescribes its own structure).
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Several e2e tests used fixed host ports (8070, 8080, 8090, etc.), global
buildx builders registered with --use, and a hardcoded IPC container name.
Under parallel execution these shared daemon-level resources caused port
conflicts, buildkitd container-name collisions, and intermittent failures.
Separately, the standalone make target hardcoded -parallel=1, forcing the
entire standalone suite to run serially even though the same tests run in
parallel in plugin mode.
Changes:
- Convert fixed host ports in four fixtures (network-test, sentences,
build-test, volume-test) to ephemeral bindings; add ServicePublishedPort
helper to framework.go to resolve the actual mapped port at runtime.
- Give each buildx-builder test a unique daemon-scoped name via BuilderName
helper, preventing container-name collisions between parallel goroutines.
Applies to TestBuildPlatformsWithCorrectBuildxConfig, TestBuildPrivileged,
TestBuildBuilder, TestBuildEntitlements, TestBuildTLS.
- Rename the fixed ipc_mode_container to a project-scoped name
(ipc_e2e-src) and update the ipc-test fixture accordingly.
- Replace the hardcoded -parallel=1 in make e2e-compose-standalone with
E2E_PARALLEL_PLUGIN?=4 and E2E_STANDALONE_PARALLEL?=4, aligning both
modes with the effective GOMAXPROCS on 4-core GitHub Actions runners.
Both variables are overridable (E2E_STANDALONE_PARALLEL=1 restores
serial local debugging).
- Raise the poll delay in watch_test.go long-running polls (90s/120s
timeout) from the default 100ms to 1s; bump cascade_test.go and
publish_test.go from 100ms to 500ms. Reduces CPU pressure without
affecting effective test duration.
The logging-driver fixture retains its fixed port 24224: the Docker fluentd
log driver encodes fluentd-address at service-start time, making ephemeral
resolution non-trivial. TestLoggingDriver is serial so there is no
collision risk in practice.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
- TestRm, TestRemoveOrphaned and TestUnnecessaryResources observe which
containers exist instead of grepping docker ps.
- TestCompatibility drops the heavyweight sentences fixture: underscore
naming shows on any model.
- TestConfig/TestConfigInterpolate build their expected rendering from
the scenario's own project name and directory; OutputMatches joins
the vocabulary for TestInitContainer's ordering assertion.
- TestStopWithDependenciesAttached inlines its model.
- Kept legacy: TestLocalComposeUp (fixed port, HTTP, shared sentences
fixture), TestDownComposefileInParentFolder (project name inferred
from a fixture-relative directory), TestAttachRestart (long-running
attached up), the COMPOSE_FILE-from-.env family and TestNestedDotEnv
(they exercise invocation without -f, which ComposeCmd always sets).
- Fixtures init-container/, external/ and dependencies/ are removed.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
- Fix stale image/container reuse across test runs - Add registry readiness check and async removal polling
- Skip multi-arch test when docker driver supports it
- Use t.Cleanup for reliable teardown, fix project name mismatches
- Re-enable 4 previously skipped tests that now pass
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Before this, assertion libraries were mixed, sometimes
even in the same file.
git grep -l '"gotest.tools/v3/' | wc -l
75
git grep -l '"github.com/stretchr/testify' | wc -l
24
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Format the code with gofumpt to prevent my IDE from reformatting
every time I open a file. gofumpt provides a superset of gofmt,
so should not impact users that are not using gofumpt.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Lots of our phony Compose files launch pointless long-lived processes
so we can assert on state. However, this means they often don't respond
well to signals on their own, requiring Compose to timeout and kill
them when doing a `down`.
Add in lots of `init: true` where appropriate so that we don't block
for no reason while running E2E tests all over the place.
Additionally, a couple tests have gotten a cleanup so they don't leave
behind containers. I still want to build this into the framework in
the future, but this is easier for the moment and won't cause any
trouble in the future.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
As of Go 1.16, the same functionality is now provided by package io or
package os, and those implementations should be preferred in new code.
So replacing all usage of ioutil pkg with io & os.
Signed-off-by: Abhinav Nair <11939846+abhinavnair@users.noreply.github.com>
The E2E tests can be run in plugin (`docker compose`) or standalone
(`docker-compose`) mode. Existing logic was in place to ensure that
the helper method is always used, which will invoke the right one
based on how tests are being executed.
However, this logic was too easy to (unintentionally) bypass given
the myriad of ways that commands can be run. The check has been
made stricter and pushed to a lower-level to more aggressively
catch instances.
As a result, a bunch of calls to `RunDockerCmd` are now updated
to be `RunDockerComposeCmd`, which will ensure that the invocation
is correct based on test mode.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
The big mechanical change here is to NOT store `t` as a field on
the `CLI` object (which has been renamed as well to fix the odd
capitalization). The way the tests are structured meant that the
"subtests" were using the _parent_ `*testing.T` instance, which
causes various oddities including confusing messages on failure
about a child test causing a panic in the parent.
Additionally, a few tests have been blocked from running in
parallel because they are sharing `compose.yaml` fixtures and
can fail as a result (e.g. due to a port conflict on the host).
I'll fix these in follow-up PRs but want to focus on correctness
for the tests before optimizing them.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>