Commit graph

207 commits

Author SHA1 Message Date
Ricardo Branco
4121ec7d13 test(e2e): bump compose-bridge images to v0.0.7
v0.0.6 dropped the Egress policyType/block from generated
NetworkPolicy manifests since it blocked CoreDNS traffic; update
the fixtures to match.

Signed-off-by: Ricardo Branco <rbranco@suse.de>
2026-09-02 18:21:19 +02:00
Guillaume Lours
8bd79da4bc e2e: fix test resource isolation and make standalone parallelism configurable
Several e2e tests used fixed host ports (8070, 8080, 8090, etc.), global
buildx builders registered with --use, and a hardcoded IPC container name.
Under parallel execution these shared daemon-level resources caused port
conflicts, buildkitd container-name collisions, and intermittent failures.
Separately, the standalone make target hardcoded -parallel=1, forcing the
entire standalone suite to run serially even though the same tests run in
parallel in plugin mode.

Changes:
- Convert fixed host ports in four fixtures (network-test, sentences,
  build-test, volume-test) to ephemeral bindings; add ServicePublishedPort
  helper to framework.go to resolve the actual mapped port at runtime.
- Give each buildx-builder test a unique daemon-scoped name via BuilderName
  helper, preventing container-name collisions between parallel goroutines.
  Applies to TestBuildPlatformsWithCorrectBuildxConfig, TestBuildPrivileged,
  TestBuildBuilder, TestBuildEntitlements, TestBuildTLS.
- Rename the fixed ipc_mode_container to a project-scoped name
  (ipc_e2e-src) and update the ipc-test fixture accordingly.
- Replace the hardcoded -parallel=1 in make e2e-compose-standalone with
  E2E_PARALLEL_PLUGIN?=4 and E2E_STANDALONE_PARALLEL?=4, aligning both
  modes with the effective GOMAXPROCS on 4-core GitHub Actions runners.
  Both variables are overridable (E2E_STANDALONE_PARALLEL=1 restores
  serial local debugging).
- Raise the poll delay in watch_test.go long-running polls (90s/120s
  timeout) from the default 100ms to 1s; bump cascade_test.go and
  publish_test.go from 100ms to 500ms. Reduces CPU pressure without
  affecting effective test duration.

The logging-driver fixture retains its fixed port 24224: the Docker fluentd
log driver encodes fluentd-address at service-start time, making ephemeral
resolution non-trivial. TestLoggingDriver is serial so there is no
collision risk in practice.

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-08-19 17:41:02 +02:00
Endika Iglesias
f519180b95 fix(watch): sync into a symlinked directory instead of failing
Signed-off-by: Endika Iglesias <endika2@gmail.com>
2026-08-18 17:25:13 +02:00
Nicolas De Loof
1fa9ae160a test(e2e): image-identity, images and model scenarios
- TestUpIdempotentContainerdStore joins the scenario style of its
  corner-case siblings: Serial + ContainerdImageStore requirement, the
  regression story in a comment, NotRecreated as the invariant. The
  requireContainerdStore helper and the last identity fixture go away.
- TestImagesAfterImageRemoved scopes its throwaway tag to the project
  and observes container state before pulling the image record away.
- TestComposeModel inlines its model; verified in CI where the
  docker-model plugin is installed (the local runner cannot pull the
  model, as with the legacy test).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
447157274e test(e2e): build scenarios — the 'Built' family inlines its models
TestBuildDependsOn, Subset, DependentImage, SubDependencies,
LongOutputLine, DependentImageWithProfile, Escaped, Tags and Secrets
become scenarios: models inline (the long-line Dockerfile generated
with strings.Repeat instead of 34 copy-pasted lines), image names
project-scoped with Defer cleanup, and the secrets build asserts
ImageExists on top of the Dockerfile's own diff checks.

Kept legacy: TestLocalComposeBuild (DOCKER_BUILDKIT=0/1 matrix over a
shared fixture with fixed names and host port), the buildx-builder
family (SSH, platforms, privileged, entitlements, builder, TLS) which
provisions builders and dind daemons, and TestBuildImageDependencies
(classic/bake matrix). Their fixtures under build-test/ stay; the
single-purpose subdirectories are removed.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
a7293130c6 test(e2e): compose command scenarios — rm, compatibility, config, orphan scoping
- TestRm, TestRemoveOrphaned and TestUnnecessaryResources observe which
  containers exist instead of grepping docker ps.
- TestCompatibility drops the heavyweight sentences fixture: underscore
  naming shows on any model.
- TestConfig/TestConfigInterpolate build their expected rendering from
  the scenario's own project name and directory; OutputMatches joins
  the vocabulary for TestInitContainer's ordering assertion.
- TestStopWithDependenciesAttached inlines its model.
- Kept legacy: TestLocalComposeUp (fixed port, HTTP, shared sentences
  fixture), TestDownComposefileInParentFolder (project name inferred
  from a fixture-relative directory), TestAttachRestart (long-running
  attached up), the COMPOSE_FILE-from-.env family and TestNestedDotEnv
  (they exercise invocation without -f, which ComposeCmd always sets).
- Fixtures init-container/, external/ and dependencies/ are removed.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
37f57744de test(e2e): exec and provider scenarios
- TestExec locks the container-selection order (service container over
  one-off) with one-off state observed at each step.
- providers: a shared providerScenario helper wires the
  example-provider binary into PATH; the env assertions anchor on the
  'test-1  | ' log prefix, replacing the hand-rolled env parser, and
  the stop-hook sentinel file is observed with FileExists.
- Fixtures exec/ and providers/ are removed.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
f4f6ac5324 test(e2e): volume and build-once scenarios
- volumes: external-volume switching, volume recreation on definition
  change and image volumes become scenarios; the two-compose-file
  fixtures collapse into one model with an interpolated volume name or
  label, so the changed definition is visible in the test. The rebuild
  scenario locks the Recreated expectation and scopes the source image
  to the project. TestLocalComposeVolume (fixed host port, HTTP) and
  TestProjectVolumeBind (host file written mid-test) stay legacy.
- build-once: the three run --build regression cases each get a
  scenario; OutputMatchesCount joins the vocabulary to count the build
  reports, replacing the hand-rolled regex counter, and image names are
  project-scoped instead of random suffixes.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
ee7f323fed test(e2e): run and up scenarios
- compose run splits into seven scenarios (one-off basics, ports, deps,
  optional deps, quiet pull, --pull always, chained build contexts).
  The orphan-warning cases fold into the basic scenario: an exited
  one-off IS the orphan the next run warns about, no second compose
  file needed. Piped-stdin and stop-signal tests stay legacy.
- up/compose_up: unhealthy service, dependency exit (both now also
  locking that dependents stay in created state), build dependencies
  (project-scoped image name instead of the fixture's global one),
  optional dependency, --all-resources, profile targeting (dropping the
  fixture's global container_names), image-by-ID, exit-code-from (42
  and 143), port ranges, stdout/stderr separation (StderrContains joins
  the vocabulary) and logging-driver reconfiguration (now locking the
  Recreated expectation). The Ctrl-C dependency test stays legacy.
- Fixtures start-fail/, resources/, stop/, port-range/, stdout-stderr/,
  logging-driver/, profiles/ and most of dependencies/ and run-test/
  are removed.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
9409c461f7 test(e2e): publish-check scenarios — WithStdin answers the prompts
The dry-run publish checks split into nine scenarios, one per model
and refusal reason; the prompt text, refusal exit code (130) and
publication report are the observables. WithStdin joins the Action
modifiers to feed prompt answers. TestPublish stays legacy: it drives
a real registry container and loads oci:// projects with multiple -f
files. Only the oci/ fixture subtree it needs survives.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
d38fa3431a test(e2e): cp, environment and config scenarios
- cp: the twelve copy directions become one scenario; Dir() joins the
  Scenario API for actions that exchange files with the host, and
  FileContains observes what landed there. The files copied in and out
  live in the txtar archive instead of writing into the fixture tree.
- environment: TestEnvPriority splits into three scenarios, one per
  compose model (environment section, bare, env_file section), each
  declaring its .env variants inline; StdoutContains joins the
  vocabulary so container output cannot collide with progress noise
  (the unused framework predicate of the same name is removed). The
  fixed 'env-compose-priority' image name becomes project-scoped.
- config: rendering assertions move to scenario steps; the config
  output is the observable here by definition.
  TestConfigHashMatchesContainerLabel stays legacy: it cross-references
  hashes captured at runtime, which checks cannot hold.
- Fixtures cp-test/, environment/ and config/ are removed.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
816709d877 test(e2e): exec and lifecycle-hook scenarios
- exec: exit-code propagation, -e env forwarding and one-off targeting
  become two scenarios with an inline model.
- hooks: the thirteen post_start/pre_stop/pre_start tests migrate. The
  volume probe is a named action (probeVolume); wc -l counts are locked
  by matching the exact '1 /mnt/tokens.txt' line instead of parsing.
  The spec-change scenario replaces its two fixture files with one
  model interpolating ${HOOK_VERSION}, so the changed spec — and the
  Recreated expectation it implies — is visible in the test source.
  pre_stop scenarios drop the fixture's fixed 'sample-data' volume
  name, which parallel projects used to share.
  Fixtures hooks/ and pre_start/ are removed.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
54caa0775c test(e2e): pull and network scenarios
- pull: the CLI's Pulled/Skipped report is the observable, so the five
  scenarios (always-pull, present-locally, build-only, failure,
  --ignore-pull-failures) legitimately match output; models are inline
  and the whole compose-pull fixture tree — including the orphaned
  duplicate-images directory no test referenced — is removed.
- networks: aliases, links, ipam, network_mode: service, subnet change,
  mac_address and interface_name become scenarios. The subnet-change
  scenario locks what actually happens: the network is replaced but the
  container is reconnected, not recreated (NotRecreated), which the
  legacy test never distinguished. TestNetworks (fixed host ports,
  HTTP fixture services) and TestNetworkRecreate (OR-shaped output
  assertion) stay legacy.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
cd1c0ca6ca test(e2e): configs, commit, export and logs scenarios
ImageExists and FileExists join the vocabulary, giving commit and
export a real observable: the legacy tests only asserted the commands
exited 0, the scenarios verify the committed image and the exported
archive actually exist. Both also drop their fixed image tags and
cwd-relative tar outputs for project-scoped tags and temp files, so
parallel runs cannot collide and nothing is left behind in the repo.

TestLocalComposeLogs waits for the echo service to exit before reading
logs (Eventually), closing the race the legacy test tolerated. The
follow and large-logs tests stay legacy: both drive a long-running
command in the background. ps_test.go stays legacy as well — it
verifies ps table/JSON formatting, which is inherently output-shaped.

Fixtures configs/, commit/ and export/ are removed.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
3923931b8c test(e2e): scale, secrets and env_file scenarios
ServiceScale and ReplicaNumbers join the vocabulary: replica counts and
numbering read from container labels replace counting 'Started' lines
in the CLI output.

- TestScaleDownNoRecreate's final assertion now actually runs: the
  legacy test built the scale-down command without executing it, so
  'scale down removes obsolete replicas' was never exercised; the
  scenario locks the real behavior (stale replicas dropped, up-to-date
  ones kept, whatever their numbers).
- secrets scenarios inline the include-based project (compose.yaml,
  child project and env_file as one txtar archive).
- TestRawEnvFile and TestRunEnvFile inline their env files;
  TestUnusedMissingEnvFile stays legacy for its completion assertions,
  which drive the plugin/standalone binaries directly.
- Fixtures scale/, env-secret/ and dotenv/raw are removed.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
fe17e31be0 test(e2e): lifecycle scenarios — cascade, profiles, healthcheck, recreate
Vocabulary grows with Recreated (the mirror of NotRecreated),
ServiceHealthy (healthcheck status observed from inspect) and a Within
action modifier bounding blocking commands like up --wait, replacing
the hand-rolled goroutine+timer of the legacy TestStartInterval.

- cascade: abort-on-container-exit/failure scenarios inline the model;
  exit-code propagation (111) is locked with ExitCode. The one-off
  variant stays legacy — it drives a long-running up in the background.
- profiles: the four scenarios observe which services materialize
  instead of grepping ps output; the dotenv variant declares its .env
  inline, making the activation mechanism visible.
- TestRecreateWithNoDeps now asserts both sides: the service is
  replaced (Recreated) and the healthy dependency is left alone.
- Fixtures start_interval/, profiles/compose.yaml+test-profile.env and
  recreate-no-deps.yaml are removed; cascade/ stays for the legacy
  one-off test.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
6bc1b36516 test(e2e): expose, container_name, ipc and wait scenarios
- TestExposeRange keeps its regression link (#13378), the invariant
  being simply that up accepts a port range in expose.
- TestUpContainerNameConflict interpolates a project-scoped
  container_name instead of the fixture's global 'test', so parallel
  runs cannot collide on it.
- TestIPC demonstrates the CLI() escape hatch: the external container
  the 'container:' mode points at is created eagerly before the steps,
  so its id is available to the declarative checks. (Runs only in CI:
  ipc container: mode is rejected by Docker Desktop and dind daemons —
  'restricted host mount' — for the legacy test as well.)
- wait scenarios observe which services exited instead of trusting the
  command's silence; TestWaitOnInfinity stays legacy, the DSL has no
  notion of a still-running command. The wait fixture stays for it;
  ipc-test and container_name fixtures are removed.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
a33cc37ad9 test(e2e): TestRemoveOrphans states its real intent — COMPOSE_REMOVE_ORPHANS from .env
The legacy test read as 'up removes leftover one-offs', but plain up
only warns about them: the removal it asserted came from the fixture's
.env declaring COMPOSE_REMOVE_ORPHANS=true (added by 'restore
setEnvWithDotEnv', d3d378b92) — an invariant the fixture kept invisible.
The scenario declares both files inline as a txtar archive, so the
mechanism under test is in plain sight, and observes the one-off's
removal from container state rather than ps output.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
695a2a759a test(e2e): start/stop scenarios observe container state, not CLI chatter
Migrate the state-friendly start/stop tests to the Scenario DSL:

- TestStartStop: stop halts containers in place, start brings the same
  ones back (NotRecreated); the ls checks match the project name exactly
  in the JSON output, immune to the prefix collisions the other
  start-stop project names would cause under parallel runs.
- TestStartStopWithDependencies: dependency start/stop propagation read
  from container state instead of 'Container X Started' messages.
- TestUpNoDeps replaces the 'Up no-deps links' subtest as its own
  scenario: up --no-deps must not create the linked dependency.
- TestStartSingleServiceAndDependency: create/start walk the dependency
  chain and nothing else — now asserting the unrelated services have no
  container at all, stronger than the legacy message matching.

The one-off and multi-file tests stay legacy: the vocabulary has no
one-off awareness yet and a scenario holds a single compose file.
The links and start-stop-deps fixtures become unused and are removed.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
5baeadf929 test(e2e): no-deps scenarios — the donor container's absence is the observable
Migrate the volumes_from and network_mode:service donor tests to the
Scenario DSL, inlining the compose model and dropping the no-deps
fixtures. Each scenario removes the donor container behind compose's
back and expects up --no-deps to be rejected with the 'container
missing' error — the CLI's reported decision being the observable.

ServiceNotCreated joins the check vocabulary: it asserts a service has
no container at all, here proving the donor really is gone before the
failing step.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
12288a7fff test(e2e): pause scenarios — paused is a container state, ExitCode locks error paths
Migrate the four pause tests to the Scenario DSL: the invariant is the
container state (paused/running), not an HTTP timeout, so the scenarios
observe state and drop the port-mapping helpers and the pause fixture.

ExitCode joins the check vocabulary for error-path steps, paired with
MayFail. TestPauseServiceDoesNotExist now locks the current behavior —
pause of an unknown service is rejected with 'no such service' — which
resolves the TODO the legacy test carried: it asserted exit 0 while
wishing for an error, and the CLI has since started validating selected
services.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
0632f222f6 test(e2e): declarative Scenario layer, image-identity tests as demonstrator
Introduce a small declarative layer over the e2e framework so a test
reads as its intent: an inline compose.yaml, a sequence of steps
(command + expected observables), and nothing else. The framework owns
project naming (derived from the test name), cleanup, and failure
reporting: on the first unmet expectation the test fails with a step
transcript, the failing command output, the project's container state,
the engine events since the scenario started and the container logs.

Checks observe real state (container IDs across steps, labels, image
manifest platform) instead of grepping the progress writer output,
which stays available as an explicit last-resort check.

The e2e tests introduced by #14041 are converted as a demonstrator;
their dedicated fixture files are inlined in the scenarios and removed.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-18 16:49:56 +02:00
Nicolas De Loof
df094b036c fix(images): tolerate containers whose image record is gone
A running container may reference an image record that no longer
exists: under the containerd image store, `up --build` with identical
content moves the tag to a new index digest (provenance attestation
churn) and the daemon drops the old index the container was created
from — without compose recreating the container, by design (#13636).
`docker rmi -f` of a running container's image produces the same state.

`compose images` used to fail the whole listing on the resulting
NotFound. Degrade to what the container itself knows (image ID, and
repository/tag when the reference is not a raw ID) instead of failing.

Fixes #14014

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-14 15:20:37 +02:00
Nicolas De Loof
7ff6748564 test: local multi-platform image must satisfy the missing pull policy
Regression test for #14007: with the containerd image store, a local
multi-platform image holding the requested non-native variant must be
used as-is by the default missing pull policy — compose used to inspect
the image without a platform, compare the host variant's platform fields
to the requested one, conclude the image is missing and try to pull the
(unpublished) tag.

The test fails on main and passes with #14011, whose platform check
resolves the requested platform against the locally available manifests.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-13 12:03:29 +02:00
Nicolas De Loof
a2e7fe6590 test: cover image-identity corner cases end to end
- dry-run up with a missing image must not hit the real daemon
- create twice with a non-native DOCKER_DEFAULT_PLATFORM is idempotent
- two services sharing an image with mixed platform pinning get their
  own platform's digest label, idempotently, whatever pull ordering
- explicit pull refreshes ahead of a daily refresh window

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-13 12:03:29 +02:00
Guillaume Lours
d98c41ba07 ci(e2e): run e2e against the containerd image store
The e2e suite only ran on graphdriver daemons, where the different kinds
of image digests coincide — the blind spot that let #13636, #13998 and
#14005 through. Add one matrix entry enabling the containerd image store,
plus TestUpIdempotentContainerdStore: two consecutive `up` runs with no
change must not recreate any container. The test is red on this
configuration (the com.docker.compose.image label is written from the
index digest on the pulling run, then compared against the per-platform
manifest digest on the next run) and skipped until the next commit
resolves the pull-path digest.

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-08-13 12:03:29 +02:00
Guillaume Lours
7bc2630b65 fix(config): resolve service environment when computing --hash
Containers are created from a project whose service environment is
resolved (env_file merged into environment), but since v2.22.0 `config
--hash` skipped that resolution, so hashes diverged from the
com.docker.compose.config-hash label for services using env_file.

Resolve the environment of the hashed services only, so a broken
env_file or platforms on an unrelated service still doesn't prevent
hashing, and honor --no-env-resolution as an escape hatch.

Fixes #14001

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-08-11 08:37:13 +02:00
Ricardo Branco
668d81d4ec test: Set stop_signal to SIGTERM in nginx-based services
The official nginx images set STOPSIGNAL to SIGQUIT which dumps core.
Set it to SIGTERM to avoid dumping core on e2e tests when containers
running "sleep infinity" are stopped.

Signed-off-by: Ricardo Branco <rbranco@suse.de>
2026-08-10 12:42:02 +02:00
Eric Wyles
c25474844c fix(bridge): skip pulling default image references for build-only services
Signed-off-by: Eric Wyles <23637493+ericwyles@users.noreply.github.com>
2026-08-06 16:00:08 +02:00
Guillaume Lours
7bdc31c5de fix(config): apply config flags to --services/--volumes/--networks/--models/--hash
The runServices, runVolumes, runNetworks, runModels and runHash paths
called ProjectOptions.ToProject directly, bypassing the configOptions
wrapper that applies --no-consistency, --no-interpolate, --no-normalize,
--no-path-resolution, --profile filtering and env_file discarding.
Restore the variadic wrapper (mirroring configOptions.ToModel) and route
the five call sites through it.

Regression introduced by b80bb0586 (LoadProject API migration).

Fixes #13974

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-07-30 11:38:20 +02:00
Domantas Petrauskas
5534be0a28 fix(oci): honor --insecure-registry when up re-loads the model
`docker compose -f oci://<insecure-registry>/... up` failed against a
plain-HTTP registry unless --yes was passed:

    failed to pull OCI resource "localhost:5000/test:interpolated":
    Head "https://localhost:5000/v2/test/manifests/interpolated":
    http: server gave HTTP response to HTTPS client

`up` loads the project twice. The first load goes through ToProject,
which built its OCI options from --insecure-registry correctly. Without
--yes, checksForRemoteStack then calls promptForInterpolatedVariables,
which re-loads the project through ToModel to list the interpolation
variables. That second load builds its own resource loaders via
remoteLoaders, and those passed an empty api.OCIOptions{}, dropping the
flag. Since the OCI loader always performs a network resolve, the
re-load spoke HTTPS to a plain-HTTP registry and failed before the
prompt could be shown.

The two construction sites had drifted apart, so rather than patching
the second one, both now share ProjectOptions.ociOptions(). `config`
and `viz` use the same ToModel path and are fixed as well.

Covered by an e2e case in TestPublish, which already runs an insecure
registry and an oci:// round-trip: it publishes a fixture carrying an
interpolation variable so the prompt fires, then runs `up` without
--yes and declines, asserting the re-load does not fail with
"server gave HTTP response to HTTPS client".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Domantas Petrauskas <dom.petrauskas@gmail.com>
2026-07-21 16:24:34 +02:00
Guillaume Lours
9d444a667a pre_start: support pre_start lifecycle hooks
Some checks failed
ci / validate (lint) (push) Has been cancelled
ci / validate (validate-docs) (push) Has been cancelled
ci / validate (validate-go-mod) (push) Has been cancelled
ci / validate (validate-headers) (push) Has been cancelled
ci / binary (push) Has been cancelled
ci / bin-image-test (push) Has been cancelled
ci / test (push) Has been cancelled
ci / e2e (plugin, oldstable) (push) Has been cancelled
ci / e2e (standalone, oldstable) (push) Has been cancelled
ci / e2e (plugin, stable) (push) Has been cancelled
ci / e2e (standalone, stable) (push) Has been cancelled
merge / bin-image-prepare (push) Has been cancelled
merge / module-image (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
ci / binary-finalize (push) Has been cancelled
ci / coverage (push) Has been cancelled
ci / release (push) Has been cancelled
merge / bin-image (push) Has been cancelled
merge / desktop-edge-test (push) Has been cancelled
runPreStart executes a service's pre_start hooks sequentially as
ephemeral containers that share the first non-running replica's
volumes via VolumesFrom and attach to the same networks. A non-zero
hook exit gates service start.

per_replica: false is the only currently supported mode; per_replica:
true is rejected up front. The donor replica is the lowest-numbered
one so the choice is deterministic. ContainerWait uses
WaitConditionNextExit, and the wait loop deterministically handles
the daemon's clean-close (nil on Error + exit code on Result) and
transport-error races to avoid spurious hook failures.

The log stream is opened before ContainerStart to avoid racing
AutoRemove on fast-exiting hooks, and runs under a derived context
so a daemon that keeps the connection open cannot deadlock the call.
Hook containers carry project/service/version labels; the two
cleanup paths force-remove the never-started container explicitly
and warn when removal fails.

pre_start runs once per service when no replica is already running
(initial up, force-recreate or spec change), and is skipped on
scale-up so additional replicas don't re-trigger the hooks.

Coverage: 11 unit tests (including scheduler-race stress) with
goroutine-leak verification via goleak, plus 10 E2E tests (success
path, hook failure gating start, build-image inheritance, idempotent
re-up, spec change, force-recreate, mid-sequence failure, ordering,
scale-up, scaled service).

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-06-30 10:49:04 +02:00
Yohta Kimura
43922d55b0 Fix rawsetenv nil-pointer guard for inherit-from-shell env vars
When a service declares an env var without a value (e.g. `- KEY` or
`KEY:`), MappingWithEquals stores it as a nil *string. The previous
condition `existing != nil && ...` skipped the warning for this case,
allowing silent overwrites. Change to `existing == nil || ...` so the
warning fires for both nil (shell-inherit) and value-mismatch cases.

Add e2e tests for both list-style (`- KEY`) and map-style (`KEY:`)
YAML forms to lock in the behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Yohta Kimura <38206553+rajyan@users.noreply.github.com>
2026-06-23 15:01:33 +02:00
Yohta Kimura
5c9d611b5f Handle rawsetenv collisions with overwrite and warning
rawsetenv injects provider variables without the service-name prefix, so
a key can collide with a value already set on the dependent service,
whether declared by the user in environment or emitted by another
provider. Log a warning and overwrite on collision, document the
precedence and the non-deterministic ordering between concurrent
providers, and cover the user-environment override with an e2e test.

Signed-off-by: Yohta Kimura <38206553+rajyan@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 15:01:33 +02:00
Yohta Kimura
332e0add14 Add rawsetenv message type for provider plugins
Providers can now send rawsetenv messages to inject environment
variables into dependent services without the automatic service name
prefix. This enables use cases where applications require exact
variable names that cannot be altered.

Closes #13727

Signed-off-by: Yohta Kimura <38206553+rajyan@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-06-23 15:01:33 +02:00
Sebastiaan van Stijn
f0744b1df2 pkg/e2e: fix malformed JWT in fixtures
This fixture was not a valid JWT; the first 2 elements decode, but the last
one is malformed;

    echo 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9' | base64 -d
    {"alg":"HS256","typ":"JWT"}⏎

    echo 'eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ' | base64 -d
    {"sub":"1234567890","name":"John Doe","iat":1516239022⏎

    echo 'SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw' | base64 -d
    I�J�IHNJ(]�O��lj~�:N�%_�u
                            ,⏎

This causes problems if the JWT parser is strict and rejecting invalid
JWT's.

It was added in 55b5f233c2, and probably copied
from an example, like https://github.com/knottx/JWTCodable#example-jwt-token,
but the last 2 bytes were truncated.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-06-19 16:07:02 +02:00
Nicolas De Loof
82a0ad9c44 watch: do not rebuild depends_on services on file change
`compose up --build` populates BuildOptions.Deps=true so the initial
startup also builds images for depends_on services. The watch rebuild
path reused the same BuildOptions pointer, only resetting Build.Services
to the watched service. Build.Deps stayed true, so s.build() switched
back to IncludeDependencies and rebuilt the upstream dependency too.

Fix it by working on a local copy of BuildOptions in rebuild() and
explicitly setting Deps=false. Using a local copy also removes the data
race on the shared pointer when concurrent file events fire.

Also fix a related leak in doBuildBake: the loop populating bake
configuration iterates over every service in the project (needed so
additional_contexts: service:xxx references can resolve), but it was
emitting the "Image X Building" progress event and tracking expected
images for services that were not part of serviceToBeBuild. Filter
those side-effects to the actual build set so the watch rebuild log
shows only the watched service.

Adds an e2e test reproducing the bug.

Fixes #13853

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-06-19 11:23:50 +02:00
Guillaume Lours
eb4b1cc3f6 fix(publish): prompt on sensitive-looking env literals
Replace the "flag any literal env var" check with a key-name heuristic
backed by the upstream DefangLabs keyword detector (password, secret,
token, api_key, …), and convert the hard error into a prompt matching
the existing checkForBindMount / checkForSensitiveData UX. --with-env
silences the env prompt; literal config.content gets its own prompt.

The previous check flagged benign vars like LOG_LEVEL=info, blocking
the 99% case, while still missing low-entropy real secrets the
existing secret-detector skips (MYSQL_ROOT_PASSWORD=toto slips through
on entropy ~1.5).

Refs: docker/compose#13394

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-05-20 15:48:38 +02:00
Guillaume Lours
672dc14d29 feat: add stop lifecycle hook for external providers
Provider-backed services were silently skipped on `docker compose stop`,
leaving external resources running after the user expected the stack to
be paused (e.g. after Ctrl+C during `up --watch`).

Compose now invokes `<provider> compose stop <service>` for providers
that advertise a `stop` block in their `metadata` subcommand output.
Providers that do not advertise stop (or do not implement metadata at
all) are silently skipped, preserving backward compatibility with
existing providers that pre-date this hook.

Closes #13772

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-05-18 11:11:38 +02:00
ibrahim yapar
56ab28aef3 compose: recreate container when mounted image digest changes
Until now, mustRecreate logic only checked for divergence in TypeVolume
mounts but ignored TypeImage mounts. This inconsistency caused containers
to erroneously retain stale images even after the source image was rebuilt.
This commit updates ensureImagesExists to resolve image volume sources to
their digests using the official reference package. This enables ServiceHash
(config hash) to naturally detect underlying image digest changes,
triggering recreation via the standard convergence logic.
An E2E test case is added to verify this behavior.
Fixes #13547

Signed-off-by: ibrahim yapar <74625807+ibrahimypr@users.noreply.github.com>
2026-01-26 15:55:44 +01:00
Suleiman Dibirov
67e39a41f2 fixes
Signed-off-by: Suleiman Dibirov <idsulik@gmail.com>
2025-11-06 08:02:59 +01:00
Suleiman Dibirov
0c854a6ab7 add e2e tests
Signed-off-by: Suleiman Dibirov <idsulik@gmail.com>
2025-10-29 10:47:18 +01:00
Nicolas De Loof
e59150baa8 fix OCI compose override support
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2025-10-23 12:04:37 +02:00
Nicolas De Loof
585c4db4f9 Compose can't create a tar with adequate uid:gid ownership
as we can't get container UID/GID as int by ContainerInspect
revert https://github.com/docker/compose/pull/13288

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2025-10-20 09:51:58 +02:00
Nicolas De Loof
ee75be342b Set secret/config uid:gid to match container's USER
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2025-10-16 17:43:04 +02:00
Nicolas De Loof
a03f2562df bake only interpolates ${*}
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2025-10-13 10:40:19 +02:00
Nicolas De Loof
cf7e31f731 escape $ in bake.json as interpolation already has been managed by compose
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2025-10-03 09:25:35 +02:00
Nicolas De Loof
2ca7b96e33 resolve secrets based on env var before executing bake
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2025-09-22 14:56:15 +02:00
Ricardo Branco
a32dc3da72 test: Set stop_signal to SIGTERM
The official nginx images set STOPSIGNAL to SIGQUIT which dumps core.
Set it to SIGTERM to avoid dumping core on e2e tests.

Signed-off-by: Ricardo Branco <rbranco@suse.de>
2025-09-19 10:31:20 +02:00
Guillaume Lours
d07c437ce8 dectect if piped run command and disable tty if so
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2025-09-19 09:51:15 +02:00