Commit graph

1616 commits

Author SHA1 Message Date
Nicolas De Loof
01a60a5e99 fix(up): drop benign context.Canceled noise from the final report
The new TestUpAttachedTerminatesOnExternalStop (#13990) occasionally
observed `up` exit 1 instead of 0 after an external stop/down cancels
a service's restart backoff. The cause: once the monitor detects
termination and cancels globalCtx, a log-streaming goroutine still
blocked on logs?follow=1 for the backoff container gets a
context.Canceled error, which appendErr collected unfiltered into the
final errors.Join.

That race could only surface once #13990 made the monitor loop
actually return in this scenario -- before that fix, up hung forever
and never reached this code path.

appendErr now drops an error that is nothing more than fallout from
our own shutdown (errors.Is(err, context.Canceled) once
u.globalCtx.Err() is set), the same way a SIGINT-triggered teardown's
residual goroutine errors are already expected to be harmless. A
genuine error is still reported regardless of shutdown state.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-09-16 13:54:05 +02:00
Nicolas De Loof
6970069cc6 fix(dry-run): commit no longer creates a real image; the interception set becomes declarative
compose commit --dry-run delegated ContainerCommit to the real client
and actually created the image. It now returns a dry-run ID without
touching the engine.

Auditing the rest of the client surface: 39 more mutating operations
were delegated — none that compose itself issues (verified caller by
caller), but any future code path reaching one would mutate the engine
under --dry-run silently. They now refuse with an explicit error.

The classification — read-only delegated / faked / refused — was only
implied by a method's position under a comment; it is now declared in
dryrunclient_test.go, which parses the client and fails on any method
missing, misclassified, or new and unclassified: extending APIClient
requires an explicit dry-run decision.

The commit e2e scenario gains a --dry-run step asserting the image is
NOT created (new ImageAbsent check).

Epic #14074, B.7.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-09-16 13:44:21 +02:00
Guillaume Lours
22d170f807 test(e2e): add attached up regression test for #13985
Unit tests alone didn't cover the exact failure: an attached up
hanging when the project is stopped/removed externally while a
service sits in restart backoff. The new test reproduces that
sequence end to end, so a future regression on this path breaks
loudly instead of slipping through as a passing unit suite. A
RestartCount guard narrows the race with the backoff expiring, so a
missed window fails loudly instead of passing silently through the
pre-existing die-based path.

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-09-16 07:54:38 +02:00
Guillaume Lours
76af86d47c fix: detect externally stopped and removed containers in up monitor
Since 2.39.3 the monitor relies solely on die events to stop tracking
containers, but a container stopped while in restart backoff emits only
stop and destroy events, so an attached `up` hangs forever after an
external `stop`/`down`. Handle destroy as terminal, and on stop inspect
the container to distinguish a definitive stop from the transient one
emitted during a ContainerRestart, which keeps watch sync+restart
(#13161) working.

A plain stop and its die aren't ordered, so evicting as soon as a stop
looks definitive can end the loop before an already-queued die is
drained, losing the exit code --exit-code-from and
--abort-on-container-exit depend on. Eviction on stop now requires a
die already observed for that container; otherwise it defers to the
pending die.

Fixes #13985

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-09-16 07:54:38 +02:00
hiroto.toyoda
ea9f118dd0 fix(publish): fold OCI fallback warning into the published event
Some checks are pending
ci / validate (lint) (push) Waiting to run
ci / validate (validate-docs) (push) Waiting to run
ci / validate (validate-go-mod) (push) Waiting to run
ci / validate (validate-headers) (push) Waiting to run
ci / validate (validate-mocks) (push) Waiting to run
ci / binary (push) Waiting to run
ci / binary-finalize (push) Blocked by required conditions
ci / bin-image-test (push) Waiting to run
ci / test (push) Waiting to run
ci / e2e (plugin, oldstable, graphdriver) (push) Waiting to run
ci / e2e (standalone, oldstable, graphdriver) (push) Waiting to run
ci / e2e (plugin, stable, containerd) (push) Waiting to run
ci / e2e (plugin, stable, graphdriver) (push) Waiting to run
ci / e2e (standalone, stable, containerd) (push) Waiting to run
ci / e2e (standalone, stable, graphdriver) (push) Waiting to run
ci / coverage (push) Blocked by required conditions
ci / release (push) Blocked by required conditions
merge / bin-image-prepare (push) Waiting to run
merge / bin-image (push) Blocked by required conditions
merge / module-image (push) Waiting to run
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
zizmor / zizmor (push) Waiting to run
Signed-off-by: hiroto.toyoda <hiroto.toyoda@dena.com>
2026-09-15 14:54:03 +02:00
hiroto.toyoda
548be16595 fix(publish): report OCI fallback as a bool, not the OCI version
Signed-off-by: hiroto.toyoda <hiroto.toyoda@dena.com>
2026-09-15 14:54:03 +02:00
hiroto.toyoda
f8e64299b4 fix(publish): warn when push falls back from OCI 1.1 to OCI 1.0
Signed-off-by: hiroto.toyoda <hiroto.toyoda@dena.com>
2026-09-15 14:54:03 +02:00
Guillaume Lours
7e3aa869ec fix(port): preserve tie order for dual-stack port publishers
`docker compose port SERVICE PORT` picks the first matching publisher
as its answer, but containerPublishers()'s sort.Slice does not
guarantee tie order, so a dual-stack publish (same target port on
both an IPv4 and an IPv6 address) could non-deterministically return
either one. Switch to sort.SliceStable to preserve the daemon's
original order for ties.

Also drops a dead length check and a duplicated host:port formatter.

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-09-14 10:48:03 +02:00
Nicolas De Loof
7ec29c82d4 fix(watch): interrupt stale rebuilds so watch converges
A rebuild whose build context was snapshotted before a newer change to
one of its own services is doomed: its output will be replaced by the
trailing rebuild anyway, and meanwhile a sync+restart change landing in
that window was silently lost — the sync went to the doomed container,
the restart was skipped, and the replacement image predated the edit.

The scheduler now gives each run its own cancellable context and
upholds one invariant: every request ends up covered by a rebuild whose
snapshot postdates it. A request naming an actively-rebuilt service
interrupts the run and folds its whole active set back into pending
(interruption kills the other services' rebuild too), so the trailing
rebuild redoes everything on a fresh snapshot. Requests for other
services keep coalescing without interrupting. handleWatchBatch drops a
restart only when the rebuild is pending (that one converges on its
own) and folds it into a Request when the rebuild is in flight —
recreate + start is the restart intent, converged. The run loop also
stops draining once the watch context is cancelled, which silences the
spurious "Build failed: context canceled" on Ctrl-C.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-09-14 10:47:07 +02:00
Guillaume Lours
53188a4896 fix(watch): coalesce rebuilds triggered during a burst of file changes
compose watch triggered one rebuild per debounced batch, so a burst of
edits arriving faster than the debounce window (e.g. from a coding
agent editing several files) queued up several rebuilds in a row
instead of a single consolidated one.

Rebuilds now run asynchronously through a small scheduler that
coalesces requests received while a rebuild is in flight into one
trailing rebuild, without adding latency to an isolated edit. This
also closes a race where a plain restart could run concurrently with
an asynchronous rebuild of the same service.

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-09-14 10:47:07 +02:00
Max Proske
0b9fb4fef9 PRIVATE_PORT optional for docker compose port
Signed-off-by: Max Proske <max@mproske.com>
2026-09-11 11:35:18 +02:00
Sebastiaan van Stijn
063ef45da1 golangci-lint: enable perfsprint linter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-09-10 10:25:34 +02:00
Nicolas De Loof
d82edca1c2 fix(display): harden termWriter lifecycle against review findings on #14119
Transposes the applicable findings from the review of the ttyWriter fix
(#14119) to the rewritten renderer:

- Done waits for the refresh goroutine to exit (outside the lock), so
  nothing can repaint after Done returns and output printed right after
  an operation cannot be garbled by a stray frame
- a nested Start retires the previous cycle instead of leaking its
  refresh goroutine until the parent context is cancelled
- build suspension no longer leaks across cycles: Start resets it and
  starts a fresh block below whatever buildkit wrote
- drop the dead strings.ToLower in startDependencies: the unexported
  start only reads projectName when no Project is passed

The remaining findings are structural to this rewrite: Done is a single
critical section, and per-cycle context cancellation replaces the
doneSignal handshake.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-09-09 16:50:05 +02:00
Guillaume Lours
8018a96fde feat(compose): warn on unsupported compose-file attributes
Compose silently ignored several spec-valid attributes it doesn't
honor outside Swarm mode: deploy.update_config, credential_spec,
ports[].mode: host, and others had no visible effect on a plain
`docker compose up`.

Detection is wired into compose-go's loader as a fail-closed
allowlist: every attribute path the schema declares, minus the
handful this runtime deliberately doesn't implement, plus a few
value-conditional checks for attributes only unsupported for one
specific value. A newly-specified attribute gets flagged
automatically, with no corresponding code change needed here.

Depends on an unreleased compose-go change; go.mod carries a
temporary replace to a fork branch until it's released.

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-09-09 16:32:04 +02:00
Sebastiaan van Stijn
f03c1b7cfb remove jonboulle/clockwork for testing/synctest
The fake clock is only used by tests to control the debounce timer,
but requires carrying a clock abstraction through production code.

Use the standard time package for the debounce ticker and use
testing/synctest to control time in tests instead.

This removes the clock from composeService and BatchDebounceEvents,
and eliminates the clockwork dependency from the watch implementation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2026-09-08 15:55:48 +02:00
Nicolas De Loof
a255af9368 feat(provider): get-service-config provider request
Providers could not see the definition of the service they manage:
options had to be duplicated between the compose file and the provider,
or the provider had to re-resolve the model on its own.

A provider may now emit {"type": "get-service-config"} on stdout;
compose answers on the provider's stdin with one JSON line holding the
resolved canonical configuration of the provider's own service,
straight from the in-memory model. The message can be repeated; each
occurrence is answered with one line. Detection is by construction: a
compose that predates the message aborts on it and never writes to
stdin, so the provider treats EOF as 'unsupported, upgrade compose'.

The example provider demonstrates the round trip, backed by an e2e
scenario; unit tests drive executePlugin against a helper-process
provider and cover the injection.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-09-08 14:53:14 +02:00
hiroto.toyoda
20717bab83 fix(down): spare dangling images of orphaned services, like their tagged image
`down --rmi` removed every dangling image for the project unconditionally,
while its tagged-image counterpart (ImagesToPrune) already skips a service
no longer in the project unless --remove-orphans is set. `down --rmi local`
on a project with an orphaned service therefore kept the tagged image but
deleted its dangling ones - an inconsistent result for one command.

Extract the "list a project's dangling images, remove what a keep predicate
doesn't spare, tolerate individual failures" logic - which down.go and
watch.go's pruneDanglingImagesOnRebuild had each partially and divergently
implemented - into one shared composeService.removeDanglingImages,
parallelized via errgroup like the rest of the codebase's bulk removals.
down.go's keep now mirrors ImagesToPrune's own RemoveOrphans/GetService
check; watch.go keeps its existing built-IDs check, behavior unchanged.

The removal fan-out now also honors --parallel via eg.SetLimit(s.maxConcurrency),
matching push.go/pull.go, instead of always running unbounded. Fixed
TestWatch_Sync's composeService{} literal to set maxConcurrency: -1
(NewComposeService's own default) - the zero value means "allow zero
concurrent goroutines" to errgroup, not "unlimited", and was silently
relying on maxConcurrency being unused before this change.

Signed-off-by: hiroto.toyoda <hiroto.toyoda@dena.com>
2026-09-03 16:00:29 +02:00
hiroto.toyoda
a966bad33a fix(down): spare dangling images of orphaned services, like their tagged image
`down --rmi` removed every dangling image for the project unconditionally,
while its tagged-image counterpart (ImagesToPrune) already skips a service
no longer in the project unless --remove-orphans is set. `down --rmi local`
on a project with an orphaned service therefore kept the tagged image but
deleted its dangling ones - an inconsistent result for one command.

Extract the "list a project's dangling images, remove what a keep predicate
doesn't spare, tolerate individual failures" logic - which down.go and
watch.go's pruneDanglingImagesOnRebuild had each partially and divergently
implemented - into one shared composeService.removeDanglingImages,
parallelized via errgroup like the rest of the codebase's bulk removals.
down.go's keep now mirrors ImagesToPrune's own RemoveOrphans/GetService
check; watch.go keeps its existing built-IDs check, behavior unchanged.

Signed-off-by: hiroto.toyoda <hiroto.toyoda@dena.com>
2026-09-03 16:00:29 +02:00
Guillaume Lours
a896529193 fix(bridge): confirm before wiping non-empty convert output dir
`bridge convert -o <dir>` unconditionally ran `os.RemoveAll` on the
output directory and discarded any error, so a typo (`-o .`, `-o
$PWD`, `-o ~`) silently destroyed unrelated user data.

An empty or missing output directory is still created silently, but a
non-empty one now requires explicit confirmation (prompted
interactively, or via the new `--yes` flag for scripts/CI) before
being wiped.

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-09-03 14:38:41 +02:00
Ricardo Branco
4121ec7d13 test(e2e): bump compose-bridge images to v0.0.7
v0.0.6 dropped the Egress policyType/block from generated
NetworkPolicy manifests since it blocked CoreDNS traffic; update
the fixtures to match.

Signed-off-by: Ricardo Branco <rbranco@suse.de>
2026-09-02 18:21:19 +02:00
Ricardo Branco
63f4764184 test(e2e): bump compose-bridge images to v0.0.7
v0.0.3 had a mis-published multi-arch manifest, causing exec format
errors on non-amd64 architectures.

Fixed upstream in docker/compose-bridge-transformer@665f67e.

Signed-off-by: Ricardo Branco <rbranco@suse.de>
2026-09-02 18:21:19 +02:00
Guillaume Lours
05066c2418 fix(watch): stop swallowing SetRecursive errors via nolint
Split SetRecursive into the existing per-GOOS watcher files instead of
a runtime check + nolint: the non-Windows stub never calls the
always-failing SetRecursive, and Windows failures now propagate as a
real error instead of silently downgrading to non-recursive.

Addresses review from thaJeztah on #14164.

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-09-02 15:07:08 +02:00
Nicolas De Loof
70e17bfc66 build: bump golangci-lint to v2.13.2
Some checks are pending
ci / validate (lint) (push) Waiting to run
ci / validate (validate-docs) (push) Waiting to run
ci / validate (validate-go-mod) (push) Waiting to run
ci / validate (validate-headers) (push) Waiting to run
ci / validate (validate-mocks) (push) Waiting to run
ci / binary (push) Waiting to run
ci / binary-finalize (push) Blocked by required conditions
ci / bin-image-test (push) Waiting to run
ci / test (push) Waiting to run
ci / e2e (plugin, oldstable, graphdriver) (push) Waiting to run
ci / e2e (standalone, oldstable, graphdriver) (push) Waiting to run
ci / e2e (plugin, stable, containerd) (push) Waiting to run
ci / e2e (plugin, stable, graphdriver) (push) Waiting to run
ci / e2e (standalone, stable, containerd) (push) Waiting to run
ci / e2e (standalone, stable, graphdriver) (push) Waiting to run
ci / coverage (push) Blocked by required conditions
ci / release (push) Blocked by required conditions
merge / bin-image-prepare (push) Waiting to run
merge / bin-image (push) Blocked by required conditions
merge / module-image (push) Waiting to run
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
zizmor / zizmor (push) Waiting to run
- migrate gomodguard to gomodguard_v2 (gomodguard is deprecated since
  v2.13 and warns on every run); same blocked modules, flat-list schema
- silence SA4023 in watcher_naive.go: tilt-dev/fsnotify SetRecursive
  only succeeds on Windows, so on any single GOOS the newer staticcheck
  proves the err == nil comparison constant — the cross-platform code
  is correct, the finding is per-GOOS

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-09-02 09:18:25 +02:00
Nicolas De Loof
2be4e2d656 fix(plugins): honor PATHEXT for provider lookup on Windows
Some checks are pending
ci / validate (lint) (push) Waiting to run
ci / validate (validate-docs) (push) Waiting to run
ci / validate (validate-go-mod) (push) Waiting to run
ci / validate (validate-headers) (push) Waiting to run
ci / validate (validate-mocks) (push) Waiting to run
ci / binary (push) Waiting to run
ci / binary-finalize (push) Blocked by required conditions
ci / bin-image-test (push) Waiting to run
ci / test (push) Waiting to run
ci / e2e (plugin, oldstable, graphdriver) (push) Waiting to run
ci / e2e (standalone, oldstable, graphdriver) (push) Waiting to run
ci / e2e (plugin, stable, containerd) (push) Waiting to run
ci / e2e (plugin, stable, graphdriver) (push) Waiting to run
ci / e2e (standalone, stable, containerd) (push) Waiting to run
ci / e2e (standalone, stable, graphdriver) (push) Waiting to run
ci / coverage (push) Blocked by required conditions
ci / release (push) Blocked by required conditions
merge / bin-image-prepare (push) Waiting to run
merge / bin-image (push) Blocked by required conditions
merge / module-image (push) Waiting to run
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
zizmor / zizmor (push) Waiting to run
The PATH fallback of provider discovery appended a hardcoded ".exe" on
Windows, restricting providers to compiled binaries. exec.LookPath
already implements Windows executable resolution (PATHEXT), which finds
.exe but also .com/.bat/.cmd scripts — the suffix only narrowed the
lookup. Drop the executable() indirection and look up the bare name.

Note this only affects the PATH fallback: the docker CLI plugin channel
(docker-<name>) keeps requiring .exe on Windows, per docker/cli plugin
candidate rules.

Fixes #14157

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-09-01 17:23:39 +02:00
Nicolas De Loof
b3284911a3 refactor: NewGraph stops mutating the project
Some checks are pending
ci / validate (lint) (push) Waiting to run
ci / validate (validate-docs) (push) Waiting to run
ci / validate (validate-go-mod) (push) Waiting to run
ci / validate (validate-headers) (push) Waiting to run
ci / validate (validate-mocks) (push) Waiting to run
ci / binary (push) Waiting to run
ci / binary-finalize (push) Blocked by required conditions
ci / bin-image-test (push) Waiting to run
ci / test (push) Waiting to run
ci / e2e (plugin, oldstable, graphdriver) (push) Waiting to run
ci / e2e (standalone, oldstable, graphdriver) (push) Waiting to run
ci / e2e (plugin, stable, containerd) (push) Waiting to run
ci / e2e (plugin, stable, graphdriver) (push) Waiting to run
ci / e2e (standalone, stable, containerd) (push) Waiting to run
ci / e2e (standalone, stable, graphdriver) (push) Waiting to run
ci / coverage (push) Blocked by required conditions
ci / release (push) Blocked by required conditions
merge / bin-image-prepare (push) Waiting to run
merge / bin-image (push) Blocked by required conditions
merge / module-image (push) Waiting to run
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
zizmor / zizmor (push) Waiting to run
Building the dependency graph silently deleted unresolvable optional
depends_on references from the caller's project — and since DependsOn
is a map, even from projects sharing it. What later readers observed
depended on how many graphs had been built before them.

The pruning becomes an explicit step, applied once where the project
consumed by the engines is established — create and start — before the
graph, the dependency waits and the com.docker.compose.depends_on
label read the model, using Project.WithoutUnresolvedOptionalDependencies
introduced by compose-spec/compose-go#922 (go.mod bumped to the
post-merge commit). NewGraph now just skips the edge; required dangling
references still flow to its existing errors (naming the profiles that
would enable the dependency).

No behavior change; epic #14081, Lot 0 step 4 — precondition for a
canonical project object across plan phases.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-09-01 10:19:10 +02:00
Nicolas De Loof
081ca8f645 chore(compose): orphan semantics stated where up and down diverge
Some checks failed
ci / validate (lint) (push) Waiting to run
ci / validate (validate-docs) (push) Waiting to run
ci / validate (validate-go-mod) (push) Waiting to run
ci / validate (validate-headers) (push) Waiting to run
ci / validate (validate-mocks) (push) Waiting to run
ci / binary (push) Waiting to run
ci / binary-finalize (push) Blocked by required conditions
ci / bin-image-test (push) Waiting to run
ci / test (push) Waiting to run
ci / e2e (plugin, oldstable, graphdriver) (push) Waiting to run
ci / e2e (standalone, oldstable, graphdriver) (push) Waiting to run
ci / e2e (plugin, stable, containerd) (push) Waiting to run
ci / e2e (plugin, stable, graphdriver) (push) Waiting to run
ci / e2e (standalone, stable, containerd) (push) Waiting to run
ci / e2e (standalone, stable, graphdriver) (push) Waiting to run
ci / coverage (push) Blocked by required conditions
ci / release (push) Blocked by required conditions
merge / bin-image-prepare (push) Waiting to run
merge / bin-image (push) Blocked by required conditions
merge / module-image (push) Waiting to run
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
zizmor / zizmor (push) Waiting to run
docs-upstream / docs-yaml (push) Has been cancelled
docs-upstream / validate (push) Has been cancelled
A container is orphaned when it is a one-off (compose run) that FINISHED
its task, or when it carries the project labels but its service is not
defined by the compose model — the typical leftover after the compose
file was edited. A still-RUNNING one-off is somebody's live session:
'up --remove-orphans' cleans up leftovers and must never kill it.
'down --remove-orphans' is the explicit stop-the-application action, so
it DOES take running one-offs down — through the per-service removal
loop, not the orphan branch.

That asymmetry was implemented but stated nowhere: isOrphaned's comment
only mentioned the model-absent half, down.go removed running one-offs
as an unexplained side effect of including one-offs in the listing, and
the observed-state collection silently dropped running one-offs with no
hint it was deliberate. No behavior change — the semantics are now
written at all three sites and pinned by tests: the predicate matrix,
the observed-state classification (a running one-off is neither a
service replica nor an orphan), and down's single stop+remove path for
a running one-off of a declared service.

Closes item C.4 of #14074.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-31 17:09:19 +02:00
hiroto.toyoda
727fa0e67d test(build): re-enable wrong ssh key id test
Some checks failed
ci / validate (lint) (push) Has been cancelled
ci / validate (validate-docs) (push) Has been cancelled
ci / validate (validate-go-mod) (push) Has been cancelled
ci / validate (validate-headers) (push) Has been cancelled
ci / validate (validate-mocks) (push) Has been cancelled
ci / binary (push) Has been cancelled
ci / bin-image-test (push) Has been cancelled
ci / test (push) Has been cancelled
ci / e2e (plugin, oldstable, graphdriver) (push) Has been cancelled
ci / e2e (standalone, oldstable, graphdriver) (push) Has been cancelled
ci / e2e (plugin, stable, containerd) (push) Has been cancelled
ci / e2e (plugin, stable, graphdriver) (push) Has been cancelled
ci / e2e (standalone, stable, containerd) (push) Has been cancelled
ci / e2e (standalone, stable, graphdriver) (push) Has been cancelled
merge / bin-image-prepare (push) Has been cancelled
merge / module-image (push) Has been cancelled
Scorecards supply-chain security / Scorecards analysis (push) Has been cancelled
zizmor / zizmor (push) Has been cancelled
ci / binary-finalize (push) Has been cancelled
ci / coverage (push) Has been cancelled
ci / release (push) Has been cancelled
merge / bin-image (push) Has been cancelled
moby/buildkit#5558, which this test was disabled for, is fixed and
released well before the buildkit v0.26.3 currently vendored here.
Verified the re-enabled case passes (3 consecutive runs).

Signed-off-by: hiroto.toyoda <hiroto.toyoda@dena.com>
2026-08-28 09:39:00 +02:00
Nicolas De Loof
720da68475 chore(compose): comments state behavior, not history
Some checks failed
ci / validate (lint) (push) Waiting to run
ci / validate (validate-docs) (push) Waiting to run
ci / validate (validate-go-mod) (push) Waiting to run
ci / validate (validate-headers) (push) Waiting to run
ci / validate (validate-mocks) (push) Waiting to run
ci / binary (push) Waiting to run
ci / binary-finalize (push) Blocked by required conditions
ci / bin-image-test (push) Waiting to run
ci / test (push) Waiting to run
ci / e2e (plugin, oldstable, graphdriver) (push) Waiting to run
ci / e2e (standalone, oldstable, graphdriver) (push) Waiting to run
ci / e2e (plugin, stable, containerd) (push) Waiting to run
ci / e2e (plugin, stable, graphdriver) (push) Waiting to run
ci / e2e (standalone, stable, containerd) (push) Waiting to run
ci / e2e (standalone, stable, graphdriver) (push) Waiting to run
ci / coverage (push) Blocked by required conditions
ci / release (push) Blocked by required conditions
merge / bin-image-prepare (push) Waiting to run
merge / bin-image (push) Blocked by required conditions
merge / module-image (push) Waiting to run
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
zizmor / zizmor (push) Waiting to run
docs-upstream / docs-yaml (push) Has been cancelled
docs-upstream / validate (push) Has been cancelled
Eleven comments in the reconciler and its neighbors justified decisions
by pointing at code that no longer exists ('matching the previous
ensureNetwork behavior', 'same way as convergence.go:138-160' — lines
that today hold unrelated code) instead of stating the behavior and its
reason. A reader following those pointers finds nothing, or worse,
something else.

Each is rewritten as a self-contained behavioral statement. Two carry
real weight:
- reconcileService's up-to-date branch now says explicitly that
  starting containers is NOT the plan's job (a separate start phase in
  start.go does it) — the previous wording invited 'fixing' the
  StateCreated case;
- sortContainers now documents that its ordering IS the scale-down
  policy (the tail it produces is what i >= expected trims), instead of
  referencing stale line numbers.

Part of #14074 (A: the code misdescribes its own structure).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-27 12:14:54 +02:00
Nicolas De Loof
e801d5fc77 chore(compose): rename convergence.go, whose convergence engine no longer exists
The convergence type this file was named after was removed in fbea647b9,
but the file kept its name and its doc comment ('convergence manages
service's container lifecycle') ended up attached to the unrelated
getScale(). Anyone asked to 'change the convergence logic' lands here,
while the actual reconciliation lives in reconcile.go.

The file is renamed to service_containers.go with a header stating what
it really contains: the per-service container helpers shared by both
lifecycle engines (naming, service-reference resolution, dependency
waiting, container creation, startup). getScale gets a doc comment that
describes getScale.

Part of #14074 (A: the code misdescribes its own structure).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-27 11:48:19 +02:00
Nicolas De Loof
8fbda262bb build: error messages name the real remedy, not a removed builder
The classic-builder rejections said 'set DOCKER_BUILDKIT=1 to use
BuildKit', a leftover from the internal BuildKit builder removed in
af579ebd4. Since then BuildKit builds go through the buildx plugin:
compose falls back to the classic builder when BuildKit is disabled
(DOCKER_BUILDKIT=0) OR when buildx is not installed — so the advertised
fix does nothing for users who are simply missing buildx.

The five messages now state the actual requirement (buildx installed,
DOCKER_BUILDKIT not forced to 0), and the silent-fallback warning when
buildx is missing now says what the fallback implies instead of a bare
'requires buildx plugin to be installed'.

Part of #14074 (A: the code misdescribes its own structure).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-27 11:05:09 +02:00
Nicolas De Loof
7af58e3626 test(e2e): gate the e2e suite behind the build tag the docs already promise
AGENTS.md documents 'Test unit: go test ./pkg/...' and 'E2E tests:
go test -tags e2e ./pkg/e2e/'. Both statements were false: no file in
pkg/e2e carried an e2e build constraint, so the -tags flag was a no-op
and the 'unit' command actually launched the full e2e suite — requiring
a Docker daemon and the locally built binary, creating and destroying
containers, for up to 20 minutes. CI only avoided this by grepping e2e
out of the package list (Dockerfile).

Every *_test.go in pkg/e2e now carries //go:build e2e (merged with the
existing !windows constraints), and the Makefile e2e targets pass the
tag. The documented commands become true:
- go test ./pkg/...          -> pkg/e2e reports [no test files]
- go test -tags e2e ./pkg/e2e -> runs the suite

AGENTS.md now also states the daemon/binary prerequisites explicitly.

Part of #14074 (A: the code misdescribes its own structure).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-27 11:02:07 +02:00
Guillaume Lours
03ad3cc1c2 fix(build): honor provenance/sbom false in per-service bake attest
Some checks are pending
ci / validate (lint) (push) Waiting to run
ci / validate (validate-docs) (push) Waiting to run
ci / validate (validate-go-mod) (push) Waiting to run
ci / validate (validate-headers) (push) Waiting to run
ci / validate (validate-mocks) (push) Waiting to run
ci / binary (push) Waiting to run
ci / binary-finalize (push) Blocked by required conditions
ci / bin-image-test (push) Waiting to run
ci / test (push) Waiting to run
ci / e2e (plugin, oldstable, graphdriver) (push) Waiting to run
ci / e2e (standalone, oldstable, graphdriver) (push) Waiting to run
ci / e2e (plugin, stable, containerd) (push) Waiting to run
ci / e2e (plugin, stable, graphdriver) (push) Waiting to run
ci / e2e (standalone, stable, containerd) (push) Waiting to run
ci / e2e (standalone, stable, graphdriver) (push) Waiting to run
ci / coverage (push) Blocked by required conditions
ci / release (push) Blocked by required conditions
docs-upstream / docs-yaml (push) Waiting to run
docs-upstream / validate (push) Blocked by required conditions
merge / bin-image-prepare (push) Waiting to run
merge / bin-image (push) Blocked by required conditions
merge / module-image (push) Waiting to run
Scorecards supply-chain security / Scorecards analysis (push) Waiting to run
zizmor / zizmor (push) Waiting to run
When a service set `build.provenance: false` (or `sbom: false`), the
bake attest entry was silently omitted (the field is `omitempty`),
so BuildKit applied its defaults and attached a provenance attestation
anyway. The attestation changes the manifest-list digest on every build,
which caused containers to be needlessly recreated on `up --build`.

Emit `"type=<t>,disabled=true"` instead, which BuildKit's
`Attest.UnmarshalText` parses as `Disabled:true` and forwards to
BuildKit as an explicit disable signal.

Extract `appendAttest` to avoid duplicating the three-branch switch
for both provenance and sbom.

Fixes #14111

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-08-25 17:46:03 +02:00
Nicolas De Loof
a472f714d5 registry: encode credentials with moby's authconfig package
EncodedAuth now delegates the X-Registry-Auth wire encoding to
moby/api/pkg/authconfig (same module, already required) instead of
hand-rolling the JSON+base64url pair, and push.go reuses it instead of
its own copy. The CLI and moby AuthConfig structs are field-for-field
identical, so a direct type conversion bridges them.

The explicit GetAuthConfigKey normalization drops out of EncodedAuth:
docker/cli's configfile.GetAuthConfig normalizes the Docker Hub domain
itself. It remains for the OCI resolver, whose credential callback
receives the network host actually contacted (registry-1.docker.io),
which the configfile does not map.

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-25 17:27:26 +02:00
Nicolas De Loof
84dcd428ca remove buildx as a Go dependency
buildx was imported for two narrow purposes, both replaced by
docker/cli machinery compose already uses elsewhere:

- storeutil.GetProxyConfig -> configfile.ParseProxyConfig, the same
  lookup docker build uses (also picks up ALL_PROXY and lowercase
  variants)
- the dry-run image resolver (buildx builder + imagetools) -> a
  DistributionInspect through the daemon with credentials from the CLI
  config file, i.e. the same path a real pull/push would take; the
  encoded-auth helper moves to internal/registry, shared with pull.go

bake is unaffected: it already execs the buildx CLI plugin. Dropping
the module also drops the transitive aws-sdk-go-v2 footprint
(-328 lines of go.sum).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-25 17:27:26 +02:00
Guillaume Lours
6d3d68c1fa fix(display): stop ttyWriter.Done from hanging after context cancel
Fixes #14114: Done() blocked forever once the render goroutine had
already exited via ctx.Done() (e.g. after SIGTERM), since it sent on an
unbuffered channel with nobody left to receive.

Closing the channel instead of sending never blocks, but a bare close()
panics the moment ttyWriter runs a second Start/Done cycle on the same
shared bus - which already happens today (run's nested create/start,
publish's nested push, rm --stop's sequential stop then remove). Start
now hands out a fresh, once-only-closable signal each cycle instead of
a single shared channel, and run/publish stop opening a redundant
nested bracket around calls already inside one.

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-08-25 11:38:50 +02:00
hiroto.toyoda
8ddbdc41a2 fix(watch): match Dockerfile/compose ignore patterns by literal name and basename
Signed-off-by: hiroto.toyoda <hiroto.toyoda@dena.com>
2026-08-24 14:48:22 +02:00
hiroto.toyoda
11d4aadf4f fix(watch): exclude Dockerfile and compose files from initial sync
Signed-off-by: hiroto.toyoda <hiroto.toyoda@dena.com>
2026-08-24 14:48:22 +02:00
Guillaume Lours
54785c4be6 feat(hooks): clean up retained pre_start containers on compose down
A pre_start hook container retained after failure (introduced in the
previous commit) is created without ConfigHashLabel so getContainers()
and every downstream compose operation (down, ps, stop, rm) is blind
to it. Without explicit cleanup it would survive compose down.

This commit adds removePreStartHookContainers(), called from down()
after regular container teardown and before network removal (so hook
containers are gone before their networks are removed). The cleanup is
project-wide on a full compose down; scoped to the requested services
when compose down <svc...> is used. Individual ContainerRemove errors
are logged at warn level and do not abort the teardown.

Changes:
- pkg/compose/down.go: removePreStartHookContainers helper + call in down()
- pkg/compose/filters.go: add //nolint:unparam to hookFilter (param is
  architecturally correct; today only preStartHookType is in use)
- pkg/compose/down_test.go: update all existing down tests to expect the
  new ContainerList call; add TestDownRemovesRetainedPreStartHookContainers
  and TestDownHookContainerRemovalFailureIsNonFatal

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-08-20 18:57:02 +02:00
Guillaume Lours
674010661a feat(hooks): retain failed pre_start containers for post-mortem
A failing pre_start hook container is now retained (AutoRemove: false)
so operators can run 'docker logs <id>' and 'docker ps -a' to diagnose
the failure. On success the container is removed explicitly, mirroring
the old AutoRemove behaviour including anonymous volumes.

Before each pre_start run, stale hook containers from a previous failed
run are detected via project+service+HookLabel filters and force-removed
so they do not accumulate.

Changes:
- pkg/api/labels.go: add HookLabel (com.docker.compose.hook)
- pkg/compose/filters.go: add hookFilter helper
- pkg/compose/pre_start.go:
  - AutoRemove: false in createPreStartContainer
  - HookLabel added to container labels
  - runPreStartHook: explicit ContainerRemove on success; retain on failure
  - removeOrphanPreStartContainers: new helper called in runPreStart
- Tests: update all existing pre_start tests for the new flow; add
  feature tests (success removes, failure retains, orphan cleanup) and
  coverage-gap tests (lowestNumberedContainer, waitPreStart cancel,
  preStartResultErr, streamPreStartLogs error paths, old-API network
  paths, ExecCreate/Attach/Inspect errors, hookExitError branches)

Coverage after: hook.go 100%, pre_start.go most functions 100%
(was 89% and 64% respectively per Codecov delta).

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-08-20 18:57:02 +02:00
Guillaume Lours
bbec45855a fix(hooks): stderr-biased tail and pre_start parity
On top of the output-tail infrastructure from the previous commit:

Stderr bias: split the combined output tail into separate stdout and stderr
buffers. stdcopy.StdCopy(wOut, wErr, reader) now routes each stream to its
own outputTail. The error message prefers stderr (where hooks write their
actual error) and falls back to stdout when stderr is empty:

  db hook exited with status 1: Table 'service.sites' doesn't exist

A new hookExitError helper encapsulates this preference.

pre_start parity: streamPreStartLogs previously short-circuited and returned
immediately when listener was nil (detached mode / compose up -d), leaving
the tail buffers empty and failure errors without context. The log stream is
now always opened. The demultiplexed streams fill separate tailOut / tailErr
buffers. On non-zero exit the stderr-biased tail is appended to the error
(same behaviour as exec hooks).

Tests:
- TestRunHook_StderrBias: stdout noise stays out of the error when stderr
  has content.
- TestPreStart_DetachedModeAttachesLogs: ContainerLogs is called even with
  a nil listener.
- TestPreStart_FailureIncludesTail: a failing pre_start hook carries its
  stderr output in the returned error.

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-08-20 18:57:02 +02:00
Nicolas De Loof
b857135cb9 fix: hold startMx on the start path actually exercised
startMx exists to serialize ContainerStart calls because the engine
assigns published ports from ranges non-atomically. It is held by the
plan executor's execStartContainer — a nearly dead code path (plans
only emit OpStartContainer for paused/dead containers) — and NOT by
startServiceContainer, the path every up/start actually runs. The
protection is thus inoperative today: concurrent services publishing
overlapping port ranges can race in the engine.

Take the mutex around the real ContainerStart call and document the
contract on the variable. Starts of different services were already
serialized per-service (replicas start sequentially); this serializes
the API call across services too, matching what the plan engine will
do once the start phase converges (#14081).

Part of #14074 (section C) / #14081 (lot 0).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-20 15:30:03 +02:00
Nicolas De Loof
06f4f9d622 fix: dependency wait timeout surfaced only by luck
waitDependency returned nil whenever its context ended, so an expired
wait deadline was reported only when the 500ms poll ticker happened to
win the select race and the condition check then failed on the expired
context. Whether `up --wait --wait-timeout` failed or silently
succeeded after the timeout was a coin toss, and the "timeout waiting
for dependencies" mapping in waitDependencies was unreachable on the
direct path.

Return the context error when the deadline expired; a plain
cancellation (Ctrl-C) is still not reported as a dependency failure.
The unit test pins both sides deterministically (a deadline shorter
than the first tick used to be swallowed with certainty); the e2e
scenario documents the user-visible contract: `up --wait
--wait-timeout` on a never-healthy service fails with "application not
healthy after Xs" while the container keeps running.

Part of #14074 (section C) / #14081 (lot 0).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-20 15:16:57 +02:00
Nicolas De Loof
6443bc1da2 chore(mocks): fix the stale mockgen invocation and pin its version
The `mocks` target still passed the pre-rename `Service` argument to
mockgen (the interface is `Compose` since the rename) and installed
whatever mockgen happened to be on PATH. Run it through `go run` with a
pinned version so regeneration is reproducible; v0.4.0 is the last
version usable here, as the package mode introduced in v0.5 copies
parameter names from the mocked source and moby's APIClient names its
variadic options `_`, which produces invalid Go code.

Regenerated pkg/mocks with the fixed invocation (header-only change).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-20 15:16:43 +02:00
Nicolas De Loof
0d7d22585a test: unit-lock the imperative start path
The imperative start engine (startService, startServiceContainer,
restart, waitDependencies) is only locked by e2e tests today. Before
converging it into the plan engine (#14081), pin its observable
behavior with unit characterization tests: start idempotence and the
silent no-op on running replicas, the "no container to start" error,
pre_start gating (mixed replicas skip the hook, all-stopped runs it
once on the lowest-numbered replica), the inject -> start -> post_start
sequence with Started only emitted after hooks, restart's depends_on
restart:true selection and its pre_stop -> ContainerRestart ->
post_start order, --wait's completed_successfully condition mapping,
and waitDependencies' missing/failing dependency semantics for
required and optional dependencies.

No production change.

Part of #14074 (section C) / #14081 (lot 0).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-20 15:09:16 +02:00
Guillaume Lours
3a62ae63ce fix(ps): honor psFormat from Docker CLI config when no --format flag given
The --format flag defaulted to "table" instead of "", so the
opts.Format == "" guard in runPs never fired for a bare compose ps.
Setting the default to "" lets the existing config-fallback path work,
matching the behaviour of docker ps.

Fixes #13643
Supersedes #14032

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-08-20 14:05:34 +02:00
Guillaume Lours
9e4a3c7457 fix(network): pass IPAM options when creating networks
createNetwork built the IPAM object in two separate blocks; the second
silently overwrote createOpts.IPAM with a fresh empty struct and never
forwarded n.Ipam.Options to the daemon. Consolidate into a single block
covering driver, config pool, and options.

Fixes #13785
Supersedes #13936

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-08-20 12:36:23 +02:00
Nicolas De Loof
5671532583 api: RunOptions documents its partial embedding and its double life as ExecOptions
RunOptions embeds CreateOptions, which suggests every creation option
applies to the one-off's dependencies; in reality only Build,
IgnoreOrphans, RemoveOrphans and QuietPull are propagated
(startDependencies builds a fresh CreateOptions from exactly those
four). Setting Recreate or Inherit on a RunOptions was a silent no-op.
The same struct also serves Exec, which honors only the exec-relevant
subset of its ~20 fields.

Both facts are now stated on the type, the embedding and the
Exec-only Index field.

Part of #14074 (B: pkg/api promises things the implementation does not
honor).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-20 12:32:25 +02:00
Nicolas De Loof
62eb97e8e5 api: StartOptions says which half of it Start actually reads
StartOptions fuses two contracts: the fields s.start honors (Project,
AttachTo-as-project-source, Wait, WaitTimeout) and the fields only Up's
foreground session reads (Attach, OnExit, ExitCodeFrom, Watch,
NavigationMenu). Passing the latter to Start compiled, returned no
error, and did nothing.

Each session field is now marked 'honored by Up's foreground session
only; ignored by Start'. Two implicit behaviors become explicit along
the way: Attach doubles as the detached/interactive mode switch (nil
means Up returns once containers are started), and AttachTo has two
unrelated meanings (project reconstruction source for Start, log
scoping for Up).

Part of #14074 (B: pkg/api promises things the implementation does not
honor).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-20 12:32:25 +02:00
Nicolas De Loof
00ce78a055 api: state the scoping invariant Services fields never had
Three structs carry a Services field with three different meanings:
ProjectLoadOptions.Services narrows the loaded project (the only real
filter), CreateOptions.Services only selects which recreation policy
applies to which service while the WHOLE project converges, and
StartOptions.Services only scopes Up's log monitor and is ignored by
Start. Nothing said so: an SDK consumer calling
Create(project, CreateOptions{Services: []string{"web"}}) reasonably
expected to create web alone, and got the full project.

The interface doc now states the underlying invariant — methods act on
the whole project they receive; scoping is done by narrowing the
project, and downstream Services fields are intent markers, not
filters — and each of the three fields documents its actual effect.

Part of #14074 (B: pkg/api promises things the implementation does not
honor).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2026-08-20 12:32:25 +02:00
Guillaume Lours
ea8d9bc3db fix(watch): sync all files on initial_sync, regardless of mtime
initialSyncFiles and initialSyncDirectory filtered out any host file
whose mtime predated the image creation time — which is exactly the
files that initial_sync is meant to copy. The imageCreatedTime lookup
(ContainerList + ImageInspect) was also a silent failure point when the
container list returned nothing. Remove the filter and the helper
entirely: sync all files that are not ignored or bind-mounted.

Fixes #13725
Supersedes #14031

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
2026-08-20 12:05:36 +02:00