The rebuild now reports "Failed to recreate and restart service(s) after
update" instead of "Application failed to start after update", which
TestRebuildOnDotEnvWithExternalNetwork still asserted was absent: the
string no longer existed anywhere, so the check could never fail and the
test no longer detected a failed rebuild.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The wiring of --wait-timeout into the plan's dependency waits was only
covered by executor tests that set the timeout by hand, and by an e2e
without depends_on. The scenario has a dependency that never gets
healthy and checks that up -d --wait-timeout fails with the dependency
timeout message (without the raw deadline error) and leaves the
dependent service created but not started.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The dependencies a one-off starts go through the regular create path
and its config-hash divergence check, with no way to opt out: any
resolution difference with the command that created them (environment,
model edit) recreates the dependency — destroying state a previous run
built up (issue #13769's database losing its users). up already offers
--no-recreate for exactly this; run now aligns: the flag maps to the
"never" recreate policy, propagated through the RunOptions.CreateOptions
fields that startDependencies previously dropped. Default behavior is
unchanged (empty policy = "diverged").
Closes#13769
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Reproduces docker/compose#14287: a hook inheriting the service's
env_file (or declaring its own) runs without those variables.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
TestBuildTLS slept a fixed second after the dind daemon logged that it
listens, hoping the client certificates were written; poll for them
instead. TestForwardKeepsStreamingAfterHalfClose had 150ms of margin
between each chunk and the 300ms idle grace, which a loaded CI runner can
eat; use 100ms chunks against a 1s grace.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
extension.md gains the Image distribution section: invocation contract
(--image/--digest/--created/--source/--policy), the get-image request
and its chunked image-stream answer (zero-chunk terminated, stdin
exclusivity and drain-before-next-answer contract), the two-branch
decision guidance (digest as identity, created as ordering fallback,
reproducible builds caveat), and the support-by-presence metadata
convention is now stated as the general rule. The example provider
implements pull end to end — request, stock chunked reader, hash
recorded at PROVIDER_PULL_MARKER — and the e2e scenario locks the
whole path: up builds the service image and streams it to the provider
under the local verdict, compose pull re-invokes it under the
freshness contract.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
compose-go's loader now silently renames both deprecated
x-initialSync and x-develop extensions to their official
initial_sync and develop attributes before compose ever sees them
as extensions, so the legacy lookups and deprecation warnings for
both can no longer be reached. That's why the e2e test asserting on
the x-initialSync warning broke under the bump.
Simplify both lookups to rely on the promoted fields directly, and
rewrite the e2e test to check the legacy attribute still triggers a
real initial sync instead of asserting on an unreachable message.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
A GitHub Copilot PR review (docker/compose#14221#pullrequestreview-5300897216)
found three real issues in the reconciliation-plan-based hook runners, all
confirmed by direct reproduction against a live daemon before being fixed:
- **Hook runners excluded from discovery, stale runners persist.**
collectObservedState's single getContainers call requires ConfigHashLabel
(getDefaultFilters), which hook runners deliberately never carry -- so they
never reached the classification loop that populates HookContainers, and
planPreStartHookRunners never saw a stale runner to purge. Reproduced: a
failed hook run followed by a second `up` failed with "container name ...
already in use" instead of recovering. Fixed by listing hook containers
separately, by project+hook label alone (new getHookContainers), and
merging them in before classification. Locked in by a new e2e scenario,
TestPreStartHookRetryAfterFailureSucceeds, that reproduces the exact
failure-then-retry sequence.
- **Prepared hook runners misclassified as service replicas.** start's
containers.filter(isService(name), isNotOneOff) has no reason to exclude a
hook runner: it carries the same project/service labels as a real replica
and no one-off label either. Added isNotHookContainer to the filter chain
in startService, so a hook runner is never counted as (or started as) a
surviving replica.
- **Per-replica hooks create runners before unsupported configuration is
rejected.** per_replica is not yet supported (#14259): runPreStart
validates every hook up front and rejects the whole service before
executing any of them, but planPreStartHookRunners had no equivalent
check, so create/up would plan (and leave behind) runners for a service
whose hooks could never run. Mirrored the same up-front, all-or-nothing
validation in planPreStartHookRunners.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
pre_start hook containers become first-class resources of the
reconciliation plan. The reconciler plans one OpCreateHookContainer per
declared hook — gated on the same predicate the start phase uses to run
the hooks: a replica to start and no running replica SURVIVING the plan
(a recreated replica is not running at start time, so its runners are
planned too). Runner creation depends on every replica node (the
executor resolves the VolumesFrom target against a final live view),
on the infrastructure, and on the purge of previously observed runners:
names are deterministic (<project>-<service>-pre_start-<i>) so repeated
plans converge on the same container instead of accumulating anonymous
ones, and a new HookIndexLabel ties each runner to its hook.
runPreStart becomes pure execution: it looks up the created-state
runner for each hook by label and never creates one. A declared hook
without a prepared runner is an actionable error naming the
reconciliation command (docker compose up <service>) — the accepted
consequence is that stop-then-start of a hooked service errors, since
runners are consumed on success (locked by e2e). Success removal,
failure retention and cancellation cleanup are unchanged.
Runners stay invisible to the start/ps container listings — they carry
no ConfigHashLabel, which getDefaultFilters requires — and down keeps
removing them by hook label.
Rebased onto main, which diverged substantially since this branch was
opened (the unrelated container-spec-layering refactor, #14093,
restructured compose-go's ServiceConfig/PreStartHook types this PR also
touches). Reconciling pre_start.go/pre_start_test.go against that
refactor surfaced a real bug: createPreStartContainer builds the hook's
config via the shared getCreateConfigs, which unconditionally stamps a
ConfigHashLabel — silently contradicting this same commit's own
"invisible to ps/start" guarantee above, since that label is exactly
what getDefaultFilters requires. Fixed by deleting the label after
getCreateConfigs returns, with a test asserting its absence (an
independent local review, via the GitHub Copilot CLI, and a
fresh-context review agent both converged on catching this during
rebase verification). Also restored two security/label tests that only
existed on one side of the rebase (SecuritySensitiveFieldsHonored,
HookLabelsMerged), ported to call createPreStartContainer directly.
Also addresses glours' three review comments on this PR: hook-container
create operations for a service no longer chain serially (each depends
on the same shared containerNodes+infraDeps+purges set, letting the
executor create them in parallel), the corresponding plan-string test
assertion, and a clearer missing-runner error message hinting this
isn't necessarily a planning bug.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A service migrated from regular replicas to a provider left its old
containers standing: not converged (a provider service has no replicas
to converge), not orphaned (the service is still in the model). The
first up after the migration then failed creating the relay, because
the leftover replica still held the service's canonical container name
the relay takes over. And even without the name conflict (replicas
2..n of a scaled service), stale replicas kept the service's network
alias, competing with the relay on DNS and serving outdated traffic.
The reconciler now plans a stop+remove for every observed non-relay
container of a provider service, and the RunProvider node depends on
those removals: the canonical name is free by the time
ensureServiceRelay creates the relay. The relay itself (RelayLabel) is
the provider service's legitimate container and is left alone —
converged by ensureServiceRelay, not by the plan.
The e2e scenario migrates a deployed service to a provider and locks
the replacement (Recreated, relay reachable at the compose-native
address); the Scenario DSL gains FromFile — a local mirror of
FromRemote — to switch the model between steps.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The exact pull-failure wording is daemon-backend dependent (classic
graphdriver: "pull access denied for ..."; containerd image store:
"...: not found"), and OutputMatches only checks stdout while the
daemon's progress line is written to stderr. Assert the image name
appears on stderr instead, which holds regardless of backend or
wording.
Signed-off-by: Ricardo Branco <rbranco@suse.de>
wantAbs was just wantSource under a different name, left over from an
earlier revision. Use wantSource directly.
(glours review on #14234)
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
BindMountSource kept iterating after locating the bind mount at the
expected destination, so a hypothetical second bind entry at the same
target would trip a spurious mismatch even though the first entry
already confirmed the source. Break out of the loop once a match is
found, per docker-agent review feedback on #14234.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
.Mounts also lists named volumes and tmpfs entries at the same
Destination; matching on Destination alone let those shadow the
intended bind mount, either failing with a confusing volume Source or
silently passing against an empty tmpfs Source. Filter to Type ==
"bind" before comparing, as the function's name and doc promise.
Reported by docker-agent.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
filepath.Abs resolves a relative wantSource against the test process
cwd, not any project directory -- silently comparing against the
wrong base for a future caller passing a relative path, even though
the doc comment frames this as a general check. Reject a relative
wantSource explicitly instead.
(docker-agent review on #14234)
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
TestOciRemoteProjectDirectory publishes a project whose only service
mounts a relative volume, then runs `up` on the oci:// artifact with an
explicit --project-directory pointing elsewhere. The relative volume
must resolve against that directory.
It currently fails against the pinned compose-go: LoadConfigFiles
defaults the working dir to the downloaded artifact's own cache
directory whenever a remote resource loader is involved, silently
overriding the explicit --project-directory the same way it would a
mere default -- docker/compose#14224. On Docker Desktop this surfaces
as a hard failure (the cache directory isn't a shared mount); on Linux
it silently mounts the wrong directory, matching the original report.
New BindMountSource check (pkg/e2e/checks.go) pins a service's bind
mount source to an exact expected path, for tests that need to verify
which working directory a relative volume path resolved against.
Needs the compose-go fix (github.com/ndeloof/compose-go@a626c70,
branch 14224-working-dir) to pass -- not yet reflected in go.mod here.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
TestWatch/debian flakes on loaded runners during 'writing to a file
until Compose watch is up and running': that bootstrap loop ran under
poll.WaitOn's default 10s budget, which must absorb image pull/build,
container start and watcher initialization. Only the bootstrap gets the
2-minute budget — every later step keeps the sharp default so a real
sync regression still fails fast.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The FinishedAt anchor did not cure the third-run loss on CI (still 2
worlds for 3 exit notices after a full minute, oldstable runner). The
remaining suspects are on both sides of the API: a line the daemon
never captured (copier torn down before a millisecond-lived run's
output) or a line compose still fails to relay. On timeout the test
now dumps `docker logs` for the container — ground truth that
discriminates the two on the next CI occurrence.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Three tests failed 9 CI runs across 5 branches this week, all on
asynchronous-observation races, none reproducible locally:
- RequireServiceState asserted on a single `compose ps` snapshot; the
daemon reports state transitions asynchronously from everything else
a test observes (TestUpDependenciesNotStopped saw 'created' while
the container's logs were already flowing). It now polls until the
state converges (15s bound).
- TestAttachRestart counted restart log lines in a snapshot taken as
soon as the third exit notice appeared; exit notices come from the
events monitor while log lines come from the re-attached logs
stream — two channels with no ordering between them. The count is
now awaited like the exit notices already were; a genuinely lost
line still fails, by timeout.
- TestUpExitCodeFromContainerKilled ran a full up+abort cycle under a
60s ceiling, once exceeded on a loaded oldstable runner; raised to
120s.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
docker-agent review: without --build, an image tagged
e2e-additional-context-base:latest left over from a previous run would
let up skip the build entirely, passing the test without exercising
the additional_contexts-to-a-disabled-service path it exists to lock
in.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A service reachable only as another service's build additional_contexts
must still build even when its own profile is inactive: it's referenced
for its image, not started as a workload -- the same tolerance the
depends_on consistency check already grants a disabled, non-required
dependency, and the same case addBuildDependencies already special-cases
on the docker/compose side.
compose-go's own consistency check doesn't grant that tolerance yet: it
rejects the reference outright, failing the whole project load before
addBuildDependencies (or anything else, up included) ever runs -- for
up, build and up --no-build alike, exactly as reported.
This is expected to fail until compose-go bumps to include
compose-spec/compose-go#931.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The previous fix only covered create/start. projectOrName backs every
other service-targeting command too (stop, kill, pause, unpause, logs,
rm, down, ps, events), so they still surfaced the same two problems: a
raw "no such service" error for a job name, and -- when
COMPOSE_PROJECT_NAME is set -- a silent no-op, since projectOrName's
label-driven fallback swallows the load error entirely.
Fold the check into projectOrName so every caller is fixed at once,
instead of duplicating the wiring per command. start.go's own
job-handling code, now redundant, is removed; jobTargetErr and
unselectedJobs (cmd/compose/run.go) are unchanged, just called from one
more place.
Adds unit coverage for projectOrName itself (including the
COMPOSE_PROJECT_NAME path and the typo-correlation edge case) and e2e
coverage for two more callers (stop, down) to confirm the fix lives in
the shared helper rather than being re-implemented per command.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
projectOrName falls back to a label-driven, file-less project on any
load failure when COMPOSE_PROJECT_NAME is set -- swallowing the load
error entirely, including "no such service" for a target that's
actually a declared job. Since that branch returns err == nil,
jobTargetErr (gated on err != nil) and rejectScheduledJobs (gated on
project != nil) were both skipped: `docker compose start migrate`
fell through to a label-driven start that finds no container for a
job that was never run, and exited 0 having silently done nothing.
Refuse a named job explicitly on that fallback path too, reusing
unselectedJobs from the previous commit.
Also adds the test coverage this and the previous fix (job-target
error messages on create/start) were entirely missing: create.go and
start.go had no unit tests at all, and pkg/e2e/jobs_test.go only
exercised up/run.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
materializeJobClosure walked job-typed depends_on edges without
re-checking triggers.manual: false on anything but the top-level run
target, so `docker compose run A` where A depends_on job B (manual:
false) silently ran B anyway.
manual: false declares a job harmful to trigger outside its
schedule; depends_on doesn't change who caused the execution or when
-- pulling B in to satisfy A's dependency is still the run command
causing that out-of-schedule execution, one hop removed. Refuse it
the same way the top-level check already does, before anything in the
closure is created.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Every publish safeguard and image-pinning path used to walk only
project.Services, leaving jobs invisible:
- the sensitive-data checks (literal environment values, env_file
scans, bind-mount warnings, build-only rejection) now cover jobs — a
job declaring AWS_SECRET_ACCESS_KEY=... was published without the
guard rail a service gets;
- the image-digest override pins job images too (jobs dressed as
services run through the exact WithImagesResolved semantics), so the
published artifact is reproducible for jobs as well;
- the application index references job images;
- `config --images` lists them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
compose-go resolves each hook against its service at load time: the
model itself carries the full container specification a hook runs with,
and the runtime consumes it as-is through the standard create path —
the runtime-side merge helper goes away.
Consuming the full spec means honoring ALL of it:
- service references it may carry — volumes_from entries, and
service:-scoped network_mode/ipc/pid, inherited or declared — resolve
to live container IDs exactly like the service create path does; the
daemon knows nothing about service names and rejected them, failing
up for any service combining volumes_from with a pre_start hook
(locked by an e2e scenario).
- hook labels — declared or inherited — merge into the container's
labels, the runtime identification set winning on conflicts.
- the <API 1.44 network-connect fallback joins the HOOK's networks,
not the parent service's: a hook overriding networks was connected
to the wrong ones.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Per the spec, any job can be triggered manually by an explicit run
command, its automated triggers notwithstanding — so run now accepts
scheduled jobs too. The exception is a job explicitly declared with
triggers.manual: false, which run rejects: meant for scheduled jobs
whose out-of-schedule execution would be harmful.
compose-go is bumped to the jobs-branch commit making Manual tri-state
(*bool) and allowing manual and schedule to be combined.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Scenario-DSL coverage for the jobs entry points:
- up refuses a project declaring an active scheduled job before creating
any resource;
- run executes a manual job like a service, starting its depends_on
services first, and refuses a schedule-only job;
- a job's own env_file feeds its environment through run — the
materialization happens before environment resolution;
- a job depending on another job runs the dependency to completion
first, through the exact machinery a service dependency uses.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
pre_start hooks carry the full container specification, but the
init-container runner only consumed a handful of attributes (image,
command, user, env, workdir) — everything else was silently dropped.
Instead of wiring attributes one by one, the hook's specification is
merged over the service's ContainerSpec through compose-go's own file
merge machinery (override.Merge on the canonical yaml tree): command
and entrypoint replace, environment merges per key with the hook
winning, extra_hosts and dns accumulate entries, ulimits merge — the
exact per-attribute rules of multi-file compose, maintained in one
place. Every ContainerSpec attribute inherits this way, current and
future, with zero attribute-specific code.
The merged spec then runs through the standard create path
(getCreateConfigs) as a service-shaped one-off, so resources,
capabilities, dns, sysctls, logging... materialize exactly as they
would for a service container. Hook containers keep their minimal
labels and carry no container-number, so tooling telling replicas
apart does not count them.
The only deliberate exception is volumes: mounts inherit at runtime
through volumes_from — the only mechanism that shares the service's
anonymous and image volumes — and the hook's own volume declarations,
materialized by the standard path, take precedence per target. This is
what lets an init container get read-write access to a volume the
service mounts read-only (fixes: see PR).
e2e scenarios lock extra_hosts inheritance and accumulation, volume
override and completion, and the unit tests pin the merge rules.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
compose-spec/compose-go#866 is merged: jobs top-level element, container
specification layered as ContainerSpec/WorkloadSpec, pre_start hooks
carrying the full container specification resolved at load time. Bump
to the merged head and adapt in the same movement — composite literals
setting moved fields wrap them into the embedded ContainerSpec /
WorkloadSpec (promoted field access was already source-compatible, so
this is literal-only, no behavior change), and pre_start handling is
typed against PreStartHook.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
`compose stop` on a provider service short-circuited straight into
runPlugin(..., "stop"), never touching stopContainers -- so the relay
container this PR gives a provider service (RestartPolicyUnlessStopped,
a real persistent container) was left running and still Up in
`compose ps`, contradicting docs/extension.md's claim that stop
treats the relay like the service.
downService already stops/removes a provider's containers before
invoking the plugin; stop.go gets the same treatment: the service's
containers are stopped unconditionally, then the plugin's own stop
hook (if any) runs for provider services. start/restart needed no
change -- neither special-cases providers, so they already operate on
whatever containers match the service filter, relay included.
Verified locally: reverting just this change makes the new e2e steps
below fail on "stop halts the relay container".
Extends TestProviderPublishEndpoint with stop/start/restart coverage
of the relay container.
(glours review on #14193)
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
down handled a provider service by running the plugin alone — but the
service may own a project container, the relay deployed when it
published endpoints. Left running, it kept the project network in use
and `down -v` failed with "Resource is still in use".
The relay is now part of the service's deprovisioning: its containers
are removed first, then the plugin removes the provider's resource —
mirroring up, which provisions the resource before deploying the relay.
The example provider's down used to answer with a hardcoded error (a
leftover no test relied on): it now succeeds, with the failure
simulation kept behind PROVIDER_DOWN_FAILURE.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A provider's resource lives outside the compose network: consumers could
only reach it through injected variables carrying a host-published
address — nothing like the compose-native experience of addressing a
service by name at its well-known port.
A provider may now publish where each endpoint of its resource actually
listens:
{"type": "publish-endpoint", "message": "80=localhost:49152"}
The endpoint is announced as seen from the provider's host: the relay —
the component that knows it runs inside a container — rewrites loopback
or unspecified upstream hosts to host.docker.internal (resolved through
its injected host-gateway extra_host); routable addresses pass through.
When at least one endpoint is published, compose deploys a relay
container in place of the service: a minimal TCP forwarder (new relay/
directory, published as docker/compose-relay, overridable with
COMPOSE_RELAY_IMAGE for internal registries) joining the networks of the
services that depend on the provider service, aliased with the service
name. Consumers then use http://<service>:<port> as if the service were
a regular container.
The relay is a first-class project container — canonical name, standard
compose labels including config-hash (label-driven commands run without
the compose file keep seeing the service: ps, logs, stop, down) — plus
the com.docker.compose.relay label declaring its role:
- the reconciler already leaves provider services' containers alone, and
the relay's identity hash (image + routes) makes up idempotent: kept
when routes are unchanged, recreated otherwise;
- process-level commands (exec, cp) refuse a relay — there is no service
process in it to act on;
- the up monitor excludes relays from the containers whose termination
ends an attached up: they are long-lived infrastructure and would
otherwise keep 'up' waiting forever.
The example provider demonstrates the flow behind PROVIDER_DEMO_ENDPOINT
(a detached helper serving a fixed HTTP response), backed by an e2e
scenario asserting the compose-native address works and exec is refused.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
git init -b requires git >= 2.28; init followed by symbolic-ref names
the initial branch on any version. Branch() now returns the working
tree to main so successive calls cut from the same base instead of
stacking on the previous branch.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The remote loaders had almost no end-to-end coverage: one test consumed
an OCI artifact (config only, plus an up declined at the prompt) and
nothing exercised the git loader at all.
Git scenarios run against a throwaway repository served over the smart
HTTP protocol by an in-process server — git http-backend as a CGI per
httptest request: no daemon, no container, no fixed port. Smart HTTP is
a hard requirement: the loader ls-remotes the ref then shallow-fetches
the raw commit, which the dumb protocol cannot serve (no shallow
capability) and which needs uploadpack.allowAnySHA1InWant. Covered:
deploying from the default branch with repository-relative files,
selecting a #branch, selecting a #ref:subdir project.
OCI scenarios publish the fixture to a throwaway local registry (the
TestPublish pattern) then deploy from oci://: a full up consuming the
bundled env-file layer, and tag selection between two published
revisions.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
FromRemote switches a scenario's -f to a reference handled by compose's
remote loaders (git URL, oci:// artifact) with optional root flags such
as --insecure-registry, while the anchored testdata copy keeps serving
as the local content the remote is built from. ContainerEnv joins the
check vocabulary: the container-config environment is the natural
observable that a remote project's bundled files (env_file) were
consumed, whatever the source of the model.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
RequireEventuallyServiceState succeeded on the first NDJSON entry for
the service matching the expected state, without checking any
remaining entries. During a recreation window compose ps can list two
containers for the same (scale: 1) service simultaneously (old and
new); the old one could still read "running" while the new one is
"starting", passing the poll prematurely on stale state.
Track whether any entry for the service was seen, and only succeed
once every one of them matches -- fail (poll.Continue) on the first
mismatch instead.
(docker-agent review on #14228)
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
`--format=json` prints one JSON object per line, not a single object:
RequireEventuallyServiceState's json.Unmarshal into a single map broke
as soon as more than one container was listed for the service (a
scaled service, or the transient window during recreation where the
old and new containers are both listed) -- every poll iteration
reported "invalid JSON" and the function always timed out instead of
detecting the actual state.
Scan stdout line by line, matching entries by Service, and succeed as
soon as one reaches the expected state. Also check the exit code
before parsing so a real `compose ps` failure is reported as such
instead of a misleading "invalid JSON" message.
(docker-agent review on #14228)
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
TestAttachRestart and TestUpDependenciesNotStopped each flaked on an
assertion checked exactly once, right after an unrelated signal on a
different channel to the daemon:
- TestAttachRestart counted "failing-1 | world" immediately after a
WaitForCondition on "failing-1 exited with code 1" reached 3. The
exited status comes from the /events stream; the log line comes
from a separate /logs?follow=1 connection (followStartedContainers
in pkg/compose/up.go). Nothing orders one relative to the other, so
the last restart's log line can still be in flight the instant the
3rd exited is observed.
- TestUpDependenciesNotStopped called RequireServiceState right after
seeing "hello app" in the attached log stream. That line reaching
the test is no guarantee the daemon-reported container state (read
via a fresh `compose ps` in a brand new process) has caught up yet.
Neither is a compose bug: `ps`/ContainerList and the log stream are
independent channels with no cross-channel ordering guarantee from
the engine, and compose does not cache container state. Both tests
now poll for the actual condition instead of using a log line as a
proxy for it: TestAttachRestart gets a second, bounded
WaitForCondition on the log count, and TestUpDependenciesNotStopped
uses a new RequireEventuallyServiceState (pkg/e2e/assert.go) that
polls `compose ps` instead of checking it once.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
compose commit --dry-run delegated ContainerCommit to the real client
and actually created the image. It now returns a dry-run ID without
touching the engine.
Auditing the rest of the client surface: 39 more mutating operations
were delegated — none that compose itself issues (verified caller by
caller), but any future code path reaching one would mutate the engine
under --dry-run silently. They now refuse with an explicit error.
The classification — read-only delegated / faked / refused — was only
implied by a method's position under a comment; it is now declared in
dryrunclient_test.go, which parses the client and fails on any method
missing, misclassified, or new and unclassified: extending APIClient
requires an explicit dry-run decision.
The commit e2e scenario gains a --dry-run step asserting the image is
NOT created (new ImageAbsent check).
Epic #14074, B.7.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Unit tests alone didn't cover the exact failure: an attached up
hanging when the project is stopped/removed externally while a
service sits in restart backoff. The new test reproduces that
sequence end to end, so a future regression on this path breaks
loudly instead of slipping through as a passing unit suite. A
RestartCount guard narrows the race with the backoff expiring, so a
missed window fails loudly instead of passing silently through the
pre-existing die-based path.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Compose silently ignored several spec-valid attributes it doesn't
honor outside Swarm mode: deploy.update_config, credential_spec,
ports[].mode: host, and others had no visible effect on a plain
`docker compose up`.
Detection is wired into compose-go's loader as a fail-closed
allowlist: every attribute path the schema declares, minus the
handful this runtime deliberately doesn't implement, plus a few
value-conditional checks for attributes only unsupported for one
specific value. A newly-specified attribute gets flagged
automatically, with no corresponding code change needed here.
Depends on an unreleased compose-go change; go.mod carries a
temporary replace to a fork branch until it's released.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Providers could not see the definition of the service they manage:
options had to be duplicated between the compose file and the provider,
or the provider had to re-resolve the model on its own.
A provider may now emit {"type": "get-service-config"} on stdout;
compose answers on the provider's stdin with one JSON line holding the
resolved canonical configuration of the provider's own service,
straight from the in-memory model. The message can be repeated; each
occurrence is answered with one line. Detection is by construction: a
compose that predates the message aborts on it and never writes to
stdin, so the provider treats EOF as 'unsupported, upgrade compose'.
The example provider demonstrates the round trip, backed by an e2e
scenario; unit tests drive executePlugin against a helper-process
provider and cover the injection.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
`bridge convert -o <dir>` unconditionally ran `os.RemoveAll` on the
output directory and discarded any error, so a typo (`-o .`, `-o
$PWD`, `-o ~`) silently destroyed unrelated user data.
An empty or missing output directory is still created silently, but a
non-empty one now requires explicit confirmation (prompted
interactively, or via the new `--yes` flag for scripts/CI) before
being wiped.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
v0.0.6 dropped the Egress policyType/block from generated
NetworkPolicy manifests since it blocked CoreDNS traffic; update
the fixtures to match.
Signed-off-by: Ricardo Branco <rbranco@suse.de>
v0.0.3 had a mis-published multi-arch manifest, causing exec format
errors on non-amd64 architectures.
Fixed upstream in docker/compose-bridge-transformer@665f67e.
Signed-off-by: Ricardo Branco <rbranco@suse.de>
moby/buildkit#5558, which this test was disabled for, is fixed and
released well before the buildkit v0.26.3 currently vendored here.
Verified the re-enabled case passes (3 consecutive runs).
Signed-off-by: hiroto.toyoda <hiroto.toyoda@dena.com>
The classic-builder rejections said 'set DOCKER_BUILDKIT=1 to use
BuildKit', a leftover from the internal BuildKit builder removed in
af579ebd4. Since then BuildKit builds go through the buildx plugin:
compose falls back to the classic builder when BuildKit is disabled
(DOCKER_BUILDKIT=0) OR when buildx is not installed — so the advertised
fix does nothing for users who are simply missing buildx.
The five messages now state the actual requirement (buildx installed,
DOCKER_BUILDKIT not forced to 0), and the silent-fallback warning when
buildx is missing now says what the fallback implies instead of a bare
'requires buildx plugin to be installed'.
Part of #14074 (A: the code misdescribes its own structure).
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>