up and create report each scheduled job like any other resource: created,
or running when an identical job is already registered. A dry run used
to say nothing about jobs, and run <job> returned without any output; it
now starts the dependencies through the dry-run client and reports the
run, without reaching the engine's jobs.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Engines before 29.10 have no jobs extension, so the jobs scenarios that
need it were always skipped. Add a leg on 29.10.0-rc.1 from the test
channel with the extension enabled through daemon.json; the stable legs
keep covering an engine without it.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
`docker compose run <job>` triggers a run of the job on the engine,
starts its dependencies first, streams its logs and returns its exit
code like a one-off service; a job accepts manual triggering unless it
declares manual: false. A scheduled job runs as up registered it: an
override, or a job up never registered, is refused. A manual job is
removed from the engine after its run, so the next run can differ. When
the engine cannot run jobs, run falls back to a one-off service with a
warning. --detach is refused, an interrupted run is cancelled, a dry run
only reports it. A job named by create, start, build, pull or push gets
an error naming run instead.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
A job with a schedule is registered through the engine's jobs extension,
which fires it on its own clock; create and up register it, up stays
re-runnable since Create is idempotent on the spec hash, and a full down
removes the jobs. The extension ships disabled by default and cannot be
listed, so Compose probes it with a List scoped to the project; when the
engine cannot run jobs, up, create and start refuse the project before
creating anything, and down ignores the jobs. A job the file no longer
declares is reported by up and removed by --remove-orphans, unless its
run is in progress. The spec is stable: sorted entries, no volatile labels,
services referenced by container name.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Describe the engine's jobs extension (org.mobyproject.jobs.v0) as a Go
interface with its request and reply types, the generated gRPC client,
and the mapping of its status codes to errdefs errors. moby/extensions
is pinned to the version the engine vendors. Nothing uses it yet.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Once every attached container is gone the monitor ends the session, and
with it the event loop that handles Ctrl+C: later interruptions are lost.
During --wait that left the stack running on exit 130: end the wait
without canceling, and stop the stack from there when the loop is gone.
During the start phase, blocked on dependencies of unattached services,
the signal was swallowed and the command hung: keep the session open.
Add tests for both and document that --wait leaves the services up.
Keep an api test valid now that Wait is accepted together with Attach.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Without --detach, `up --wait` now attaches to the containers and streams
their logs until the services are running|healthy, then ends the
session and leaves them up. Until now --wait silently forced --detach,
so a service failing to become healthy left no clue about why.
--detach --wait keeps the previous silent behavior, and --dry-run
stays on the detached plan. The library Validate accepts Wait with
Attach, and still rejects OnExit, ExitCodeFrom and Watch with it. The
navigation menu is disabled while waiting.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A pre_start hook container that fails to start (an executable missing from
the image of the hook, a bad mount, ...) was reported with the bare error of
the engine. It names neither the hook nor its image, and it surfaces on the row
of the service container, so it reads as the service failing to start.
Wrap it with the service, the index of the hook and its image, like the other
pre_start errors: service "app" pre_start[0] (image hello-world): failed to
start the hook container: <engine error>. The wrapped error is unchanged for
errors.Is and errors.As.
Refs #14513
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A stopped container still refers to the networks it was attached to. When
one of them was removed, up created the network again but started the old
container, and the engine refused: "network <id> not found".
The network check of mustRecreate only ran for running containers, so a
stopped one was never compared with the network of the project. It now runs
whatever the state of the container, like the check of the volumes.
A network that no longer exists is not in the observed state, so a container
attached to a network of that name refers to the removed one and is
recreated. A container that was created but never started has no network ID
yet: it is left alone, rather than taken for one attached to another network.
Fixes#14516
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
compose-go v2.17.0 resolves them in WithImagesResolved. Running the
workaround on top reparsed the digested reference and dropped its tag.
The job digest of its test was two characters short, which the native
resolution now rejects: use a valid one.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
It requires go >= 1.26.11, already used by .go-version and the
Dockerfile, so the go directive moves from 1.26.10 to 1.26.11.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Several exported symbols describe a contract Compose never honors, and a
reader of the package cannot tell: the stack status constants (Stack.Status is
never one of them), ConfigOptions and ServiceStatus (no operation takes or
returns them), the error sentinels ErrAlreadyExists, ErrForbidden, ErrUnknown
and ErrUnsupportedFlag (never returned, with their Is helpers), and the
ExitCodeLoginRequired exit code of the cloud login Compose no longer has.
Mark them Deprecated, with no change of behavior, so that they can be
removed with the next breaking release. Nothing in the repository uses them.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Exec, port, export, commit and attach fail with "service X is not running"
when they find no container to act on. A program driving Compose as a library
could only recognise that failure by matching the message.
Return a ServiceNotRunningError, which matches the new ErrNotRunning and
carries the service and the replica index. The message is unchanged.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
CreateOptions.Recreate and RecreateDependencies are strings, and any value
other than force or never silently behaved like diverged: a typo such as
"forced" was accepted and did the opposite of what the caller meant.
Add CreateOptions.Validate, which rejects such values with an
InvalidOptionError matching the new ErrInvalidOption, and call it from
Create and, through UpOptions.Validate, from Up, before anything is
touched. The empty string keeps meaning diverged, and the Recreate*
constants are unchanged, so existing callers are not affected.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
An options struct left at its zero value does not always behave like the
matching command run without flags, because the CLI fills in some defaults
before calling the API (Inherit is the sharpest case: its zero value is the
opposite of the CLI default). Index 0 and All also mean different things
depending on the operation.
Describe them in a package comment, and on the fields they concern. No code
change.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The credentials assertion in TestImageDigestResolverUsesDistributionDigest
was vacuous: EncodedAuth encodes even an empty AuthConfig, so
EncodedRegistryAuth != "" always holds. Decode the header and check the
username and password instead.
Build the digest resolver once in generateImageDigestsOverride rather than
once for the services and once for the jobs.
Move the "registry descriptor digest, not the local content digest"
rationale from the pkg/compose wrapper onto registry.ImageDigestResolver,
where the code now lives.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
fakeBackend.ImageDigestResolver in the publish tests was a verbatim copy of
imageDigestResolver in pkg/compose, and Backend.ImageDigestResolver was
redundant with APIClient() + AuthProvider().
Move the resolver to registry.ImageDigestResolver, call it from pull and from
internal/publish, and drop the method from the Backend interface, from
publishBackend and from the fake. The publish digest tests now run on the real
helper. The resolver test moves with it and also pins the caller's context and
a non-empty registry auth, which the publishBackend wiring test used to cover.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Publish already runs inside the "publish" progress bracket, and the public
Push would open a second one. The comment in internal/publish only covers
the other half of that invariant.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The publish tests run against their own fake backend, so nothing checked
the adapter between a composeService and internal/publish. Pin it the way
TestImagesBackendWiring pins the images one: the accessors follow the
service, dry run follows s.dryRun, a refused prompt stays refused, Push
reaches the unexported push with its options (the public Push would open a
second "push" bracket inside the "publish" one), and the image digest
resolver uses the caller's context and the service's credentials.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
COMPOSE_REMOVE_ORPHANS and COMPOSE_IGNORE_ORPHANS resolve through
different sources by design, not by accident: the destructive variable
goes through the process environment, completed from the local .env
only — a remote model cannot enable container removal — while the
benign one reads project.Environment, remote configs included, where
the worst a remote model can do is suppress a warning. Stated once, on
the resolveIgnoreOrphans helper shared by up, create and run, so future
reviewers don't have to rediscover the decision. The helper is covered
by a test, and another one runs the real up/down/kill/run/create
commands to pin that each of them resolves COMPOSE_REMOVE_ORPHANS.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
run resolved both COMPOSE_REMOVE_ORPHANS and COMPOSE_IGNORE_ORPHANS
but was the only command accepting the contradictory combination
silently; it now rejects it exactly like up and create.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The variable had three resolution mechanisms: up read it in PreRunE
after the project's .env was injected into the process environment;
down and kill read it at cobra-tree construction time — before that
injection, so a value declared in the project's .env worked for up but
not for them; create and run never read it at all.
All five commands now resolve it through one helper, from PreRunE:
explicit flag wins, then the process environment as completed by
setEnvWithDotEnv from the local .env. create also gains the
COMPOSE_IGNORE_ORPHANS handling (its ignoreOrphans field was sent to
the API without ever being assigned) with the same conflict guard as
up.
setEnvWithDotEnv's remote-config exclusion is now documented as a
product decision, not a technical constraint: COMPOSE_* variables
exist so a local user doesn't repeat flags — they are the local user's
choice, and a remote model must not steer the CLI consuming it.
Unit tests pin the .env → process env → flag resolution order and the
remote exclusion (epic #14074, D.1 and G.3).
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
AGENTS.md covered build/test/lint mechanics but nothing about how the
codebase is organized, so a reader's first mental model comes from file
names and local comments — several of which used to be misleading (see
the previous commits in this series).
The new section states, in ~35 lines, the facts a contributor cannot
derive from local reading: the two coexisting lifecycle engines and
which commands enter the plan-based reconciler versus the dependency
graph walk, the Run()/EventProcessor operation convention, the
stdout/stderr split, the two environment-resolution idioms and the
setEnvWithDotEnv timing, the config-hash label as the actual project
membership invariant, and the Docker Desktop integration points that
must fail silently.
Part of #14074 (A: the code misdescribes its own structure).
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Move to the final v29.10.0 release. github.com/moby/moby/api v1.57.0
and github.com/moby/moby/client v0.6.3 are already the final releases
of the matching moby version, so they stay as they are.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The e2e framework (framework.go, scenario.go, checks.go, assert.go) and
the e2e tests lived under pkg/e2e, so the framework was published as part
of the public /v5 API although it is only test support code. Move it to
internal/e2e.
This is a pure move: the files keep their content, only the references to
the old path (Makefile, .golangci.yml, codecov.yml, .gitignore, AGENTS.md
and SCENARIO.md) are updated. internal/e2e is at the same depth as
pkg/e2e, so the relative paths to bin/, relay/ and testdata keep working.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Rebuild only forwarded to rebuild, and its sole consumer was the
composeService.rebuild forwarder, itself called by a single test. Keep one
exported Rebuild and let that test reach it through the devwatch service.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The createAndStart and removeDanglingImages fakes run on the rebuild's goroutine, where a failed assertion would only end that goroutine instead of failing the test cleanly. They now record what they observe and the checks run once the rebuild has settled.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The watch loop, its rebuild scheduler and their tests move to
internal/devwatch. The package declares the Backend it expects from the
engine; composeService implements it and keeps one-line forwarders.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The classic and bake builders, with the dispatch between them, become
internal/build, which declares what it needs from the engine as a small
Backend. composeService implements it and keeps a one-line build
forwarder, so Build, watch and the image phase of up and run do not change.
The environment variable resolver, shared with run, becomes its own
internal/envresolver package, and the Building/Built events move to
internal/progress next to the other events.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The documented contract of EventProcessor.Done is success == true when
the operation succeeded, but Run was passing err != nil, so custom
processors received an inverted value. No built-in writer uses the
argument, which is why this went unnoticed.
Pass err == nil, document the meaning of success on the interface, and
add a test covering both the success and the failure case.
Fixes#14498
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Hooks are skipped silently under --dry-run. Log each skipped hook at
debug level so --verbose shows which one would have run.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
WithDryRun re-initializes the Docker CLI with fresh client options,
whose empty log level makes Initialize reset logrus to info. Under
--dry-run, --verbose was therefore dropped as soon as the backend was
created, hiding every debug log of the operation.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The dry-run client cannot attach to an exec, so any lifecycle hook
reached under --dry-run (pre_stop on down/stop/restart, post_start on
up/restart) made the command fail with "interactive exec is not
supported in dry-run mode".
A dry-run must have no side effects, so skip the hook in runHook, which
every hook phase goes through, rather than guarding each call site.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
A Copilot review finding on TestProviderScaleReachesTheProvider: the
scenario left both central guarantees unobserved -- the second `compose
scale` call only checked a successful exit, and no relay container ever
existed to assert singularity against (PROVIDER_DEMO_ENDPOINT was unset).
It would have passed even if a later scale silently lost the requested
value, or if relay creation mishandled it.
The scenario now enables endpoint publication (as in
TestProviderPublishEndpoint) so a real relay container exists, includes
the dependent service in each scale call's targets so it is (re)created
in the same call, and asserts against real state per the project's own
e2e conventions (pkg/e2e/SCENARIO.md: state-based checks over output
matching) -- ContainerEnv for the exact value the provider received,
ServiceScale("db", 1) for the relay staying a single container across
two different scale requests.
Verified: the full provider e2e subset re-run clean against a locally
built binary, golangci-lint clean.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A provider's relay is a singleton network stand-in, never itself a
replica: reconcileService's provider branch always plans exactly one
OpRunProvider node, regardless of the scale requested through
`compose scale` or `scale:`. That value is neither rejected nor acted
on by Compose -- it is already passed through unmodified as part of
the full service config a provider receives over its control channel
(GetServiceConfigType marshals the whole resolved ServiceConfig).
Interpreting it -- scaling the real resource the provider manages,
ignoring it, or rejecting it itself -- is entirely the provider's own
call.
The example provider used by the e2e suite is extended to read that
response into the real types.ServiceConfig (the same struct Compose
marshaled) and call GetScale() on it -- the canonical accessor, which
already reconciles the scale: and deploy.replicas: spellings -- then
surfaces the result as CONFIG_SCALE, exercising the path end to end
the same way it already does for CONFIG_TYPE.
Two regression tests:
- TestReconcileService_ProviderScaleNeverMultipliesTheRelay locks the
plan-level invariant: a declared scale of 3 still produces exactly
one OpRunProvider node.
- TestProviderScaleReachesTheProvider (e2e) locks the value actually
reaching the provider through a real daemon, that `compose scale`
itself converges without error on a provider-backed service, and
that the relay stays a single container across two different scale
requests.
Verified: go build, go test ./pkg/compose/..., and golangci-lint all
clean; the full provider e2e subset re-run clean against a locally
built binary.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A provider service declaring neither image: nor build: has no image
reference to list, yet config --images printed the <project>-<service>
name GetImageNameOrDefault fabricates: a reference that is never built
and never pulled, while the image phase of up and pull already skips
such services. Consumers feeding `compose config --images` output to
a registry would try to pull an image that does not exist.
Mirror the pure-resource provider guard of pkg/compose/provider_images.go.
Signed-off-by: Aucoto <tdnorthgarden@gmail.com>
The relay image ran its Go toolchain stage on the target platform, so every
platform it ships for needed a golang:alpine image of its own. The
golang:1.27.4-alpine tag has no linux/riscv64 variant (1.26 had one), which
fails the riscv64 leg of relay-image and relay-image-test on main since the
move to Go 1.27.
Build it the way the compose binary is built: the toolchain runs on the build
platform, from the same pinned golang alpine image, and xx cross-compiles for
the target and checks that the result is static. No platform depends on what
the golang image publishes any more.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>