An options struct left at its zero value does not always behave like the
matching command run without flags, because the CLI fills in some defaults
before calling the API (Inherit is the sharpest case: its zero value is the
opposite of the CLI default). Index 0 and All also mean different things
depending on the operation.
Describe them in a package comment, and on the fields they concern. No code
change.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The credentials assertion in TestImageDigestResolverUsesDistributionDigest
was vacuous: EncodedAuth encodes even an empty AuthConfig, so
EncodedRegistryAuth != "" always holds. Decode the header and check the
username and password instead.
Build the digest resolver once in generateImageDigestsOverride rather than
once for the services and once for the jobs.
Move the "registry descriptor digest, not the local content digest"
rationale from the pkg/compose wrapper onto registry.ImageDigestResolver,
where the code now lives.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
fakeBackend.ImageDigestResolver in the publish tests was a verbatim copy of
imageDigestResolver in pkg/compose, and Backend.ImageDigestResolver was
redundant with APIClient() + AuthProvider().
Move the resolver to registry.ImageDigestResolver, call it from pull and from
internal/publish, and drop the method from the Backend interface, from
publishBackend and from the fake. The publish digest tests now run on the real
helper. The resolver test moves with it and also pins the caller's context and
a non-empty registry auth, which the publishBackend wiring test used to cover.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Publish already runs inside the "publish" progress bracket, and the public
Push would open a second one. The comment in internal/publish only covers
the other half of that invariant.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The publish tests run against their own fake backend, so nothing checked
the adapter between a composeService and internal/publish. Pin it the way
TestImagesBackendWiring pins the images one: the accessors follow the
service, dry run follows s.dryRun, a refused prompt stays refused, Push
reaches the unexported push with its options (the public Push would open a
second "push" bracket inside the "publish" one), and the image digest
resolver uses the caller's context and the service's credentials.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The e2e framework (framework.go, scenario.go, checks.go, assert.go) and
the e2e tests lived under pkg/e2e, so the framework was published as part
of the public /v5 API although it is only test support code. Move it to
internal/e2e.
This is a pure move: the files keep their content, only the references to
the old path (Makefile, .golangci.yml, codecov.yml, .gitignore, AGENTS.md
and SCENARIO.md) are updated. internal/e2e is at the same depth as
pkg/e2e, so the relative paths to bin/, relay/ and testdata keep working.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Rebuild only forwarded to rebuild, and its sole consumer was the
composeService.rebuild forwarder, itself called by a single test. Keep one
exported Rebuild and let that test reach it through the devwatch service.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The watch loop, its rebuild scheduler and their tests move to
internal/devwatch. The package declares the Backend it expects from the
engine; composeService implements it and keeps one-line forwarders.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The classic and bake builders, with the dispatch between them, become
internal/build, which declares what it needs from the engine as a small
Backend. composeService implements it and keeps a one-line build
forwarder, so Build, watch and the image phase of up and run do not change.
The environment variable resolver, shared with run, becomes its own
internal/envresolver package, and the Building/Built events move to
internal/progress next to the other events.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The documented contract of EventProcessor.Done is success == true when
the operation succeeded, but Run was passing err != nil, so custom
processors received an inverted value. No built-in writer uses the
argument, which is why this went unnoticed.
Pass err == nil, document the meaning of success on the interface, and
add a test covering both the success and the failure case.
Fixes#14498
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Hooks are skipped silently under --dry-run. Log each skipped hook at
debug level so --verbose shows which one would have run.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
WithDryRun re-initializes the Docker CLI with fresh client options,
whose empty log level makes Initialize reset logrus to info. Under
--dry-run, --verbose was therefore dropped as soon as the backend was
created, hiding every debug log of the operation.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The dry-run client cannot attach to an exec, so any lifecycle hook
reached under --dry-run (pre_stop on down/stop/restart, post_start on
up/restart) made the command fail with "interactive exec is not
supported in dry-run mode".
A dry-run must have no side effects, so skip the hook in runHook, which
every hook phase goes through, rather than guarding each call site.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
A Copilot review finding on TestProviderScaleReachesTheProvider: the
scenario left both central guarantees unobserved -- the second `compose
scale` call only checked a successful exit, and no relay container ever
existed to assert singularity against (PROVIDER_DEMO_ENDPOINT was unset).
It would have passed even if a later scale silently lost the requested
value, or if relay creation mishandled it.
The scenario now enables endpoint publication (as in
TestProviderPublishEndpoint) so a real relay container exists, includes
the dependent service in each scale call's targets so it is (re)created
in the same call, and asserts against real state per the project's own
e2e conventions (pkg/e2e/SCENARIO.md: state-based checks over output
matching) -- ContainerEnv for the exact value the provider received,
ServiceScale("db", 1) for the relay staying a single container across
two different scale requests.
Verified: the full provider e2e subset re-run clean against a locally
built binary, golangci-lint clean.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A provider's relay is a singleton network stand-in, never itself a
replica: reconcileService's provider branch always plans exactly one
OpRunProvider node, regardless of the scale requested through
`compose scale` or `scale:`. That value is neither rejected nor acted
on by Compose -- it is already passed through unmodified as part of
the full service config a provider receives over its control channel
(GetServiceConfigType marshals the whole resolved ServiceConfig).
Interpreting it -- scaling the real resource the provider manages,
ignoring it, or rejecting it itself -- is entirely the provider's own
call.
The example provider used by the e2e suite is extended to read that
response into the real types.ServiceConfig (the same struct Compose
marshaled) and call GetScale() on it -- the canonical accessor, which
already reconciles the scale: and deploy.replicas: spellings -- then
surfaces the result as CONFIG_SCALE, exercising the path end to end
the same way it already does for CONFIG_TYPE.
Two regression tests:
- TestReconcileService_ProviderScaleNeverMultipliesTheRelay locks the
plan-level invariant: a declared scale of 3 still produces exactly
one OpRunProvider node.
- TestProviderScaleReachesTheProvider (e2e) locks the value actually
reaching the provider through a real daemon, that `compose scale`
itself converges without error on a provider-backed service, and
that the relay stays a single container across two different scale
requests.
Verified: go build, go test ./pkg/compose/..., and golangci-lint all
clean; the full provider e2e subset re-run clean against a locally
built binary.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A manifest with no media type was taken for an image index when it had a
"manifests" key, even when its value is null. It is now an index only when
that value is a list.
writeCacheEntryManifest reported every failure to create the directory
holding the manifest as an invalid OCI artifact. Only the collision with a
file of the artifact of that name is one; any other failure (missing
directory, permission, full disk) is the host's and is now returned as it is.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The media type a registry reports with a manifest is not authenticated, only
the content is: a registry could serve the authentic bytes of an image index
under the media type of a manifest, which was then read as one, extracted as
nothing and cached as a complete entry that holds no compose file.
The document is now read as what it says it is, and a registry that reports
another type for it is refused. An entry is only put in place once it holds the
compose file that is loaded from it.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
An entry of the cache of extracted projects was taken as complete as soon as
its directory, or a marker file in it, was there. An artifact chooses, through
the annotations of its layers, the files it writes in that directory, so it can
write a marker itself, and so did what releases that did not verify the
manifests they fetched had cached there: neither proves that the directory is
the extraction of the artifact it is named after, or that the extraction ended.
The entries of OCI artifacts, in the oci directory of the cache, now hold the
manifest they were extracted from, as the registry served it (the index when
the reference is an image index), in a .manifest directory of their own. An
artifact names the files of an entry without a path separator, so it can't
write in a directory of it: what is in .manifest was put there by Compose,
which refuses an artifact that has a file of that name. A directory is an entry
for a digest only when it is named after it and holds the manifest that has it,
a check which needs no registry and holds offline. A directory without one, or
with the manifest of another digest, is not an entry.
To make the manifest mean that the entry is complete, the entry is extracted
aside, the manifest is written last, and the directory is moved in place in one
step: nothing is in place which is not complete, so a run never finds an entry
without its manifest. What is in place and is not an entry is set aside and
replaced, never completed, and a run that finds the entry in place after another
run uses it. What earlier releases cached is at the root of the cache, where
nothing looks for it any more.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Get returned the manifest the registry served without checking it against
the digest of the descriptor: the one of the reference when it is digested,
the one the registry announces in Docker-Content-Digest when it is a tag.
Neither is checked by containerd on this path, which doesn't go through its
content store, and GetBlob already checks what it fetches for that reason.
The extracted project cache is keyed by that digest alone, whatever the
registry, so a registry could have its content cached as the one of another
manifest, which a later load of the pinned digest, from the registry that
holds it, then returned.
Get now fails when the manifest doesn't hash to its digest, and shares the
check with GetBlob.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The cache of remote resources now has one directory for each kind of resource:
the checkouts of Git repositories are in "git", and the extracted OCI artifacts
are in "oci" rather than at the root of the cache.
What earlier releases left at the root of the cache is no longer looked for: an
artifact is pulled again the first time it is used.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A commit was checked out directly into its directory of the cache, which was
then taken for complete as soon as it existed: a run interrupted during the
fetch or the checkout left a partial directory that every later run used.
The checkout is now made in a directory of its own and renamed to the one of the
commit once it is complete, so the directory of a commit is never a partial
checkout, and no file beside it is needed to tell. Checkouts live in a "git"
directory of the cache, and what earlier releases left at the root of the cache
is no longer looked for.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A run whose replacement of a directory that is not an entry failed put
that directory back unconditionally. Another run having set aside the
entry that took its place in the meantime, the directory put back stood
in the way of the final rename of that run, whose deferred removal then
deleted the complete entry it had set aside: the cache ended with the
incomplete directory after a run returned success.
Only a complete entry is worth restoring, so the restore is guarded.
A run that loses to another may still return an error: the concurrency
test now pins the path of the entry for the runs that succeed, and the
state of the cache once all of them are over.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
When a run replaces a directory of the cache that is not a complete entry, it
sets it aside before removing it, so that it does not remove the entry that
another run has put in place in the meantime. The place it is set aside in
is now a temporary directory of its own, so that two runs replacing the same
entry at the same time never share one.
If the extracted entry can't be moved in place after that, and nothing else
took the place of what was set aside, it is put back instead of being removed
with the rest.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
An entry of the cache of extracted projects was taken as complete as soon as
its directory existed, while the files were written into it one after the
other. A run interrupted in the middle of an extraction, or another run that
loads the same artifact at the same time, left a directory that was then used
as it was, with some of the files missing.
The project is now extracted in a temporary directory of the cache, a
.complete marker is written last, and the directory is moved in place in one
step. A directory of the cache without the marker is not an entry: it is
extracted again, and replaced, never completed. A run that finds the entry
in place after another run uses it.
The entries written by the previous releases have no marker, so each one is
extracted again the first time it is used.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Networks and volumes are created by plan nodes, yet createNetwork and
createVolume still emitted their own events on top of the executor's, so
each line showed twice.
A failing node also reported its error under the plan's internal ID
(service:web:1, network:front), not under the row it had opened. Container
rows stayed on Creating and consumers keying on the ID saw rows nothing
opened.
Make the executor the only emitter and name the row through one helper.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Scope image pruning to selected services and their dependents. Keep
images of declared but unselected services when removing orphans.
Fixes#14453
Signed-off-by: Seyed Noorollah Aghayenoor <noorollah.aghayenoor@gmail.com>
run forces the scale of its target to one, since a container is created
whatever the project says, but the warning looked the service up again in
the project and skipped it when declared with `scale: 0` or
`deploy.replicas: 0`. The per-service check now takes the service
itself, and run hands it the copy it actually creates the container
from. create and up keep skipping services scaled to zero.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
uid, gid and mode on a service-level configs:/secrets: reference are applied
when the object is copied into the container (content: and environment:
sources) and dropped when it is bind-mounted (file: sources). That is a
property of how the container is created, not of the compose file, and an
engine able to set ownership on bind mounts would change the answer.
Report them from create(), next to the other creation-time warnings, once
per requested service, instead of from the load-time unsupported-attribute
check. The false positive on inline and environment sources goes away
without post-filtering findings against the loaded project.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Add TestImagesBackendWiring: the images tests now run against their own fake
backend, so nothing checked that the adapter follows the service's concurrency
bound and the negotiated API version.
Comments only otherwise: say why Backend is narrow rather than embedding
engine.Backend, cross-reference the API version constants declared in both
packages, name down instead of the pkg/compose symbols, point provider_images
to LocalContentDigest, document Summarize, and use the Pruner and PruneOptions
names.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Move the images command, the identity of a local image and the removal of
project images out of pkg/compose into internal/images, behind a small
Backend interface. composeService keeps one-line forwarders so its callers
(build, pull, down, watch) don't change, and ImagePruner, ImagePruneOptions
and NewImagePruner stay exported as aliases.
No behavior change.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The e2e job installed Docker through get-docker.sh, which resolves
packages from the distribution's apt repository. ubuntu-latest moved to
26.04, whose repository has no Docker 28, so the oldstable rows could
no longer install their engine.
Install it with docker/setup-docker-action instead, with binaries that
do not depend on the distribution: stable follows the latest release,
oldstable follows the 28 image tag. The action starts its engine on its
own socket, selected with set-host, so the e2e framework now forwards
DOCKER_HOST, and TestBuildTLS clears it to keep reaching its dind
builder.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
newMonitorTestFixtureListing still returned the unexported monitor type
that moved to internal/logstream, so the compose package tests no longer
compiled. Return *logstream.Monitor like its sibling fixture,
newMonitorTestFixture.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The rebuild forces the recreation of the service it rebuilds and leaves the
providers alone, and the test passed without either option. Give it an up to
date container of the rebuilt service, expected to be removed and created
anyway, and a provider-backed service depending on it, whose provider must not
run.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Merge inspectOf into inspectResultOf (extra healthcheck parameter), add a
container carrying the hook label to the wait fixture the comment talks about,
and drop the executor test map that was written and never read.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The watch rebuild test recorded every touch alike, so a create alone satisfied
it: record the starts apart and require them for the rebuilt service and the
one depending on it. Scale asked for the project's default replica count, which
made applying the requested count invisible: ask for 2 and count the calls.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A relay is also started, and onContainerStart adds a watched container to the
set on its own: send the start event too, so that dropping its IsRelay guard
is caught (a relay restarted after having been created before the monitor
subscribed would be waited for forever).
Also use naming.LabelFilter, which labelFilter became on main.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Close the started channel before the start event is emitted, so that a late
starter stopped in reaction to it always finds the container started, and
close both ends of the pipe the hook logs test hands to the session.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Pin that restart waits for a restart:true dependency to hold its condition
again before restarting its dependent, and that its timeout reaches the
daemon in whole seconds.
Part of #14081.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Extend the interactive-up daemon fake and pin the behaviors mutation checks
of the interactive-up switchover found unlocked: the monitor's Services
scope, a Start-phase error while terminated, the cleanup when attaching
fails, signal handling starting only after the Create phase, late starters
stopped after the teardown sweep, hook logs reaching the session, the wait
timeout translation, --exit-code-from and the Running events. Add the
wiring of detached up (with its --wait verification), scale, the rebuild of
watch and the dependencies of run onto the plan.
Part of #14081.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Pin that every ContainerStart holds startMx, that the start, pre_start,
post_start and hook-runner nodes do nothing once the context is canceled,
that a failure relays untouched down a replica's chain, that the node
bookkeeping records cancellations as failures, that the phase returns the
originating error, that an invalid plan is refused, that a concurrency cap
cannot deadlock the node goroutines, that Starting is reported when the
container starts (not during pre_start), that the hook runner targets the
lowest-numbered replica, and that stop timeouts reach the daemon in whole
seconds.
Part of #14081.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>