The cache of remote resources now has one directory for each kind of resource:
the checkouts of Git repositories are in "git", and the extracted OCI artifacts
are in "oci" rather than at the root of the cache.
What earlier releases left at the root of the cache is no longer looked for: an
artifact is pulled again the first time it is used.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A commit was checked out directly into its directory of the cache, which was
then taken for complete as soon as it existed: a run interrupted during the
fetch or the checkout left a partial directory that every later run used.
The checkout is now made in a directory of its own and renamed to the one of the
commit once it is complete, so the directory of a commit is never a partial
checkout, and no file beside it is needed to tell. Checkouts live in a "git"
directory of the cache, and what earlier releases left at the root of the cache
is no longer looked for.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A run whose replacement of a directory that is not an entry failed put
that directory back unconditionally. Another run having set aside the
entry that took its place in the meantime, the directory put back stood
in the way of the final rename of that run, whose deferred removal then
deleted the complete entry it had set aside: the cache ended with the
incomplete directory after a run returned success.
Only a complete entry is worth restoring, so the restore is guarded.
A run that loses to another may still return an error: the concurrency
test now pins the path of the entry for the runs that succeed, and the
state of the cache once all of them are over.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
When a run replaces a directory of the cache that is not a complete entry, it
sets it aside before removing it, so that it does not remove the entry that
another run has put in place in the meantime. The place it is set aside in
is now a temporary directory of its own, so that two runs replacing the same
entry at the same time never share one.
If the extracted entry can't be moved in place after that, and nothing else
took the place of what was set aside, it is put back instead of being removed
with the rest.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
An entry of the cache of extracted projects was taken as complete as soon as
its directory existed, while the files were written into it one after the
other. A run interrupted in the middle of an extraction, or another run that
loads the same artifact at the same time, left a directory that was then used
as it was, with some of the files missing.
The project is now extracted in a temporary directory of the cache, a
.complete marker is written last, and the directory is moved in place in one
step. A directory of the cache without the marker is not an entry: it is
extracted again, and replaced, never completed. A run that finds the entry
in place after another run uses it.
The entries written by the previous releases have no marker, so each one is
extracted again the first time it is used.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Networks and volumes are created by plan nodes, yet createNetwork and
createVolume still emitted their own events on top of the executor's, so
each line showed twice.
A failing node also reported its error under the plan's internal ID
(service:web:1, network:front), not under the row it had opened. Container
rows stayed on Creating and consumers keying on the ID saw rows nothing
opened.
Make the executor the only emitter and name the row through one helper.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Scope image pruning to selected services and their dependents. Keep
images of declared but unselected services when removing orphans.
Fixes#14453
Signed-off-by: Seyed Noorollah Aghayenoor <noorollah.aghayenoor@gmail.com>
run forces the scale of its target to one, since a container is created
whatever the project says, but the warning looked the service up again in
the project and skipped it when declared with `scale: 0` or
`deploy.replicas: 0`. The per-service check now takes the service
itself, and run hands it the copy it actually creates the container
from. create and up keep skipping services scaled to zero.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
uid, gid and mode on a service-level configs:/secrets: reference are applied
when the object is copied into the container (content: and environment:
sources) and dropped when it is bind-mounted (file: sources). That is a
property of how the container is created, not of the compose file, and an
engine able to set ownership on bind mounts would change the answer.
Report them from create(), next to the other creation-time warnings, once
per requested service, instead of from the load-time unsupported-attribute
check. The false positive on inline and environment sources goes away
without post-filtering findings against the loaded project.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Add TestImagesBackendWiring: the images tests now run against their own fake
backend, so nothing checked that the adapter follows the service's concurrency
bound and the negotiated API version.
Comments only otherwise: say why Backend is narrow rather than embedding
engine.Backend, cross-reference the API version constants declared in both
packages, name down instead of the pkg/compose symbols, point provider_images
to LocalContentDigest, document Summarize, and use the Pruner and PruneOptions
names.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Move the images command, the identity of a local image and the removal of
project images out of pkg/compose into internal/images, behind a small
Backend interface. composeService keeps one-line forwarders so its callers
(build, pull, down, watch) don't change, and ImagePruner, ImagePruneOptions
and NewImagePruner stay exported as aliases.
No behavior change.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The e2e job installed Docker through get-docker.sh, which resolves
packages from the distribution's apt repository. ubuntu-latest moved to
26.04, whose repository has no Docker 28, so the oldstable rows could
no longer install their engine.
Install it with docker/setup-docker-action instead, with binaries that
do not depend on the distribution: stable follows the latest release,
oldstable follows the 28 image tag. The action starts its engine on its
own socket, selected with set-host, so the e2e framework now forwards
DOCKER_HOST, and TestBuildTLS clears it to keep reaching its dind
builder.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
newMonitorTestFixtureListing still returned the unexported monitor type
that moved to internal/logstream, so the compose package tests no longer
compiled. Return *logstream.Monitor like its sibling fixture,
newMonitorTestFixture.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The rebuild forces the recreation of the service it rebuilds and leaves the
providers alone, and the test passed without either option. Give it an up to
date container of the rebuilt service, expected to be removed and created
anyway, and a provider-backed service depending on it, whose provider must not
run.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Merge inspectOf into inspectResultOf (extra healthcheck parameter), add a
container carrying the hook label to the wait fixture the comment talks about,
and drop the executor test map that was written and never read.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The watch rebuild test recorded every touch alike, so a create alone satisfied
it: record the starts apart and require them for the rebuilt service and the
one depending on it. Scale asked for the project's default replica count, which
made applying the requested count invisible: ask for 2 and count the calls.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A relay is also started, and onContainerStart adds a watched container to the
set on its own: send the start event too, so that dropping its IsRelay guard
is caught (a relay restarted after having been created before the monitor
subscribed would be waited for forever).
Also use naming.LabelFilter, which labelFilter became on main.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Close the started channel before the start event is emitted, so that a late
starter stopped in reaction to it always finds the container started, and
close both ends of the pipe the hook logs test hands to the session.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Pin that restart waits for a restart:true dependency to hold its condition
again before restarting its dependent, and that its timeout reaches the
daemon in whole seconds.
Part of #14081.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Extend the interactive-up daemon fake and pin the behaviors mutation checks
of the interactive-up switchover found unlocked: the monitor's Services
scope, a Start-phase error while terminated, the cleanup when attaching
fails, signal handling starting only after the Create phase, late starters
stopped after the teardown sweep, hook logs reaching the session, the wait
timeout translation, --exit-code-from and the Running events. Add the
wiring of detached up (with its --wait verification), scale, the rebuild of
watch and the dependencies of run onto the plan.
Part of #14081.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Pin that every ContainerStart holds startMx, that the start, pre_start,
post_start and hook-runner nodes do nothing once the context is canceled,
that a failure relays untouched down a replica's chain, that the node
bookkeeping records cancellations as failures, that the phase returns the
originating error, that an invalid plan is refused, that a concurrency cap
cannot deadlock the node goroutines, that Starting is reported when the
container starts (not during pre_start), that the hook runner targets the
lowest-numbered replica, and that stop timeouts reach the daemon in whole
seconds.
Part of #14081.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Pin which dependencies are worth a wait (shouldWaitForDependency), how each
container state reads for service_healthy and
service_completed_successfully (required and optional, dry-run), the
Waiting/Healthy rows of a wait outside a plan, and that the end of a wait's
context wins over a ready tick. Add the plan shapes for a wait following its
dependency's start, a project of scale: 0 services, and the Start phase
covering every service of the project (the current behavior of scale, whose
change would be a decision of its own).
Part of #14081.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Pin what compose start does around its plan: the --wait verification (ready
or not, timeout message, one-shot services satisfied by their completion),
the project rebuilt from the project's non-one-off containers when no
compose file is given, only the selected services started, and the replicas
started in numeric order.
Part of #14081.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The monitor ends with the application it follows: pin that its initial
listing and its event subscription are scoped to the project's own
containers (no one-offs, no hook runners without a config hash), and that a
provider's relay, which never terminates, is not waited for, whether it is
there at the start or created later.
Part of #14081.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
logs, attach, the container event monitor they follow and the log
printer move to internal/logstream. The package declares what it needs
from the engine as a small Backend interface; composeService implements
it and keeps one-line forwarders, so `up` and the other callers are
unchanged. Behavior is unchanged: the tests that exercise the moved code
keep their assertions.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
TestStartAlreadyRunning, TestStopAlreadyStopped, TestStartStopMultipleServices
and TestStartStopMultipleFiles now state their intent and observe container
state (running/exited, not recreated) instead of matching progress lines.
The shared fixtures/start-stop had no other user and is removed.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
State what the test locks (a missing env_file only breaks the commands that
need the service referencing it) as a sequence of steps, and observe the
containers (running, scaled, not created) rather than only exit codes.
Completion is covered in both plugin and standalone modes through CompleteCmd.
The fixtures/env_file directory had no other user and is removed.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Express the intent (events refuses a service outside the project) with the
Scenario DSL; the project now lives in testdata/TestEventsUnknownService.
The shared fixtures/simple-composefile is still used by another test.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Shell completion is reached through a different entry point in plugin mode
(docker __complete compose) and in standalone mode (docker-compose
__complete). Expose it as an action so that scenarios can cover completion
in both modes, as the legacy tests did by hand.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A command that is a no-op (e.g. start on an already running service) must
stay silent on stdout. The legacy tests asserted it by hand; expose it as a
generic observable of the scenario vocabulary.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The mode of the injected config file was declared once for the whole
project, so every service that received the credentials held a pointer to
the same value, and changing it for one would have changed it for all of
them. Nothing does that today, but the sharing is easy to trip on.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Align the credentials use_api_socket gives a service with what
`docker run --use-api-socket` does in the Docker CLI:
- the config file is created with mode 0600 instead of the 0444 default
of a config file, so that only the container's root reads it;
- a service which sets DOCKER_CONFIG itself gets the socket but no
credentials, as the comment of the function always said;
- nothing is injected, and DOCKER_CONFIG is not set, when the host has no
registry credentials, and they are not looked up when no service is to
get them.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
dupl reports blocks of syntax that are the same up to a renaming of identifiers and literals, which is the kind of copy this series removed; it does not see semantic equivalents such as reordered operands, which stay a matter for review. The threshold is 80 tokens; tests and the e2e DSL are excluded, and the few remaining pairs (declarative cobra commands, mirrored event and lookup functions) carry a targeted nolint with the reason.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The service depends_on was written to a container label in create.go and read back in compose.go, each side knowing the other's format only by convention. Put the encoder and decoder side by side, with a round-trip test; the label content is unchanged.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The four throttle device lists of the blkio configuration were converted by four identical loops.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
create and createAndStart prepared, announced and executed their plan identically, differing only by scope.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The volume, network and relay link network removals of down each spelled out the same acquire/release of the shared limiter. Wrap them with a helper built on internal/concurrency.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
ReplaceExtendsFile and ReplaceEnvFile started with the same parsing and lookup of the service node. Move it to a helper.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
pause and unpause listed and filtered their containers identically, and down, kill, stop and restart each re-spelled how to fall back to the project rebuilt from the engine. Each now has a single helper.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Several places copied the keys of a map into a slice and sorted it by hand, some through their own generic helper. Use slices.Sorted(maps.Keys(m)), keeping the sorted order of the output.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
pull and push decoded the daemon's JSON message stream with the same loop and computed the completion percentage the same way. Move both to helpers.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The quiet display, pkg/compose and three test files each defined an api.EventProcessor doing nothing. Provide it once as internal/progress.Noop.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
internal/containerops and pkg/compose both had a copy of forEachContainerConcurrent. Provide it once, generic, in internal/concurrency.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
images and volumes each looped over the project containers to keep those of the selected services, as internal/inspect already did for ps and top. Use one helper for all four.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>