A command that is a no-op (e.g. start on an already running service) must
stay silent on stdout. The legacy tests asserted it by hand; expose it as a
generic observable of the scenario vocabulary.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The mode of the injected config file was declared once for the whole
project, so every service that received the credentials held a pointer to
the same value, and changing it for one would have changed it for all of
them. Nothing does that today, but the sharing is easy to trip on.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Align the credentials use_api_socket gives a service with what
`docker run --use-api-socket` does in the Docker CLI:
- the config file is created with mode 0600 instead of the 0444 default
of a config file, so that only the container's root reads it;
- a service which sets DOCKER_CONFIG itself gets the socket but no
credentials, as the comment of the function always said;
- nothing is injected, and DOCKER_CONFIG is not set, when the host has no
registry credentials, and they are not looked up when no service is to
get them.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
dupl reports blocks of syntax that are the same up to a renaming of identifiers and literals, which is the kind of copy this series removed; it does not see semantic equivalents such as reordered operands, which stay a matter for review. The threshold is 80 tokens; tests and the e2e DSL are excluded, and the few remaining pairs (declarative cobra commands, mirrored event and lookup functions) carry a targeted nolint with the reason.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The service depends_on was written to a container label in create.go and read back in compose.go, each side knowing the other's format only by convention. Put the encoder and decoder side by side, with a round-trip test; the label content is unchanged.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The four throttle device lists of the blkio configuration were converted by four identical loops.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
create and createAndStart prepared, announced and executed their plan identically, differing only by scope.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The volume, network and relay link network removals of down each spelled out the same acquire/release of the shared limiter. Wrap them with a helper built on internal/concurrency.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
ReplaceExtendsFile and ReplaceEnvFile started with the same parsing and lookup of the service node. Move it to a helper.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
pause and unpause listed and filtered their containers identically, and down, kill, stop and restart each re-spelled how to fall back to the project rebuilt from the engine. Each now has a single helper.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Several places copied the keys of a map into a slice and sorted it by hand, some through their own generic helper. Use slices.Sorted(maps.Keys(m)), keeping the sorted order of the output.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
pull and push decoded the daemon's JSON message stream with the same loop and computed the completion percentage the same way. Move both to helpers.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The quiet display, pkg/compose and three test files each defined an api.EventProcessor doing nothing. Provide it once as internal/progress.Noop.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
internal/containerops and pkg/compose both had a copy of forEachContainerConcurrent. Provide it once, generic, in internal/concurrency.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
images and volumes each looped over the project containers to keep those of the selected services, as internal/inspect already did for ps and top. Use one helper for all four.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
internal/inspect and internal/containerops declared the same engine methods and one-off enum, and pkg/compose adapted them twice. Declare them in internal/engine, embed them in both Backend interfaces and keep a single adapter.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
internal/projectloader and pkg/compose each rebuilt a service definition from the containers of a project with the same bootstrap and replica counting. Keep it in internal/projectloader and call it from both.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The label=key=value filter was rebuilt in pkg/compose, internal/inspect, internal/relay, cmd/compose and pkg/bridge. Build it once in internal/naming and use it everywhere.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
pkg/compose kept private copies of the canonical, default and project-less container name helpers that internal/naming already provides. Use the shared ones, and host the project-prefix stripping that compose.go and monitor.go each implemented.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
pkg/compose carried its own copy of the event constructors that internal/progress already provides. Use the shared ones and move the generic waiting, healthy, exited and skipped events there too.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The "errgroup bounded to maxConcurrency, where <= 0 means unlimited" helper was copied in several places (pkg/compose, internal/containerops, internal/inspect, internal/sync), together with the semaphore-based slot helpers used to share one budget across concurrent calls. Gather them in a leaf package so the extractions in progress can import it instead of duplicating it again. Behavior is unchanged.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
ServiceHash, NetworkHash and VolumeHash only serve Compose's own change
detection (the config-hash labels stamped on containers, networks and
volumes); they have no business being part of the exported API of
pkg/compose. Move them, with their tests, to internal/confighash.
The hashes are byte-for-byte unchanged (the golden values test moves with
the code, and `compose config --hash` prints the same output before and
after). The only dependency on the package they came from, a sorted-keys
helper, is replaced by slices.Sorted(maps.Keys(...)).
This removes exported symbols from the /v5 module: compose.ServiceHash,
compose.NetworkHash and compose.VolumeHash.
Part of #14396
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The release updates errcheck (1.20 to 1.30) and gocognit (1.2.1 to 1.3.0).
errcheck now reports the receives from an error channel whose value is dropped
on purpose, in three places: waiting for the attach copy to end after the
context is done, draining the container wait channel of a failed hook, and
waiting for Up to return in a test. Say why on each, rather than handling an
error nobody can act on. gocognit's new count is handled by the previous
commit.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The header of service_containers.go still listed the dependency waits among
what it covers.
Comments only.
Part of #14081
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
waitDependencies, the polling of a depends_on condition (waitDependency,
waitDependencyReached and the per-condition checks), shouldWaitForDependency
and the container health probes they use leave service_containers.go for
dependency_wait.go, and their tests for dependency_wait_test.go. The
remaining clients of the waits are the plan's OpWaitCondition nodes,
restart, run and the --wait verification of up and start.
A pure move: every declaration, its comments included, is unchanged.
Part of #14081
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Say that a relay restarted by a start plan does take startMx while the
deployment of the relay does not, that the lock was unlikely (rather than
unable) to help against #12846, and name isReplicaRelay, which is what
gates pre_start. Rewrap the test comment.
Comments only.
Part of #14081
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
docker/compose#12846 was not a race between container starts but a flaw of
the daemon's port allocator, fixed in Engine 28.3.0: the mutex added as its
workaround could not help, and the comment said the opposite. Say what the
mutex does and does not provide, and that the sequential start of a
service's replicas (#8530) is the plan's replica chain, not this mutex.
Also fix the comment of the test of a stale non-relay container of a
provider-backed service, which still described a special case the start
path no longer has: planServiceStart routes every provider to
planProviderStart, which decides bare-vs-enriched per container.
Comments only: the token stream of every Go file, comments excluded, is
unchanged.
Part of #14081
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Review follow-ups of the removal of waitDependencies' timeout parameter:
the unsupported-condition test lost the 2s bound the parameter gave it, so
a regression that waits would hang instead of failing; and the test
comparing a timeout that expires during the inspection with one that
expires between polls now also compares whether they are a deadline.
Also reword the comment of execWaitCondition, which still said that
waitDependencies threads a timeout through its waits.
Part of #14081
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Now that waitDependencies has no timeout parameter, the plan's wait nodes
are the only ones to wrap the context of a wait: tell so in the comment of
waitDependency, which compares the wrapped context with the original.
Comments only.
Part of #14081
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Its four callers (up --wait, start --wait, run, restart) all pass 0, so the
wrapper it built around the context was dead: a wait only gets a timeout of
its own through the plan's OpWaitCondition nodes, which bound it with
exec.waitTimeout in execBoundedWaitCondition and never go through
waitDependencies. A deadline on the caller's context, which is how up and
start --wait bound their wait, is untouched.
The tests that passed a timeout now exercise the same semantics through
the live paths: a deadline on the caller's context for waitDependencies,
and execBoundedWaitCondition with exec.waitTimeout for the wait's own
timeout (tolerated on an optional dependency, an error on a required one).
Part of #14081
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
startMx works around duplicate published host ports on Engine 28.0 to
28.2 (docker/compose#12846), a daemon port-allocator flaw fixed in Engine
28.3.0 (moby/moby#50054). Say so in its comment, state which start paths
take it, and add a TODO to remove it once Engine 28.x is out of support.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Follow-up of the two previous commits, from their review: rewrap the
comments they left with an orphan line, give two of them back a subject,
point one at the plan's wait nodes, and drop the "imperative engine"
wording of the restart tests, restart being the one command that still
walks the dependency order itself.
Comments only: the token stream of every Go file, comments excluded, is
unchanged.
Part of #14081
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Same cleanup as the previous commit, for the comments of the unit tests:
they cite startService, isNotRunning and the "old imperative engine" as
the source of a behavior the plan now owns.
Comments only: the token stream of every Go file, comments excluded, is
unchanged.
Part of #14081
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
startService and startServiceContainer were removed by the migration of
compose start to the plan engine, yet many comments still cite them, or
isNotRunning, or describe the start as walked by InDependencyOrder.
Rewrite them to state the behavior itself, and fix the ones that say
which commands still walk the dependency order.
Comments only: the token stream of every Go file, comments excluded, is
unchanged.
Part of #14081
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
These four operations act from the outside on the container of a service.
They only need the Docker client, the concurrency bound, the event bus, the
dry-run switch, the Docker CLI (which owns the terminal streams) and the two
container lookups, so they move out of the flat pkg/compose package into
internal/containerops behind a Backend interface declaring exactly that.
composeService implements the interface in a new file and keeps Exec, Copy,
Export and Commit as one-line forwarders (the progress wrapper of the last
three stays there), so no caller changes. The now unused checkRelayTarget
forwarder is dropped. Behavior is unchanged: the cp test still runs through
the forwarders.
Part of #14396
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Epic #14081 moved the start phase into the plan engine. Since #14302 and
#14305, a caller that streams pre_start/post_start hook logs into an attached
session hands its listener to planRun.runStart when it enters the Start phase,
because its printer only exists once the Create phase is done. As a result
every production caller builds the executor with a nil listener, and only a
few tests still passed one at construction; runStart kept a "nil keeps the
construction-time listener" branch just for them.
Drop the listener parameter of newPlanExecutor and make runStart the only way
to give the executor a listener (nil means hook logs are not streamed). The
doc comments that described a construction-time listener now describe that
single path. Production behavior is unchanged.
In tests, the one that fed a listener at construction sets it on the executor
it exercises, the two that only pinned "construction-time listener kept or
replaced" are removed, and a test pins that a nil listener leaves hooks
running without one.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
- a project that already went through use_api_socket, as the one `run` hands
to the dependencies it starts, is injected once by start(): rewriting it
again would duplicate the config entry, and the file would be copied twice.
The first test also checks that the option was applied;
- the comments say which maps the rewriting may find nil: the service
Environment in the project `compose start` loads, the Configs of a
programmatic one, and that a project rebuilt from the labels knows nothing of
the option.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The project `compose start` loads has no environment nor configs map, which
the rewriting used to write into; a project built with empty maps is the
other shape. Both are started with the config file in place, and neither
caller's project is written to.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A container created beforehand and only started by `compose start`
(`create` then `start`, `up --no-start` then `start`, or a dependency named
to `start`) came up with DOCKER_CONFIG pointing at /run/secrets/docker and no
file there: use_api_socket is applied by rewriting the project, and the
config file it adds is injected when a container starts, from the project the
start works on. Create and run applied the rewriting, start did not.
start() now applies it too, on the copy it already makes of the project, so
the caller's is left alone, and a project already rewritten (a dependency
started by run) is not rewritten twice. The rewriting tolerates a project
without environment or configs maps, as the one `compose start` loads.
up -d, a foreground up, scale and run already provided the file.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
- the plan's own project, not the caller's, is what the session, the Start
phase, the watcher and the teardown work on: a test checks that the Create
phase hands it over, with use_api_socket's config only the plan's project
has. It was found by mutation to survive the other tests;
- the interactive up's doc no longer over-claims: an interruption is only
honored while the session attaches or reads the daemon's time, and one
arriving later does not stop the Start phase;
- readDaemonTime's doc says the error is the client's, which carries the
context's;
- two test comments say what they rely on: that only the monitor uses the
events API, and that the fake engine reports an interrupted call.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Tests:
- the monitor goroutine runs before the Start phase: a fake container start
holds until the events subscription is opened, so a Start phase run before
it would show;
- an interruption met by the Create phase ends the up with it and sets up no
session: the phase runs on the caller's context.
Both were found by mutation to survive the existing tests.
Comments and scenarios:
- useAPISocket's guard says what a second pass really does: duplicate volume
and config entries that also change the service's config hash;
- the interactive up's doc says an early interruption starts nothing, the
readDaemonTime's that callers decide how to report an interruption, and a
test comment on the listener no longer announces what is already wired;
- the use_api_socket scenario links docker/compose#14386, as SCENARIO.md asks
of a regression test, and the hook-runner scenario drops an output assertion
its ExitCode already covers.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
UpOptions.Validate now refuses the combination, so the field's doc says that
instead of describing a foreground Up that silently skips the verification.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Production:
- detached and interactive up prepare their plan the same way: prepareUp
holds the preparation and the --wait-timeout rationale both share, so the
two cannot diverge;
- useAPISocket leaves alone a project it already transformed. The session,
the watcher and the teardown now work on the plan's own project, which a
watch rebuild hands to a later plan: transforming it again mounted the
socket and the config file twice;
- the docs say what the code does: an interruption met while reading the
daemon's time is returned wrapping the context's error (the CLI maps it to
its interruption exit code), Up ignores StartOptions.Project and does not
verify StartOptions.Wait in the foreground, and the Start phase's listener
goes through runStart.
Tests:
- the Start phase is shown to survive the caller's cancellation, and an
interruption before it to start nothing;
- the dry-run test no longer claims to exercise the dry-run client: it pins
the gate of Up;
- the phase order test also checks the "up" scope opens before the create;
- useAPISocket's idempotence is tested.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A pre_start runner carries the project and service labels but not the one-off
label the monitor's event filter requires, so its exit never reaches the
session. Nothing locked that: a runner exiting successfully would otherwise
win over the service's own exit code under --exit-code-from, or abort the
application under --abort-on-container-exit.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Interactive up used to create() the project, then start() it: two
daemon snapshots, the second built from the project as the caller gave
it rather than the canonical one the Create phase ran on.
It now prepares one Create+Start plan, like detached up. The Create
phase runs under the "up" progress scope, then the attach/printer/
monitor session is set up exactly as before, and the Start phase is
dispatched last, outside that scope, on a context that survives
Ctrl+C, with the printer as the hook-log listener. A failed Create
phase reaches neither the session nor the Start phase, and a dry run
only ever runs the Create phase.
The monitor's starting point is the daemon's time, which is read right
before the Start phase: a daemon that answers with a time that can't be
used fails the up with an explicit error, with no container started. One
that can't be asked at all (a read-only socket proxy without the info
endpoint) doesn't: the monitor then follows live events, with a warning.
The session, the watcher and the teardown now use the plan's own
project, which carries what preparation resolved. That makes
use_api_socket work in a foreground up: its config file lives only in
that project. A scenario locks it.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>