logs, attach, the container event monitor they follow and the log
printer move to internal/logstream. The package declares what it needs
from the engine as a small Backend interface; composeService implements
it and keeps one-line forwarders, so `up` and the other callers are
unchanged. Behavior is unchanged: the tests that exercise the moved code
keep their assertions.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
forbidigo rejects context.Background() in tests; t.Context() is the
intended replacement and carries the test's cancellation.
Signed-off-by: Zhuoming Wang <zhuomingwang0901@outlook.com>
The Engine client sends io.EOF over EventsResult.Err once the stream has
been read completely, which is what happens when `events --until` reaches
its deadline. Service.Events returned that error verbatim, so the command
printed "EOF" and exited 1 even though every event had been delivered.
Return nil for io.EOF and keep propagating real stream errors.
Signed-off-by: Zhuoming Wang <zhuomingwang0901@outlook.com>
DescriptorForComposeFile and DescriptorForEnvFile only differed by their media type and annotation key.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Ping and FeatureFlags issued the same GET and decoded the same way. Share the request, status check and decoding in get[T].
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Several places copied the keys of a map into a slice and sorted it by hand, some through their own generic helper. Use slices.Sorted(maps.Keys(m)), keeping the sorted order of the output.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The quiet display, pkg/compose and three test files each defined an api.EventProcessor doing nothing. Provide it once as internal/progress.Noop.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
internal/containerops and pkg/compose both had a copy of forEachContainerConcurrent. Provide it once, generic, in internal/concurrency.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
images and volumes each looped over the project containers to keep those of the selected services, as internal/inspect already did for ps and top. Use one helper for all four.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
internal/inspect and internal/containerops declared the same engine methods and one-off enum, and pkg/compose adapted them twice. Declare them in internal/engine, embed them in both Backend interfaces and keep a single adapter.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
internal/projectloader and pkg/compose each rebuilt a service definition from the containers of a project with the same bootstrap and replica counting. Keep it in internal/projectloader and call it from both.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The label=key=value filter was rebuilt in pkg/compose, internal/inspect, internal/relay, cmd/compose and pkg/bridge. Build it once in internal/naming and use it everywhere.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
pkg/compose kept private copies of the canonical, default and project-less container name helpers that internal/naming already provides. Use the shared ones, and host the project-prefix stripping that compose.go and monitor.go each implemented.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
pkg/compose carried its own copy of the event constructors that internal/progress already provides. Use the shared ones and move the generic waiting, healthy, exited and skipped events there too.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The "errgroup bounded to maxConcurrency, where <= 0 means unlimited" helper was copied in several places (pkg/compose, internal/containerops, internal/inspect, internal/sync), together with the semaphore-based slot helpers used to share one budget across concurrent calls. Gather them in a leaf package so the extractions in progress can import it instead of duplicating it again. Behavior is unchanged.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
ServiceHash, NetworkHash and VolumeHash only serve Compose's own change
detection (the config-hash labels stamped on containers, networks and
volumes); they have no business being part of the exported API of
pkg/compose. Move them, with their tests, to internal/confighash.
The hashes are byte-for-byte unchanged (the golden values test moves with
the code, and `compose config --hash` prints the same output before and
after). The only dependency on the package they came from, a sorted-keys
helper, is replaced by slices.Sorted(maps.Keys(...)).
This removes exported symbols from the /v5 module: compose.ServiceHash,
compose.NetworkHash and compose.VolumeHash.
Part of #14396
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Two comments still located waitDependency's runtime fallback in
service_containers.go.
Comments only.
Part of #14081
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Cover the container selection and the refusals of Copy through a fake
Backend: the replica asked for, one-off containers only with --all, a copy
from a service reading its first container only, a service without
container, copying between services and a copy with no service at all.
Part of #14396
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
These four operations act from the outside on the container of a service.
They only need the Docker client, the concurrency bound, the event bus, the
dry-run switch, the Docker CLI (which owns the terminal streams) and the two
container lookups, so they move out of the flat pkg/compose package into
internal/containerops behind a Backend interface declaring exactly that.
composeService implements the interface in a new file and keeps Exec, Copy,
Export and Commit as one-line forwarders (the progress wrapper of the last
three stays there), so no caller changes. The now unused checkRelayTarget
forwarder is dropped. Behavior is unchanged: the cp test still runs through
the forwarders.
Part of #14396
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Unescape the remaining path in both branches and cover the
ports[].mode reporting path with a dotted service name.
Signed-off-by: huang <13165998033@163.com>
compose-go escapes a literal dot in a path segment as an internal
sentinel, and splitServicePath returned the raw segment. A service named
"my.service" was therefore reported as "my👻service" in warnings.
Unescape the segment with tree.Path(...).String().
Signed-off-by: huang <13165998033@163.com>
Docker Desktop's docker-auth store fills in the access-token claims, expiry included, from the JWT when it saves a Hub credential, and when it reads one from the legacy credential helper. secrets-engine/client v0.2.2 rejects tokens that are not JWTs. So compose no longer needs to parse exp out of the token itself. A session without an expiry claim is still reused for sessionTTL.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
The error message produced by containerd changed in [containerd@7f0c0b8];
"reading blob sha256:536319e163bc3cc681d5ce1a5c88aab6e4afb4b7315ce62acfca619ad36717ee: httpReadSeeker: read past expected size 35: failed precondition
[containerd@7f0c0b8]: 7f0c0b822e
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Running a provider plugin (finding its binary, executing it, serving the
JSON messages it sends, applying the variables and endpoints it publishes,
streaming it an image) needs from the engine the Docker API, the event bus,
the Docker CLI that locates the plugin, the preparation of the environment
of a shelled-out command, and the lock that guards the project model every
concurrent provider run mutates.
It now lives in internal/provider behind a Backend interface that
composeService implements in pkg/compose/plugins.go, next to runPlugin, which
stays as a one-line forwarder so none of its callers (up, down, stop, pull)
changes. The relay it deploys goes through internal/relay, whose Backend the
new one extends. The public types and constants of the protocol (JsonMessage,
ProviderMetadata, the *Type message constants...) stay exported from
pkg/compose as aliases of the new package's.
The tests of the plugin protocol move with the code, assertions unchanged;
those of the provider image phase and of the relay link network removal
stay in pkg/compose. The forwarders to the relay that only the plugin code
used are dropped.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The relay container and the relay link networks that stand in for a
provider service need three things from the engine: the Docker API, the
event bus and whether Docker Desktop is the engine.
They now live in internal/relay behind a Backend interface that
composeService implements in pkg/compose/relay.go, which keeps the names
the lifecycle code calls (ensureServiceRelay, isRelayContainer,
checkRelayTarget...) as one-line forwarders, so none of its callers change.
The one down-specific piece, turning the relay link networks into down
operations gated on the shared limiter, stays in pkg/compose together with
its tests; the other relay tests move with the code, assertions unchanged.
The progress events and the container name the package needs come from two
small shared packages, internal/progress and internal/naming, which the
following moves reuse. pkg/compose keeps its own copies until the shared
kernel is extracted.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
LoadProject and Generate (the api.ProjectLoader operations) need two things
from the engine: the Docker API, to read the containers a project is
generated from, and the Docker CLI, which resolves remote Compose files.
They now live in internal/projectloader, along with the unsupported
attributes detection only the loader uses, behind a Backend interface that
composeService implements in a new file of pkg/compose, like internal/inspect.
The tests move with the code; their assertions are unchanged, only the way
the service is built differs.
The canonical container name, which both this package and internal/inspect
need, moves to a tiny internal/naming package instead of being copied a
third time. pkg/compose keeps its own copy until the shared kernel is
extracted.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
ps, ls, top, port, volumes, events and viz only read the state of a
project: they need the Docker API, the list of the project's containers
and the concurrency bound, nothing else of composeService.
They now live in internal/inspect, which declares that need as a Backend
interface. composeService implements it in a new file of pkg/compose and
forwards the seven api.Inspector methods, so no other file of pkg/compose
changes, and neither does the behavior: the service-level tests stay in
pkg/compose and run unchanged through the forwarders.
The few tiny helpers the operations share with the rest of pkg/compose
(canonical container name, concurrency-limited errgroup, project filter)
are copied for now: the shared kernel they belong to is a later step.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The client replaced ErrSecretsEngineNotAvailable with a typed ConnectError. Any failure to reach the secrets engine now falls back to the Docker CLI credentials and is logged at debug level.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
Docker Desktop stores Docker Hardened Images credentials under https://dhi.io, so return that as the server address of dhi.io credentials, as StagingIndexServer does for the staging registry.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
Docker Hardened Images on dhi.io authenticate with Docker Hub accounts (the registry advertises service=registry.docker.io), so resolve dhi.io credentials from the production Docker Hub session as well, with dhi.io as the server address. Other registries, including look-alikes of dhi.io, still use the Docker CLI credentials.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
Rename isExpectedMiss to isSessionUnavailable, after what it reports, and shorten or drop doc comments in the Docker Desktop credentials provider.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
A lookup that found no usable session disabled Desktop credentials for the rest of the process. That is right within one command, where retrying a hung engine would add the lookup timeout to every image, but the sessions are process wide and pkg/compose is also used as a library: in a long-running process a transient timeout, or signing in to Desktop later, was never picked up. Hold lookups off for a minute instead of for good.
Also name the expected-miss classification and test it, including dockerhub.ErrNoDefaultProfile, which wraps dockerhub.ErrNoSession.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
It covered the resolver credential mapping and containerd grant selection, neither changed by this PR. The token-as-password contract is asserted on the provider output and on the encoded X-Registry-Auth in internal/registry.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
Shorten the doc comments added for the Docker Desktop credentials provider to what the code does not already say, and drop them from the tests.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
Docker Desktop keeps the Docker Hub staging session in separate secrets engine realms, read with dockerhub.Staging(). The environment is picked per registry host rather than with a global stage flag: the engine resolves docker.io to the production registry even when Desktop runs in stage mode, so the staging session is only used for registry-1-stage.docker.io and the production session only for the production Hub hosts. A token is never sent to the registry of the other environment.
Each environment has its own process-wide session and cache, and falls back to the Docker CLI config file independently. Staging credentials use https://registry-1-stage.docker.io/ as server address.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
The session payload may carry no claims, in which case the access token was reused for a minute without any expiry check, so an expired token could reach the registry and fail the pull instead of falling back. Read the exp claim from the JWT itself in that case. An expiry that cannot be read is still tolerated.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
Docker Hub credentials are now read first from the session of the account signed in to Docker Desktop, through the Docker Desktop secrets engine (github.com/docker/secrets-engine/client), and from the Docker CLI config file and its credential helpers otherwise. Other registries keep using the config file only.
The OAuth access token is passed as the password of the signed-in account, never as an IdentityToken, which containerd sends as a refresh token. It flows unchanged to the engine in X-Registry-Auth and through the OCI resolver as an OAuth password grant.
Compose falls back to the config file when Desktop is not running, nobody is signed in, the session is expired or about to expire, or the lookup fails (the last logged as a warning). The session is shared process wide, cached until shortly before it expires, and an unusable session is not asked for again. Lookups are bounded by a 5s timeout, as the SDK applies none by default.
Used by pull, push, publish, config --resolve-image-digests, oci:// remote resources and dry-run. The classic builder and use_api_socket keep config file credentials, as a short-lived token copied into a container would go stale.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
down's outer ops dispatch bounded only how many ops ran concurrently,
separate from the semaphore already shared by image removal -- a
single network plus --rmi could spend up to 2x the configured
budget. Network and volume removal now share that same semaphore.
up's post-attach log streaming called ContainerInspect directly and
passed no limiter to doLogContainer, unlike `compose logs --follow`'s
equivalent path, so it never shared --parallel's budget either.
The plan executor also held a node's slot for its whole dependency
wait instead of just its own work, and internal/sync/tar.go's file
sync used a hardcoded cap of 16 instead of --parallel.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
compose-spec/compose-go#866 is merged: jobs top-level element, container
specification layered as ContainerSpec/WorkloadSpec, pre_start hooks
carrying the full container specification resolved at load time. Bump
to the merged head and adapt in the same movement — composite literals
setting moved fields wrap them into the embedded ContainerSpec /
WorkloadSpec (promoted field access was already source-compatible, so
this is literal-only, no behavior change), and pre_start handling is
typed against PreStartHook.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
OTel SDK/exporter errors already go through docker/cli's own
otel.SetErrorHandler(debug.OTELErrorHandler), installed by the plugin
framework at startup. Our own handler, installed at package init time,
was always clobbered before any command ran (plugin.Run() sets its handler
after all inits execute), so it never had any effect - verified by tracing
the exact call order and reproducing the flag behavior against a built
binary.
Drop that dead handler along with the COMPOSE_OTEL_DEBUG env var it
introduced. The one genuinely-discarded error (the tracer shutdown/flush
error in wrapRunE) now goes through logrus like everything else in the
codebase: it's already gated by --debug/-D via logrus's own level, so no
explicit check is needed.
Signed-off-by: hiroto.toyoda <hiroto.toyoda@dena.com>
EncodedAuth now delegates the X-Registry-Auth wire encoding to
moby/api/pkg/authconfig (same module, already required) instead of
hand-rolling the JSON+base64url pair, and push.go reuses it instead of
its own copy. The CLI and moby AuthConfig structs are field-for-field
identical, so a direct type conversion bridges them.
The explicit GetAuthConfigKey normalization drops out of EncodedAuth:
docker/cli's configfile.GetAuthConfig normalizes the Docker Hub domain
itself. It remains for the OCI resolver, whose credential callback
receives the network host actually contacted (registry-1.docker.io),
which the configfile does not map.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
buildx was imported for two narrow purposes, both replaced by
docker/cli machinery compose already uses elsewhere:
- storeutil.GetProxyConfig -> configfile.ParseProxyConfig, the same
lookup docker build uses (also picks up ALL_PROXY and lowercase
variants)
- the dry-run image resolver (buildx builder + imagetools) -> a
DistributionInspect through the daemon with credentials from the CLI
config file, i.e. the same path a real pull/push would take; the
encoded-auth helper moves to internal/registry, shared with pull.go
bake is unaffected: it already execs the buildx CLI plugin. Dropping
the module also drops the transitive aws-sdk-go-v2 footprint
(-328 lines of go.sum).
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Pulling an oci:// resource resolved each layer digest through the
registry manifests endpoint, which answers 500 when the digest points
to a non-manifest blob. containerd v2.3.0+ (pulled in by buildx v0.36
and buildkit v0.32) no longer falls back to the blobs endpoint unless
manifests returned 404, so publish/pull of compose artifacts broke.
Fetch layers directly with the descriptors already listed in the
manifest instead of resolving them again.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>