When a service declares an env var without a value (e.g. `- KEY` or
`KEY:`), MappingWithEquals stores it as a nil *string. The previous
condition `existing != nil && ...` skipped the warning for this case,
allowing silent overwrites. Change to `existing == nil || ...` so the
warning fires for both nil (shell-inherit) and value-mismatch cases.
Add e2e tests for both list-style (`- KEY`) and map-style (`KEY:`)
YAML forms to lock in the behavior.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Yohta Kimura <38206553+rajyan@users.noreply.github.com>
rawsetenv injects provider variables without the service-name prefix, so
a key can collide with a value already set on the dependent service,
whether declared by the user in environment or emitted by another
provider. Log a warning and overwrite on collision, document the
precedence and the non-deterministic ordering between concurrent
providers, and cover the user-environment override with an e2e test.
Signed-off-by: Yohta Kimura <38206553+rajyan@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Providers can now send rawsetenv messages to inject environment
variables into dependent services without the automatic service name
prefix. This enables use cases where applications require exact
variable names that cannot be altered.
Closes#13727
Signed-off-by: Yohta Kimura <38206553+rajyan@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The run parameter was always passed as false at the single call site
and the run==true branch was dead code. Remove it so unparam stops
flagging callers added by PR #13742.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
This fixture was not a valid JWT; the first 2 elements decode, but the last
one is malformed;
echo 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9' | base64 -d
{"alg":"HS256","typ":"JWT"}⏎
echo 'eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ' | base64 -d
{"sub":"1234567890","name":"John Doe","iat":1516239022⏎
echo 'SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw' | base64 -d
I�J�IHNJ(]�O��lj~�:N�%_�u
,⏎
This causes problems if the JWT parser is strict and rejecting invalid
JWT's.
It was added in 55b5f233c2, and probably copied
from an example, like https://github.com/knottx/JWTCodable#example-jwt-token,
but the last 2 bytes were truncated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
`compose up --build` populates BuildOptions.Deps=true so the initial
startup also builds images for depends_on services. The watch rebuild
path reused the same BuildOptions pointer, only resetting Build.Services
to the watched service. Build.Deps stayed true, so s.build() switched
back to IncludeDependencies and rebuilt the upstream dependency too.
Fix it by working on a local copy of BuildOptions in rebuild() and
explicitly setting Deps=false. Using a local copy also removes the data
race on the shared pointer when concurrent file events fire.
Also fix a related leak in doBuildBake: the loop populating bake
configuration iterates over every service in the project (needed so
additional_contexts: service:xxx references can resolve), but it was
emitting the "Image X Building" progress event and tracking expected
images for services that were not part of serviceToBeBuild. Filter
those side-effects to the actual build set so the watch rebuild log
shows only the watched service.
Adds an e2e test reproducing the bug.
Fixes#13853
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
External reviewer noted that the alreadyStopped branch adds createNode
to removeDeps while the !alreadyStopped branch does not — semantically
correct but fragile, since it relies on the implicit invariant that
stopNode.DependsOn contains createNode in the !alreadyStopped path.
Spell out the invariant in a comment so a future maintainer who edits
the stop → create edge in the normal path knows they must also add
createNode unconditionally in the remove deps.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Two coverage gaps surfaced by an external review:
- TestReconcileContainers_DependsOnChain: asserts that a service B with
depends_on: [A] produces a CreateContainer for B that depends on A's
last plan node (the serviceNodes mechanism in infrastructureDeps).
This was the only depends_on-via-plan-DAG behavior untested before.
- TestReconcileContainers_DependsOnScaleDown: companion test that
exercises the scale-down → dependent path specifically, verifying
that the previous commit's lastNode-on-scale-down fix actually wires
the dependency through.
- TestOperationTypeString: adds OpRunProvider to the table; all other
OperationType values were already covered.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Three fixes surfaced by an external code review of the new reconciler:
1. Scale-down now propagates through serviceNodes. When a service is
scaled down (all containers in excess), reconcileService used to
continue without assigning lastNode, leaving r.serviceNodes[svc]
unset. Dependent services then declared no edge on the scale-down
ops and could start before the cleanup finished. Track the
RemoveContainer node as lastNode so depends_on chains pick it up.
2. mustRecreate errors are no longer silently ignored by sortContainers.
The comparator used `obsi, _ := r.mustRecreate(...)`, falling back
to false on any hashing error. Pre-compute obsolescence into a map
keyed by container ID before sorting and propagate the error to
reconcileService.
3. A container is no longer Stopped twice when its network and its
config both diverge. planRecreateNetwork already stops the affected
container as part of the disconnect/remove/recreate dance; the
subsequent planRecreateContainer (triggered via hasNetworkMismatch)
used to add another OpStopContainer against the now-stopped target.
Track stops in r.stoppedByPlan; planRecreateContainer reuses an
existing Stop node when present, and chains its Remove on both that
Stop and the replacement Create.
Two golden tests (TestReconcileNetworks_Diverged*) are updated to
reflect the new, dedupe'd plan shape (one Stop instead of two per
recreated container).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Three improvements identified in the Principal Engineer pass but
deliberately deferred:
1. Test fidelity. Split executePlan into newPlanExecutor (constructs
the executor seeded from observed state) and (*planExecutor).run
(walks the DAG). Production callers go through executePlan
unchanged. TestExecutePlanRemoveContainerDropsFromCache now uses
newPlanExecutor + run, exercising the same errgroup, done-channel
and group-tracker wiring as production instead of a hand-rolled
loop over executeNode.
2. //nolint:unused chain. The three preserved helpers
(reconciler.prompt, planRecreateVolume, servicesUsingVolume) each
carried a separate "kept for future" comment. Consolidate the
rationale on the reconciler.prompt field doc and point the helper
nolint directives there, so a future cleanup is a single grep.
3. Concurrency test. Add TestExecutePlanConcurrentRemovesCacheCoherence
which builds N independent Stop→Remove chains in one plan; the
errgroup fans them out across goroutines that all hit
containersByService under the mutex. Passes under -race. Failure
would expose a missing or incorrect lock.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Two related changes to the executor, plus the small cleanups they
attracted in review:
* Rename node consults a planner-set CreateNodeID instead of walking
ancestors. The old execRenameContainer searched node.DependsOn for a
CreateContainer result and fell back to a recursive walk through the
group chain. That worked only by convention; a future op with cross-
node data needs would have to rediscover or copy the pattern. Now the
rename op carries an explicit CreateNodeID int set at plan time and
the executor reads pctx[CreateNodeID].ContainerID directly. The
recursive findCreatedIDInChain is gone.
* Stop re-listing containers on every create. execCreateContainer used
to call getContainersByService(ctx, projectName) — a fresh
ContainerList per create — to resolve service references at execute
time. The executor now holds a live containersByService view seeded
from ObservedState (via observed.containersByService()) and grown as
OpCreateContainer nodes complete, so service references resolve from
memory. On OpRemoveContainer the removed container is dropped from
the view via slices.DeleteFunc, so a dependent's create that resolves
network_mode: service:x against the just-removed container cannot
pick up a stale ID (Containers.sorted() orders by canonical name and
would otherwise return the removed container).
* Defensive slices.Clone of op.Service.VolumesFrom in execCreateContainer.
resolveServiceReferences mutates VolumesFrom in place, and the
shallow struct copy of *op.Service still shares the backing array.
Single-execution-per-node makes it safe today, but the clone removes
the trap for any future parallel-execution mode.
* Operation gains a CreateNodeID int (not a *PlanNode pointer) to avoid
a structural cycle between Operation and PlanNode. OperationType
values are pinned to explicit integers so adding an op in the middle
cannot silently shift the others.
* execRenameContainer carries two checks so a missing CreateNodeID and
an empty produced ID are distinguishable in logs. Both are programmer
invariants (prefixed "internal:").
* containersByServiceFromObserved moved from a package-level helper to
a method on *ObservedState.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
ensureProjectVolumes already prompts the user when a volume's config hash
diverges from the compose file (create.go:1626) and recreates it on confirm.
The reconciler ran after ensureProjectVolumes and prompted again with the
exact same message — so a user who declined the first prompt was asked the
same question a second time.
Drop the prompt + recreate call from reconcileVolumes(). Recreation of
diverged volumes stays owned by ensureProjectVolumes; the reconciler only
plans the creation of missing volumes. If the user declined recreation,
the existing container's mounts still match the existing volume name and
hasVolumeMismatch correctly returns false, so containers are not falsely
flagged as obsolete.
Keep the supporting infrastructure available for future use, when
divergence detection migrates fully into the reconciler:
- reconciler.prompt field
- prompt parameter on reconcile()
- planRecreateVolume function (//nolint:unused)
- servicesUsingVolume function (//nolint:unused)
- noPrompt test helper
The reframed test (TestReconcileVolumes_DivergedIsIgnored) asserts the
new contract: a diverged volume produces no plan operations from the
reconciler.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
- plan.go: pin OperationType constants to explicit values so adding an op
in the middle doesn't shift the others.
- executor.go: remove the meaningless `var _ = getContainerProgressName`
line — same-package functions are always accessible.
- reconcile.go: fix the stale switch-default comment that contradicted
the case clause above it.
- reconcile.go: drop the local `serviceLabel` const that shadowed
`api.ServiceLabel` and use the shared constant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
- Fix ContainerReplaceLabel detection: use op.Inherited != nil (not
op.Container) as signal for recreate in execCreateContainer
- Use observed network name (not desired) for DisconnectNetwork and
RemoveNetwork operations, in case the name changed
- Use observed volume name (not desired) for RemoveVolume operations
- Update reconciliation.md with 3 new lessons learned (7.8, 7.9, 7.10)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
- Remove the `convergence` struct and `newConvergence` constructor
- Extract `resolveServiceReferences` as a standalone function taking
`map[string]Containers` instead of a method on convergence
- Add `getContainersByService` helper on composeService
- Update run.go and executor.go to use the new standalone function
- Remove dead code: `getObservedState`, `setObservedState`
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
When a service's build context is a remote git/HTTP URL, the path was
unconditionally added to bake's --allow fs.read= entitlements. On Windows,
bake then tries to evaluate the URL as a local filesystem path and fails
because `https:` is invalid path syntax (colon is reserved for drive
letters).
Apply the same gitutil.ParseGitRef + "://" check already used for
additional_contexts so that remote contexts are skipped from the fs.read
allowlist.
Fixes#13815
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Nicolas De loof <nicolas.deloof@gmail.com>
Replace the "flag any literal env var" check with a key-name heuristic
backed by the upstream DefangLabs keyword detector (password, secret,
token, api_key, …), and convert the hard error into a prompt matching
the existing checkForBindMount / checkForSensitiveData UX. --with-env
silences the env prompt; literal config.content gets its own prompt.
The previous check flagged benign vars like LOG_LEVEL=info, blocking
the 99% case, while still missing low-entropy real secrets the
existing secret-detector skips (MYSQL_ROOT_PASSWORD=toto slips through
on entropy ~1.5).
Refs: docker/compose#13394
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Docker Desktop is removing the "Enable Logs view" beta setting, so drop
the /app/settings check and rely on /features alone. With the setting
gate gone, the compose hook subprocess would print the Logs view hint
regardless of LogsTab; add a flag check in handleHook. Consolidate
engine-label discovery and feature-flag evaluation into internal/desktop.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The stop lifecycle hook added in 672dc14d2 calls stoppingEvent and
stoppedEvent from pkg/compose/plugins.go, but the helpers had been
removed by the earlier wrapper-cleanup in da530c723. The result was a
broken go build for pkg/compose on upstream/main.
Restore the two helpers alongside the symmetric creating/created and
removing/removed pairs so call sites in plugins.go match the existing
pattern used in the same switch.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
A previous change extended IsEmpty() to include `Stop == nil`. For a
provider that advertises only a `stop` block, this caused
commandMetadataIsEmpty to be false in setupPluginCommand, which made
the option-forwarding filter reject every key — silently dropping
all provider options on `up` and `down`.
The Stop-presence signal lives independently in stop.go and the
"stop" case of setupPluginCommand, so reverting the IsEmpty check
to its original semantics is sufficient.
Addresses PR #13779 review feedback.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The previous implementation fetched the provider binary path and
metadata twice per service during `compose stop`: once in stop.go
to gate on the `stop` capability, and again inside runPlugin via
setupPluginCommand.
setupPluginCommand now signals "skip" by returning (nil, nil) when
the requested command is absent from the provider's metadata.
stop.go calls runPlugin directly; the skip-when-unadvertised check
moves into runPlugin.
Addresses PR #13779 review feedback.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Provider-backed services were silently skipped on `docker compose stop`,
leaving external resources running after the user expected the stack to
be paused (e.g. after Ctrl+C during `up --watch`).
Compose now invokes `<provider> compose stop <service>` for providers
that advertise a `stop` block in their `metadata` subcommand output.
Providers that do not advertise stop (or do not implement metadata at
all) are silently skipped, preserving backward compatibility with
existing providers that pre-date this hook.
Closes#13772
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
- Fix stale image/container reuse across test runs - Add registry readiness check and async removal polling
- Skip multi-arch test when docker driver supports it
- Use t.Cleanup for reliable teardown, fix project name mismatches
- Re-enable 4 previously skipped tests that now pass
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The compose process performs OCI artifact fetches in-process via
containerd's docker resolver, whose default transport only honors
HTTP_PROXY/HTTPS_PROXY/NO_PROXY env vars. Users behind PAC-only
corporate proxies hit i/o timeouts on `oci://` includes and on
`compose publish`.
When Docker Desktop is the active engine and exposes httpproxy.sock,
route the resolver through it (PAC-aware). Falls back to the default
transport when DD is unavailable or the socket is missing. Modeled on
docker/mcp-gateway PR #354.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Provider info and error messages containing newlines broke the TTY
progress display (timer drifting to a new line, broken cursor
movement). Extract only the first line for progress events via
firstLine(). Full messages remain available through the provider's
own debug message type.
Skip provider services during watch rebuild convergence by adding a
SkipProviders flag to CreateOptions, set only by the watch rebuild
path. This prevents unnecessary re-invocation of providers on every
file change while preserving normal provider execution for all other
commands (up, create, run, scale).
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Add CLI hooks handler to show "What's next:" hints pointing to the
Docker Desktop Logs view after `docker logs`, `docker compose logs`,
and `docker compose up -d`.
Add `l` keyboard shortcut in the `compose up` navigation menu to
open the Logs view, gated on Docker Desktop feature flag and settings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
