A manifest with no media type was taken for an image index when it had a
"manifests" key, even when its value is null. It is now an index only when
that value is a list.
writeCacheEntryManifest reported every failure to create the directory
holding the manifest as an invalid OCI artifact. Only the collision with a
file of the artifact of that name is one; any other failure (missing
directory, permission, full disk) is the host's and is now returned as it is.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The media type a registry reports with a manifest is not authenticated, only
the content is: a registry could serve the authentic bytes of an image index
under the media type of a manifest, which was then read as one, extracted as
nothing and cached as a complete entry that holds no compose file.
The document is now read as what it says it is, and a registry that reports
another type for it is refused. An entry is only put in place once it holds the
compose file that is loaded from it.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
An entry of the cache of extracted projects was taken as complete as soon as
its directory, or a marker file in it, was there. An artifact chooses, through
the annotations of its layers, the files it writes in that directory, so it can
write a marker itself, and so did what releases that did not verify the
manifests they fetched had cached there: neither proves that the directory is
the extraction of the artifact it is named after, or that the extraction ended.
The entries of OCI artifacts, in the oci directory of the cache, now hold the
manifest they were extracted from, as the registry served it (the index when
the reference is an image index), in a .manifest directory of their own. An
artifact names the files of an entry without a path separator, so it can't
write in a directory of it: what is in .manifest was put there by Compose,
which refuses an artifact that has a file of that name. A directory is an entry
for a digest only when it is named after it and holds the manifest that has it,
a check which needs no registry and holds offline. A directory without one, or
with the manifest of another digest, is not an entry.
To make the manifest mean that the entry is complete, the entry is extracted
aside, the manifest is written last, and the directory is moved in place in one
step: nothing is in place which is not complete, so a run never finds an entry
without its manifest. What is in place and is not an entry is set aside and
replaced, never completed, and a run that finds the entry in place after another
run uses it. What earlier releases cached is at the root of the cache, where
nothing looks for it any more.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Get returned the manifest the registry served without checking it against
the digest of the descriptor: the one of the reference when it is digested,
the one the registry announces in Docker-Content-Digest when it is a tag.
Neither is checked by containerd on this path, which doesn't go through its
content store, and GetBlob already checks what it fetches for that reason.
The extracted project cache is keyed by that digest alone, whatever the
registry, so a registry could have its content cached as the one of another
manifest, which a later load of the pinned digest, from the registry that
holds it, then returned.
Get now fails when the manifest doesn't hash to its digest, and shares the
check with GetBlob.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The cache of remote resources now has one directory for each kind of resource:
the checkouts of Git repositories are in "git", and the extracted OCI artifacts
are in "oci" rather than at the root of the cache.
What earlier releases left at the root of the cache is no longer looked for: an
artifact is pulled again the first time it is used.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A commit was checked out directly into its directory of the cache, which was
then taken for complete as soon as it existed: a run interrupted during the
fetch or the checkout left a partial directory that every later run used.
The checkout is now made in a directory of its own and renamed to the one of the
commit once it is complete, so the directory of a commit is never a partial
checkout, and no file beside it is needed to tell. Checkouts live in a "git"
directory of the cache, and what earlier releases left at the root of the cache
is no longer looked for.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
A run whose replacement of a directory that is not an entry failed put
that directory back unconditionally. Another run having set aside the
entry that took its place in the meantime, the directory put back stood
in the way of the final rename of that run, whose deferred removal then
deleted the complete entry it had set aside: the cache ended with the
incomplete directory after a run returned success.
Only a complete entry is worth restoring, so the restore is guarded.
A run that loses to another may still return an error: the concurrency
test now pins the path of the entry for the runs that succeed, and the
state of the cache once all of them are over.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
When a run replaces a directory of the cache that is not a complete entry, it
sets it aside before removing it, so that it does not remove the entry that
another run has put in place in the meantime. The place it is set aside in
is now a temporary directory of its own, so that two runs replacing the same
entry at the same time never share one.
If the extracted entry can't be moved in place after that, and nothing else
took the place of what was set aside, it is put back instead of being removed
with the rest.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
An entry of the cache of extracted projects was taken as complete as soon as
its directory existed, while the files were written into it one after the
other. A run interrupted in the middle of an extraction, or another run that
loads the same artifact at the same time, left a directory that was then used
as it was, with some of the files missing.
The project is now extracted in a temporary directory of the cache, a
.complete marker is written last, and the directory is moved in place in one
step. A directory of the cache without the marker is not an entry: it is
extracted again, and replaced, never completed. A run that finds the entry
in place after another run uses it.
The entries written by the previous releases have no marker, so each one is
extracted again the first time it is used.
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Shorten the doc comments added for the Docker Desktop credentials provider to what the code does not already say, and drop them from the tests.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
Docker Hub credentials are now read first from the session of the account signed in to Docker Desktop, through the Docker Desktop secrets engine (github.com/docker/secrets-engine/client), and from the Docker CLI config file and its credential helpers otherwise. Other registries keep using the config file only.
The OAuth access token is passed as the password of the signed-in account, never as an IdentityToken, which containerd sends as a refresh token. It flows unchanged to the engine in X-Registry-Auth and through the OCI resolver as an OAuth password grant.
Compose falls back to the config file when Desktop is not running, nobody is signed in, the session is expired or about to expire, or the lookup fails (the last logged as a warning). The session is shared process wide, cached until shortly before it expires, and an unusable session is not asked for again. Lookups are bounded by a 5s timeout, as the SDK applies none by default.
Used by pull, push, publish, config --resolve-image-digests, oci:// remote resources and dry-run. The classic builder and use_api_socket keep config file credentials, as a short-lived token copied into a container would go stale.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
Load (cognitive complexity 48) mixed the enable/offline guards with
the artifact pull, the cache check and the image-index indirection.
The pull path moves to pullComposeArtifact and the index-to-manifest
resolution to resolveComposeManifest. Load reads as: guards, cache
lookup, pull on miss.
No behavior change, including the last-matching-manifest-wins loop
and the silent cache hit on a non-NotExist stat error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Cognitive complexity (gocognit, threshold 30 — the linter default) fits
this codebase better than raw cyclomatic complexity: it barely charges
guard clauses and early returns, and penalizes nesting instead. As a
result 8 of the 21 //nolint:gocyclo suppressions become unnecessary,
while deeply nested functions gocyclo never flagged are now caught.
The 19 functions still above the threshold keep a suppression, each
marked FIXME to complete the migration by restructuring them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Pulling an oci:// resource resolved each layer digest through the
registry manifests endpoint, which answers 500 when the digest points
to a non-manifest blob. containerd v2.3.0+ (pulled in by buildx v0.36
and buildkit v0.32) no longer falls back to the blobs endpoint unless
manifests returned 404, so publish/pull of compose artifacts broke.
Fetch layers directly with the descriptors already listed in the
manifest instead of resolving them again.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
The compose process performs OCI artifact fetches in-process via
containerd's docker resolver, whose default transport only honors
HTTP_PROXY/HTTPS_PROXY/NO_PROXY env vars. Users behind PAC-only
corporate proxies hit i/o timeouts on `oci://` includes and on
`compose publish`.
When Docker Desktop is the active engine and exposes httpproxy.sock,
route the resolver through it (PAC-aware). Falls back to the default
transport when DD is unavailable or the socket is missing. Modeled on
docker/mcp-gateway PR #354.
Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
Most files already grouped imports into "stdlib -> other -> local",
but some files didn't. The gci formatter is similar to goimports, but
has better options to make sure imports are grouped in the expected
order (and to make sure no additional groups are present).
This formatter has a 'fix' function, so code can be re-formatted auto-
matically;
golangci-lint run -v --fix
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>