mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-10-21 01:22:54 +00:00
🫂 ci: Unify Docker Image Publishing Workflows (#15447)
* 🛰️ ci: Publish Every Image Through One Native-ARM Workflow #15441 removed QEMU from the dev-branch image build and #15444 fixed the prefix-glob bug it shipped with. Both changes lived in one of five workflows that build images the same way; the other four still emulate arm64 with no layer cache and take 20-40 minutes. Rather than paste the pattern four more times, the build/merge logic moves to a reusable `docker-publish.yml` that every image workflow calls. #15444 is the argument for this: one wrong artifact-name expression would otherwise have needed correcting in five files, and the same prefix hazard exists in all of them (`librechat`/`librechat-api`, `librechat-dev`/`librechat-dev-api`, `lc-dev-staging`/`lc-dev-staging-api`). The shared implementation is the dev-branch one that already ran green, generalised only where callers differ. Callers pass what actually varies: - `images` — the target/file/image_name triples - `tag_suffixes` — newline-separated tags applied in both registries - `checkout_ref` / `build_branch` — main-image-workflow builds `main`, not the dispatch ref `main-image-workflow` and `tag-images` resolve their tags from git history, so each grew a small `resolve-tag(s)` job whose output feeds the publish. That also removes a duplicated `git fetch --tags` that previously ran once per matrix leg. Two details worth noting: - The digest-count assertion now derives its expected value from the `arches` input rather than a hard-coded 2, so the architecture list has a single source of truth shared by the build matrix and the merge guard. - Nothing is interpolated into a `run:` block any more; `build_branch` and the build digest arrive through `env`, so a ref name cannot reach a shell. Net 135 lines lighter, and QEMU no longer appears anywhere in .github/workflows. Workflow names are unchanged, so retry-docker-builds.yml still matches all five. * 🔗 fix: Pin Release Builds to One Commit and Survive Full Re-runs Review findings on the reusable publish workflow. **P1 — every build leg could resolve `main` separately.** `main-image-workflow` passed the symbolic ref `main` as `checkout_ref`, so the four build jobs each resolved it at whatever moment they were scheduled. If `main` advanced in between, the amd64 and arm64 halves of a release — or the API and application images — could be built from different commits and published under the same release and `latest` tags. `resolve-tag` now exports the commit it resolved the tag from, and the publish pins to that SHA. The single-job workflow this replaced checked out once, so this was a regression introduced by the split. `docker-publish` likewise now falls back to `github.sha` rather than passing an empty `ref` to `actions/checkout`, so callers that omit `checkout_ref` also get an immutable commit instead of relying on checkout's empty-ref handling. **P2 — digest artifacts on a full re-run.** `upload-artifact@v4` will not upload over an existing artifact name, and a full re-run keeps the run id, so a retry would fail before the merge could run. The suggested fix was to qualify the artifact name with `run_attempt`, but `retry-docker-builds` picks its endpoint by conclusion: `rerun` only when a job was cancelled, and `rerun-failed-jobs` otherwise. Under the latter the merge re-runs alone and must still find digests uploaded by the *previous* attempt's build legs, which a run_attempt-qualified name would hide — precisely the shape of the #15444 failure, where the builds passed and only the merge failed. `overwrite: true` resolves the conflict while keeping names stable across attempts. **Unreachable error branch.** `LATEST_TAG=$(git tag ... | grep ... | head -1)` under `set -euo pipefail` exits on grep's non-match, so the "No stable v<semver> tag found" message could never print. Added `|| true`, matching what `tag-images` already does. Pre-existing, carried over verbatim in the split.
This commit is contained in:
Danny Avila
GitHub
parent
22a9368604
commit
083be12ddc
6 changed files with 333 additions and 451 deletions
192
.github/workflows/dev-branch-images.yml
vendored
192
.github/workflows/dev-branch-images.yml
vendored
|
|
@ -33,182 +33,16 @@ concurrency:
|
|||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
# Each platform builds on a runner of its own architecture and pushes an
|
||||
# untagged, digest-addressed manifest. Building linux/arm64 on an amd64 runner
|
||||
# needs QEMU, which measured 5-6x slower than native on this image
|
||||
# (npm ci 52.6s -> 333.2s, npm run frontend 90.4s -> 442.8s) and made the
|
||||
# emulated leg ~84% of the build. The `merge` job below stitches the digests
|
||||
# into the multi-platform manifest lists that consumers actually pull.
|
||||
build:
|
||||
name: Build ${{ matrix.image_name }} (${{ matrix.platform }})
|
||||
runs-on: ${{ matrix.runner }}
|
||||
timeout-minutes: 90
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- target: api-build
|
||||
file: Dockerfile.multi
|
||||
image_name: lc-dev-api
|
||||
platform: linux/amd64
|
||||
arch: amd64
|
||||
runner: ubuntu-latest
|
||||
- target: api-build
|
||||
file: Dockerfile.multi
|
||||
image_name: lc-dev-api
|
||||
platform: linux/arm64
|
||||
arch: arm64
|
||||
runner: ubuntu-24.04-arm
|
||||
- target: node
|
||||
file: Dockerfile
|
||||
image_name: lc-dev
|
||||
platform: linux/amd64
|
||||
arch: amd64
|
||||
runner: ubuntu-latest
|
||||
- target: node
|
||||
file: Dockerfile
|
||||
image_name: lc-dev
|
||||
platform: linux/arm64
|
||||
arch: arm64
|
||||
runner: ubuntu-24.04-arm
|
||||
|
||||
steps:
|
||||
# Check out the repository
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v5
|
||||
|
||||
# Set up Docker Buildx (no QEMU: the runner is already the target arch)
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
# Log in to GitHub Container Registry
|
||||
- name: Log in to GitHub Container Registry
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
registry: ghcr.io
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
# Login to Docker Hub
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
# Prepare the environment
|
||||
- name: Prepare environment
|
||||
run: |
|
||||
cp .env.example .env
|
||||
|
||||
- name: Compute build metadata
|
||||
run: |
|
||||
echo "BUILD_COMMIT=${{ github.sha }}" >> $GITHUB_ENV
|
||||
echo "BUILD_BRANCH=${{ github.ref_name }}" >> $GITHUB_ENV
|
||||
echo "BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_ENV
|
||||
|
||||
# Build and push a digest-only image for this one platform. The layer cache
|
||||
# lives in GHCR beside the image (no 10GB Actions-cache cap, and readable
|
||||
# from every branch and workflow, unlike type=gha which is branch-scoped).
|
||||
# The cache ref is per-architecture so the two platform jobs cannot clobber
|
||||
# each other's manifest.
|
||||
- name: Build and push by digest
|
||||
id: build
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: .
|
||||
file: ${{ matrix.file }}
|
||||
platforms: ${{ matrix.platform }}
|
||||
target: ${{ matrix.target }}
|
||||
cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:buildcache-${{ matrix.arch }}
|
||||
cache-to: type=registry,ref=ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:buildcache-${{ matrix.arch }},mode=max
|
||||
outputs: type=image,"name=ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }},docker.io/${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}",push-by-digest=true,name-canonical=true,push=true
|
||||
build-args: |
|
||||
BUILD_COMMIT=${{ env.BUILD_COMMIT }}
|
||||
BUILD_BRANCH=${{ env.BUILD_BRANCH }}
|
||||
BUILD_DATE=${{ env.BUILD_DATE }}
|
||||
|
||||
- name: Export digest
|
||||
run: |
|
||||
mkdir -p /tmp/digests
|
||||
digest="${{ steps.build.outputs.digest }}"
|
||||
touch "/tmp/digests/${digest#sha256:}"
|
||||
|
||||
# The `-arch-` separator keeps one image's artifacts out of another's glob:
|
||||
# `lc-dev` is a prefix of `lc-dev-api`, so a bare `digests-lc-dev-*` pattern
|
||||
# matches all four digests and the merge resolves an API digest against the
|
||||
# app repository.
|
||||
- name: Upload digest
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: digests-${{ matrix.image_name }}-arch-${{ matrix.arch }}
|
||||
path: /tmp/digests/*
|
||||
if-no-files-found: error
|
||||
retention-days: 1
|
||||
|
||||
# Combine the per-architecture digests into one multi-platform manifest list
|
||||
# per registry, and apply the tags consumers pull.
|
||||
merge:
|
||||
name: Merge manifests (${{ matrix.image_name }})
|
||||
runs-on: ubuntu-latest
|
||||
needs: build
|
||||
timeout-minutes: 15
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
image_name: [lc-dev-api, lc-dev]
|
||||
|
||||
steps:
|
||||
- name: Download digests
|
||||
uses: actions/download-artifact@v7
|
||||
with:
|
||||
pattern: digests-${{ matrix.image_name }}-arch-*
|
||||
merge-multiple: true
|
||||
path: /tmp/digests
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
- name: Log in to GitHub Container Registry
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
registry: ghcr.io
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
# The digests are identical across registries (the manifest is
|
||||
# content-addressed), so the same set sources both manifest lists.
|
||||
- name: Create manifest lists and push
|
||||
working-directory: /tmp/digests
|
||||
env:
|
||||
GHCR_IMAGE: ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}
|
||||
DOCKERHUB_IMAGE: docker.io/${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
shopt -s nullglob
|
||||
digests=(*)
|
||||
# One digest per platform in the build matrix. Assert the exact count:
|
||||
# too few would publish a manifest missing an architecture, too many
|
||||
# means the artifact glob picked up another image's digests.
|
||||
expected=2
|
||||
if [ "${#digests[@]}" -ne "$expected" ]; then
|
||||
echo "Expected $expected platform digests, found ${#digests[@]}: ${digests[*]}" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "Merging ${#digests[@]} platform digests: ${digests[*]}"
|
||||
for image in "$GHCR_IMAGE" "$DOCKERHUB_IMAGE"; do
|
||||
docker buildx imagetools create \
|
||||
-t "${image}:${{ github.sha }}" \
|
||||
-t "${image}:latest" \
|
||||
"${digests[@]/#/${image}@sha256:}"
|
||||
done
|
||||
|
||||
- name: Inspect image
|
||||
run: |
|
||||
docker buildx imagetools inspect ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:${{ github.sha }}
|
||||
publish:
|
||||
uses: ./.github/workflows/docker-publish.yml
|
||||
with:
|
||||
images: >-
|
||||
[{"target":"api-build","file":"Dockerfile.multi","image_name":"lc-dev-api"},
|
||||
{"target":"node","file":"Dockerfile","image_name":"lc-dev"}]
|
||||
tag_suffixes: |
|
||||
${{ github.sha }}
|
||||
latest
|
||||
build_branch: ${{ github.ref_name }}
|
||||
secrets:
|
||||
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
|
|
|||
83
.github/workflows/dev-images.yml
vendored
83
.github/workflows/dev-images.yml
vendored
|
|
@ -29,73 +29,16 @@ permissions:
|
|||
packages: write
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 130
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- target: api-build
|
||||
file: Dockerfile.multi
|
||||
image_name: librechat-dev-api
|
||||
- target: node
|
||||
file: Dockerfile
|
||||
image_name: librechat-dev
|
||||
|
||||
steps:
|
||||
# Check out the repository
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v5
|
||||
|
||||
# Set up QEMU
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@v4
|
||||
|
||||
# Set up Docker Buildx
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
# Log in to GitHub Container Registry
|
||||
- name: Log in to GitHub Container Registry
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
registry: ghcr.io
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
# Login to Docker Hub
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
# Prepare the environment
|
||||
- name: Prepare environment
|
||||
run: |
|
||||
cp .env.example .env
|
||||
|
||||
- name: Compute build metadata
|
||||
run: |
|
||||
echo "BUILD_COMMIT=${{ github.sha }}" >> $GITHUB_ENV
|
||||
echo "BUILD_BRANCH=${{ github.ref_name }}" >> $GITHUB_ENV
|
||||
echo "BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_ENV
|
||||
|
||||
# Build and push Docker images for each target
|
||||
- name: Build and push Docker images
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: .
|
||||
file: ${{ matrix.file }}
|
||||
push: true
|
||||
tags: |
|
||||
ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:${{ github.sha }}
|
||||
ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:latest
|
||||
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}:${{ github.sha }}
|
||||
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}:latest
|
||||
platforms: linux/amd64,linux/arm64
|
||||
target: ${{ matrix.target }}
|
||||
build-args: |
|
||||
BUILD_COMMIT=${{ env.BUILD_COMMIT }}
|
||||
BUILD_BRANCH=${{ env.BUILD_BRANCH }}
|
||||
BUILD_DATE=${{ env.BUILD_DATE }}
|
||||
publish:
|
||||
uses: ./.github/workflows/docker-publish.yml
|
||||
with:
|
||||
images: >-
|
||||
[{"target":"api-build","file":"Dockerfile.multi","image_name":"librechat-dev-api"},
|
||||
{"target":"node","file":"Dockerfile","image_name":"librechat-dev"}]
|
||||
tag_suffixes: |
|
||||
${{ github.sha }}
|
||||
latest
|
||||
build_branch: ${{ github.ref_name }}
|
||||
secrets:
|
||||
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
|
|
|||
82
.github/workflows/dev-staging-images.yml
vendored
82
.github/workflows/dev-staging-images.yml
vendored
|
|
@ -8,72 +8,16 @@ permissions:
|
|||
packages: write
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- target: api-build
|
||||
file: Dockerfile.multi
|
||||
image_name: lc-dev-staging-api
|
||||
- target: node
|
||||
file: Dockerfile
|
||||
image_name: lc-dev-staging
|
||||
|
||||
steps:
|
||||
# Check out the repository
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v5
|
||||
|
||||
# Set up QEMU
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@v4
|
||||
|
||||
# Set up Docker Buildx
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
# Log in to GitHub Container Registry
|
||||
- name: Log in to GitHub Container Registry
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
registry: ghcr.io
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
# Login to Docker Hub
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
# Prepare the environment
|
||||
- name: Prepare environment
|
||||
run: |
|
||||
cp .env.example .env
|
||||
|
||||
- name: Compute build metadata
|
||||
run: |
|
||||
echo "BUILD_COMMIT=${{ github.sha }}" >> $GITHUB_ENV
|
||||
echo "BUILD_BRANCH=${{ github.ref_name }}" >> $GITHUB_ENV
|
||||
echo "BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_ENV
|
||||
|
||||
# Build and push Docker images for each target
|
||||
- name: Build and push Docker images
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: .
|
||||
file: ${{ matrix.file }}
|
||||
push: true
|
||||
tags: |
|
||||
ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:${{ github.sha }}
|
||||
ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:latest
|
||||
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}:${{ github.sha }}
|
||||
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}:latest
|
||||
platforms: linux/amd64,linux/arm64
|
||||
target: ${{ matrix.target }}
|
||||
build-args: |
|
||||
BUILD_COMMIT=${{ env.BUILD_COMMIT }}
|
||||
BUILD_BRANCH=${{ env.BUILD_BRANCH }}
|
||||
BUILD_DATE=${{ env.BUILD_DATE }}
|
||||
publish:
|
||||
uses: ./.github/workflows/docker-publish.yml
|
||||
with:
|
||||
images: >-
|
||||
[{"target":"api-build","file":"Dockerfile.multi","image_name":"lc-dev-staging-api"},
|
||||
{"target":"node","file":"Dockerfile","image_name":"lc-dev-staging"}]
|
||||
tag_suffixes: |
|
||||
${{ github.sha }}
|
||||
latest
|
||||
build_branch: ${{ github.ref_name }}
|
||||
secrets:
|
||||
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
|
|
|||
237
.github/workflows/docker-publish.yml
vendored
Normal file
237
.github/workflows/docker-publish.yml
vendored
Normal file
|
|
@ -0,0 +1,237 @@
|
|||
name: Reusable Docker Publish
|
||||
|
||||
# Builds each image once per architecture on a runner of that architecture, pushes
|
||||
# untagged digest-addressed manifests, then merges the digests into multi-platform
|
||||
# manifest lists. Building linux/arm64 on an amd64 runner needs QEMU, which measured
|
||||
# 5-6x slower than native on these images (npm ci 52.6s -> 333.2s, npm run frontend
|
||||
# 90.4s -> 442.8s) and made the emulated leg ~84% of the build.
|
||||
#
|
||||
# Every image-publishing workflow calls this, so a fix here reaches all of them. The
|
||||
# prefix-glob bug in #15446 was one artifact-name expression that would otherwise have
|
||||
# needed correcting in five separate files.
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
images:
|
||||
description: >-
|
||||
JSON array of objects with `target`, `file` and `image_name`, e.g.
|
||||
[{"target":"node","file":"Dockerfile","image_name":"lc-dev"}]
|
||||
required: true
|
||||
type: string
|
||||
tag_suffixes:
|
||||
description: >-
|
||||
Newline-separated tags applied to every image in every registry, e.g.
|
||||
"abc1234\nlatest". Blank lines are ignored.
|
||||
required: true
|
||||
type: string
|
||||
arches:
|
||||
description: JSON array of architectures to build.
|
||||
type: string
|
||||
default: '["amd64","arm64"]'
|
||||
checkout_ref:
|
||||
description: Ref to build. Defaults to the ref that triggered the caller.
|
||||
type: string
|
||||
default: ''
|
||||
build_branch:
|
||||
description: Value for the BUILD_BRANCH build-arg and runtime env.
|
||||
type: string
|
||||
default: ''
|
||||
build_timeout_minutes:
|
||||
type: number
|
||||
default: 90
|
||||
secrets:
|
||||
DOCKERHUB_USERNAME:
|
||||
required: true
|
||||
DOCKERHUB_TOKEN:
|
||||
required: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
|
||||
jobs:
|
||||
build:
|
||||
name: Build ${{ matrix.image.image_name }} (${{ matrix.arch }})
|
||||
runs-on: ${{ matrix.arch == 'arm64' && 'ubuntu-24.04-arm' || 'ubuntu-latest' }}
|
||||
timeout-minutes: ${{ inputs.build_timeout_minutes }}
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
image: ${{ fromJSON(inputs.images) }}
|
||||
arch: ${{ fromJSON(inputs.arches) }}
|
||||
|
||||
steps:
|
||||
# Falls back to the caller's commit SHA rather than an empty ref, so every
|
||||
# build leg of a run checks out the same immutable commit even if the
|
||||
# branch advances mid-run.
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v5
|
||||
with:
|
||||
ref: ${{ inputs.checkout_ref || github.sha }}
|
||||
|
||||
# No QEMU: the runner is already the target architecture.
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
- name: Log in to GitHub Container Registry
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
registry: ghcr.io
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Prepare environment
|
||||
run: cp .env.example .env
|
||||
|
||||
# Read from the checkout rather than github.sha, so a caller that builds a
|
||||
# specific ref (main-image-workflow) stamps that ref's commit.
|
||||
- name: Compute build metadata
|
||||
env:
|
||||
BUILD_BRANCH_INPUT: ${{ inputs.build_branch }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
printf 'BUILD_COMMIT=%s\n' "$(git rev-parse HEAD)" >> "$GITHUB_ENV"
|
||||
printf 'BUILD_BRANCH=%s\n' "$BUILD_BRANCH_INPUT" >> "$GITHUB_ENV"
|
||||
printf 'BUILD_DATE=%s\n' "$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> "$GITHUB_ENV"
|
||||
|
||||
# The layer cache lives in GHCR beside the image: no 10GB Actions-cache cap,
|
||||
# and readable from every branch and workflow, unlike type=gha which is
|
||||
# branch-scoped. The ref is per-architecture so the platform jobs cannot
|
||||
# clobber each other's cache manifest.
|
||||
- name: Build and push by digest
|
||||
id: build
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: .
|
||||
file: ${{ matrix.image.file }}
|
||||
platforms: linux/${{ matrix.arch }}
|
||||
target: ${{ matrix.image.target }}
|
||||
cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/${{ matrix.image.image_name }}:buildcache-${{ matrix.arch }}
|
||||
cache-to: type=registry,ref=ghcr.io/${{ github.repository_owner }}/${{ matrix.image.image_name }}:buildcache-${{ matrix.arch }},mode=max
|
||||
outputs: type=image,"name=ghcr.io/${{ github.repository_owner }}/${{ matrix.image.image_name }},docker.io/${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image.image_name }}",push-by-digest=true,name-canonical=true,push=true
|
||||
build-args: |
|
||||
BUILD_COMMIT=${{ env.BUILD_COMMIT }}
|
||||
BUILD_BRANCH=${{ env.BUILD_BRANCH }}
|
||||
BUILD_DATE=${{ env.BUILD_DATE }}
|
||||
|
||||
- name: Export digest
|
||||
env:
|
||||
BUILD_DIGEST: ${{ steps.build.outputs.digest }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
mkdir -p /tmp/digests
|
||||
digest="$BUILD_DIGEST"
|
||||
if [ -z "$digest" ]; then
|
||||
echo "Build produced no digest." >&2
|
||||
exit 1
|
||||
fi
|
||||
touch "/tmp/digests/${digest#sha256:}"
|
||||
|
||||
# The `-arch-` separator keeps one image's artifacts out of another's glob.
|
||||
# Image names commonly prefix one another (`librechat` / `librechat-api`,
|
||||
# `lc-dev` / `lc-dev-api`), and a bare `digests-<image>-*` pattern matches
|
||||
# both, which is exactly the failure in #15446.
|
||||
- name: Upload digest
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: digests-${{ matrix.image.image_name }}-arch-${{ matrix.arch }}
|
||||
path: /tmp/digests/*
|
||||
if-no-files-found: error
|
||||
retention-days: 1
|
||||
# A full re-run (retry-docker-builds uses `rerun` when a job was
|
||||
# cancelled) keeps the run id, and v4 refuses to upload over an existing
|
||||
# artifact name. Overwrite instead of qualifying the name by
|
||||
# run_attempt: `rerun-failed-jobs` re-runs the merge alone, which must
|
||||
# still find the digests the earlier attempt's build legs uploaded.
|
||||
overwrite: true
|
||||
|
||||
merge:
|
||||
name: Merge manifests (${{ matrix.image.image_name }})
|
||||
runs-on: ubuntu-latest
|
||||
needs: build
|
||||
timeout-minutes: 15
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
image: ${{ fromJSON(inputs.images) }}
|
||||
|
||||
steps:
|
||||
- name: Download digests
|
||||
uses: actions/download-artifact@v7
|
||||
with:
|
||||
pattern: digests-${{ matrix.image.image_name }}-arch-*
|
||||
merge-multiple: true
|
||||
path: /tmp/digests
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
- name: Log in to GitHub Container Registry
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
registry: ghcr.io
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
# The digests are identical across registries (manifests are content-addressed),
|
||||
# so the same set sources both manifest lists.
|
||||
- name: Create manifest lists and push
|
||||
working-directory: /tmp/digests
|
||||
env:
|
||||
GHCR_IMAGE: ghcr.io/${{ github.repository_owner }}/${{ matrix.image.image_name }}
|
||||
DOCKERHUB_IMAGE: docker.io/${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image.image_name }}
|
||||
EXPECTED_ARCHES: ${{ join(fromJSON(inputs.arches), ' ') }}
|
||||
TAG_SUFFIXES: ${{ inputs.tag_suffixes }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
shopt -s nullglob
|
||||
|
||||
# One digest per architecture. Assert the exact count: too few would
|
||||
# publish a manifest missing an architecture, too many means the artifact
|
||||
# glob crossed images (#15446). The old guard only rejected an empty
|
||||
# directory, which is why four digests sailed through.
|
||||
expected=$(wc -w <<< "$EXPECTED_ARCHES")
|
||||
digests=(*)
|
||||
if [ "${#digests[@]}" -ne "$expected" ]; then
|
||||
echo "Expected $expected platform digests ($EXPECTED_ARCHES), found ${#digests[@]}: ${digests[*]}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
tag_args=()
|
||||
while IFS= read -r suffix; do
|
||||
[ -n "$suffix" ] || continue
|
||||
tag_args+=(-t "IMAGE:${suffix}")
|
||||
done <<< "$TAG_SUFFIXES"
|
||||
if [ "${#tag_args[@]}" -eq 0 ]; then
|
||||
echo "No tag suffixes supplied; refusing to publish an untagged manifest." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Merging ${#digests[@]} digests into $(( ${#tag_args[@]} / 2 )) tag(s) per registry"
|
||||
for image in "$GHCR_IMAGE" "$DOCKERHUB_IMAGE"; do
|
||||
docker buildx imagetools create \
|
||||
"${tag_args[@]/IMAGE:/${image}:}" \
|
||||
"${digests[@]/#/${image}@sha256:}"
|
||||
done
|
||||
|
||||
- name: Inspect image
|
||||
env:
|
||||
GHCR_IMAGE: ghcr.io/${{ github.repository_owner }}/${{ matrix.image.image_name }}
|
||||
TAG_SUFFIXES: ${{ inputs.tag_suffixes }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
first_tag=$(grep -m1 -v '^[[:space:]]*$' <<< "$TAG_SUFFIXES")
|
||||
docker buildx imagetools inspect "${GHCR_IMAGE}:${first_tag}"
|
||||
97
.github/workflows/main-image-workflow.yml
vendored
97
.github/workflows/main-image-workflow.yml
vendored
|
|
@ -8,18 +8,14 @@ permissions:
|
|||
packages: write
|
||||
|
||||
jobs:
|
||||
build:
|
||||
# Resolved once here rather than per build leg: the tag is an input to the
|
||||
# publish, and the merge job needs it too.
|
||||
resolve-tag:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- target: api-build
|
||||
file: Dockerfile.multi
|
||||
image_name: librechat-api
|
||||
- target: node
|
||||
file: Dockerfile
|
||||
image_name: librechat
|
||||
|
||||
timeout-minutes: 10
|
||||
outputs:
|
||||
latest_tag: ${{ steps.tag.outputs.latest_tag }}
|
||||
sha: ${{ steps.tag.outputs.sha }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v5
|
||||
|
|
@ -28,65 +24,36 @@ jobs:
|
|||
fetch-depth: 0
|
||||
|
||||
- name: Fetch tags and set the latest tag
|
||||
id: tag
|
||||
run: |
|
||||
set -euo pipefail
|
||||
git fetch --tags --force
|
||||
LATEST_TAG=$(git tag --list 'v[0-9]*' --sort=-v:refname | grep -E '^v[0-9]+[.][0-9]+[.][0-9]+$' | head -n 1)
|
||||
# `|| true` keeps pipefail from killing the step when grep matches
|
||||
# nothing, so the explicit error below is reachable.
|
||||
LATEST_TAG=$(git tag --list 'v[0-9]*' --sort=-v:refname | grep -E '^v[0-9]+[.][0-9]+[.][0-9]+$' | head -n 1 || true)
|
||||
if [ -z "$LATEST_TAG" ]; then
|
||||
echo "::error::No stable v<semver> tag found"
|
||||
exit 1
|
||||
fi
|
||||
printf 'LATEST_TAG=%s\n' "$LATEST_TAG" >> "$GITHUB_ENV"
|
||||
printf 'latest_tag=%s\n' "$LATEST_TAG" >> "$GITHUB_OUTPUT"
|
||||
# Pin the publish to the commit this job resolved the tag from. `main`
|
||||
# is a moving ref: were each build leg to resolve it independently, the
|
||||
# amd64 and arm64 halves of one release could come from different
|
||||
# commits and still be published under the same tags.
|
||||
printf 'sha=%s\n' "$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Compute build metadata
|
||||
run: |
|
||||
printf 'BUILD_COMMIT=%s\n' "$(git rev-parse HEAD)" >> "$GITHUB_ENV"
|
||||
printf 'BUILD_BRANCH=main\n' >> "$GITHUB_ENV"
|
||||
printf 'BUILD_DATE=%s\n' "$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> "$GITHUB_ENV"
|
||||
|
||||
# Set up QEMU
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@v4
|
||||
|
||||
# Set up Docker Buildx
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
# Log in to GitHub Container Registry
|
||||
- name: Log in to GitHub Container Registry
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
registry: ghcr.io
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
# Login to Docker Hub
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
# Prepare the environment
|
||||
- name: Prepare environment
|
||||
run: |
|
||||
cp .env.example .env
|
||||
|
||||
# Build and push Docker images for each target
|
||||
- name: Build and push Docker images
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: .
|
||||
file: ${{ matrix.file }}
|
||||
push: true
|
||||
tags: |
|
||||
ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:${{ env.LATEST_TAG }}
|
||||
ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:latest
|
||||
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}:${{ env.LATEST_TAG }}
|
||||
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}:latest
|
||||
platforms: linux/amd64,linux/arm64
|
||||
target: ${{ matrix.target }}
|
||||
build-args: |
|
||||
BUILD_COMMIT=${{ env.BUILD_COMMIT }}
|
||||
BUILD_BRANCH=${{ env.BUILD_BRANCH }}
|
||||
BUILD_DATE=${{ env.BUILD_DATE }}
|
||||
publish:
|
||||
needs: resolve-tag
|
||||
uses: ./.github/workflows/docker-publish.yml
|
||||
with:
|
||||
images: >-
|
||||
[{"target":"api-build","file":"Dockerfile.multi","image_name":"librechat-api"},
|
||||
{"target":"node","file":"Dockerfile","image_name":"librechat"}]
|
||||
tag_suffixes: |
|
||||
${{ needs.resolve-tag.outputs.latest_tag }}
|
||||
latest
|
||||
checkout_ref: ${{ needs.resolve-tag.outputs.sha }}
|
||||
build_branch: main
|
||||
secrets:
|
||||
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
|
|
|||
93
.github/workflows/tag-images.yml
vendored
93
.github/workflows/tag-images.yml
vendored
|
|
@ -10,22 +10,19 @@ permissions:
|
|||
packages: write
|
||||
|
||||
jobs:
|
||||
build:
|
||||
# Validates the release tag and resolves which tags the images should carry.
|
||||
# `latest` is only applied when this tag is both stable and the newest stable
|
||||
# tag in the repository, so re-cutting an older patch cannot move `latest`.
|
||||
resolve-tags:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- target: api-build
|
||||
file: Dockerfile.multi
|
||||
image_name: librechat-api
|
||||
- target: node
|
||||
file: Dockerfile
|
||||
image_name: librechat
|
||||
|
||||
timeout-minutes: 10
|
||||
outputs:
|
||||
tag_suffixes: ${{ steps.tags.outputs.tag_suffixes }}
|
||||
steps:
|
||||
# Check out the repository
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v5
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Validate release tag
|
||||
id: release-tag
|
||||
|
|
@ -46,45 +43,9 @@ jobs:
|
|||
echo "is_stable=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
# Set up QEMU
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@v4
|
||||
|
||||
# Set up Docker Buildx
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
# Log in to GitHub Container Registry
|
||||
- name: Log in to GitHub Container Registry
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
registry: ghcr.io
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
# Login to Docker Hub
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
# Prepare the environment
|
||||
- name: Prepare environment
|
||||
run: |
|
||||
cp .env.example .env
|
||||
|
||||
- name: Compute build metadata
|
||||
run: |
|
||||
echo "BUILD_COMMIT=${{ github.sha }}" >> $GITHUB_ENV
|
||||
echo "BUILD_BRANCH=${{ github.ref_name }}" >> $GITHUB_ENV
|
||||
echo "BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_ENV
|
||||
|
||||
- name: Resolve image tags
|
||||
id: image-tags
|
||||
id: tags
|
||||
env:
|
||||
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
IMAGE_NAME: ${{ matrix.image_name }}
|
||||
IMAGE_TAG: ${{ steps.release-tag.outputs.image_tag }}
|
||||
IS_STABLE: ${{ steps.release-tag.outputs.is_stable }}
|
||||
run: |
|
||||
|
|
@ -92,27 +53,23 @@ jobs:
|
|||
git fetch --tags --force
|
||||
LATEST_STABLE_TAG=$(git tag --list 'v[0-9]*' --sort=-v:refname | grep -E '^v[0-9]+[.][0-9]+[.][0-9]+$' | head -n 1 || true)
|
||||
{
|
||||
echo 'tags<<EOF'
|
||||
printf 'ghcr.io/%s/%s:%s\n' "$GITHUB_REPOSITORY_OWNER" "$IMAGE_NAME" "$IMAGE_TAG"
|
||||
printf '%s/%s:%s\n' "$DOCKERHUB_USERNAME" "$IMAGE_NAME" "$IMAGE_TAG"
|
||||
echo 'tag_suffixes<<EOF'
|
||||
printf '%s\n' "$IMAGE_TAG"
|
||||
if [ "$IS_STABLE" = "true" ] && [ "$IMAGE_TAG" = "$LATEST_STABLE_TAG" ]; then
|
||||
printf 'ghcr.io/%s/%s:latest\n' "$GITHUB_REPOSITORY_OWNER" "$IMAGE_NAME"
|
||||
printf '%s/%s:latest\n' "$DOCKERHUB_USERNAME" "$IMAGE_NAME"
|
||||
printf 'latest\n'
|
||||
fi
|
||||
echo 'EOF'
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Build and push Docker images for each target
|
||||
- name: Build and push Docker images
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: .
|
||||
file: ${{ matrix.file }}
|
||||
push: true
|
||||
tags: ${{ steps.image-tags.outputs.tags }}
|
||||
platforms: linux/amd64,linux/arm64
|
||||
target: ${{ matrix.target }}
|
||||
build-args: |
|
||||
BUILD_COMMIT=${{ env.BUILD_COMMIT }}
|
||||
BUILD_BRANCH=${{ env.BUILD_BRANCH }}
|
||||
BUILD_DATE=${{ env.BUILD_DATE }}
|
||||
publish:
|
||||
needs: resolve-tags
|
||||
uses: ./.github/workflows/docker-publish.yml
|
||||
with:
|
||||
images: >-
|
||||
[{"target":"api-build","file":"Dockerfile.multi","image_name":"librechat-api"},
|
||||
{"target":"node","file":"Dockerfile","image_name":"librechat"}]
|
||||
tag_suffixes: ${{ needs.resolve-tags.outputs.tag_suffixes }}
|
||||
build_branch: ${{ github.ref_name }}
|
||||
secrets:
|
||||
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue