mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-18 14:08:21 +00:00
* feat: Scaffold the @librechat/chat package Adds packages/chat with core, react and components entries, a tsdown build, a node-environment jest config, turbo and root build wiring, import boundary rules, design-lint scope, and core types for the chat transport, stream, host and status. No code moves yet. * test: Cover the @librechat/chat boundaries as scenarios * fix: Run the chat suite in test:turbo and build it for the mock lane test:turbo now selects @librechat/chat, the mock e2e job builds the package the chat-package scenarios load, and the synthetic repository in the suppression scenarios carries the packages/chat/src root lint:design:record names. * chore: Build the chat package in frontend:ci too * chore: Run the chat suite in test:all * fix: Hold core TSX to the chat boundary and scan the package for cycles * fix: Apply the chat import boundary to JavaScript sources * fix: Keep nested core files off the components entry * fix: Close the remaining chat package gates and fence aborts by generation Blocks imports through the @librechat/frontend workspace name, adds the chat sources to the i18n scan, the CSS color gate and the client Tailwind content, and lets ChatAbortRequest carry the generation it means to stop. * fix: Keep JSX out of the chat core and carry abort and reconnect outcomes Rejects JSX under the core, copies the chat manifest before the Docker install, keys the client app build caches on the chat sources, fences reconnects by generation, and types the abort result the transport returns. * test: Give the CSS gate's scratch tree the chat root it now scans * fix: Close the remaining @librechat/chat gates and wiring Rejects import() and require() across the package boundaries, holds chat components to the literal-copy rule, compiles TSX in the package's jest transform, builds the package in Frontend Unit Tests, requires its test job for main promotion, adds it to the unused-dependency scan, and lists it wherever a sibling workspace is listed: Dockerfile.multi, the docker smoke paths, the remaining node_modules caches, the reinstall helper, the bun build chain and AGENTS.md. * fix: Tighten the chat boundaries and carry steering and Assistants aborts Template-literal import() and require() and self-imports through the package name are rejected, user-facing JSX attributes need localized text, the cycle scan resolves self-imports, Dockerfile.multi builds the package for the client stage, and the transport carries steer, cancelSteer, armSteer and the Assistants abort key. * feat: Match the chat transport and host contracts to the client's The transport now reports settled and replaced starts, the stream's protocol version and resumed flag, reconnect replay, generation status, paused-run resume with approvals or answers, abort and steer protocol versions, and the request error shape. The host config and auth carry what the client's ChatSettings now supplies: rename-while-running, model specs, retention mode and the user. * fix: Keep lazy React-entry loads out of the core and type the Assistants stop Relative import() and require() of the /react entry from core are rejected, and an abortKey stop resolves to the terminal conversation and run messages the Assistants route returns. * fix: Fence start controls by protocol and close extension-bearing entry loads Settled and replaced start results carry the protocol version, core loads of the /react or /components entry with a file extension are rejected, and test:packages:chat builds data-provider first.
687 lines
33 KiB
YAML
687 lines
33 KiB
YAML
name: Playwright E2E Tests
|
|
|
|
on:
|
|
pull_request:
|
|
paths:
|
|
- '**'
|
|
- '!**.md'
|
|
- '!.github/workflows/**'
|
|
- '.github/workflows/playwright-mock.yml'
|
|
schedule:
|
|
- cron: '0 5 * * *'
|
|
workflow_dispatch:
|
|
inputs:
|
|
reason:
|
|
description: 'Reason for manual trigger'
|
|
required: false
|
|
default: 'Manual e2e run'
|
|
coverage:
|
|
description: 'Coverage profile: full suite on both stores, or the PR shards'
|
|
type: choice
|
|
options:
|
|
- full
|
|
- pr
|
|
default: full
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
concurrency:
|
|
group: playwright-mock-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
env:
|
|
NODE_OPTIONS: '--max-old-space-size=${{ secrets.NODE_MAX_OLD_SPACE_SIZE || 6144 }}'
|
|
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: '1'
|
|
|
|
jobs:
|
|
# Stage 2 of codegraph gating (stage 1 = backend jest in backend-review.yml). The graph decides
|
|
# two matrix lanes whose relevance is a reachability question: the redis-transport shards (the
|
|
# stream-boundary specs under a real Redis round-trip) and the MCP list_changed lanes. Memory
|
|
# shards always run. Monotone and fail-open: a lane is dropped ONLY on an explicit `false` from
|
|
# the service; unavailable/unconfigured/non-synchronize events keep the full matrix. Lock
|
|
# attribution rides along so a backend dependency bump (reaches nothing in the graph) still
|
|
# earns both lanes. Kill switch: repo variable CODEGRAPH_GATING=off.
|
|
codegraph_select:
|
|
name: Codegraph select
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
if: >-
|
|
github.event_name == 'pull_request' &&
|
|
github.event.action == 'synchronize' &&
|
|
vars.CODEGRAPH_GATING != 'off' &&
|
|
github.event.pull_request != null &&
|
|
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association)
|
|
outputs:
|
|
decided: ${{ steps.sel.outputs.decided }}
|
|
e2e_include: ${{ steps.sel.outputs.e2e_include }}
|
|
mcp_run: ${{ steps.sel.outputs.mcp_run }}
|
|
e2e_skip: ${{ steps.sel.outputs.e2e_skip }}
|
|
steps:
|
|
- name: Select matrix lanes, fail open on any doubt
|
|
id: sel
|
|
env:
|
|
URL: ${{ secrets.CODEGRAPH_URL }}
|
|
TOKEN: ${{ secrets.CODEGRAPH_TOKEN }}
|
|
GH_TOKEN: ${{ github.token }}
|
|
REPO: ${{ github.repository }}
|
|
PR: ${{ github.event.pull_request.number }}
|
|
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
|
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
|
CHANGED: ${{ github.event.pull_request.changed_files }}
|
|
E2E_SKIP_ARMED: ${{ vars.CODEGRAPH_E2E_SKIP }}
|
|
FULL_INCLUDE: '{"include":[{"name":"memory, shard 1/6","stream_store":"memory","redis_image":"","suite":"full","shard":"1/6","artifact":"memory-1-of-6","email_change":"true"},{"name":"memory, shard 2/6","stream_store":"memory","redis_image":"","suite":"full","shard":"2/6","artifact":"memory-2-of-6"},{"name":"memory, shard 3/6","stream_store":"memory","redis_image":"","suite":"full","shard":"3/6","artifact":"memory-3-of-6"},{"name":"memory, shard 4/6","stream_store":"memory","redis_image":"","suite":"full","shard":"4/6","artifact":"memory-4-of-6"},{"name":"memory, shard 5/6","stream_store":"memory","redis_image":"","suite":"full","shard":"5/6","artifact":"memory-5-of-6"},{"name":"memory, shard 6/6","stream_store":"memory","redis_image":"","suite":"full","shard":"6/6","artifact":"memory-6-of-6"},{"name":"redis transport, shard 1/2","stream_store":"redis","redis_image":"redis:7-alpine","suite":"transport","shard":"1/2","artifact":"redis-transport-1-of-2"},{"name":"redis transport, shard 2/2","stream_store":"redis","redis_image":"redis:7-alpine","suite":"transport","shard":"2/2","artifact":"redis-transport-2-of-2"}]}'
|
|
run: |
|
|
set +e
|
|
note() { echo "$1" >> "$GITHUB_STEP_SUMMARY"; }
|
|
note "### Codegraph select — GATING (Playwright matrix lanes)"
|
|
if [ -z "$URL" ] || [ -z "$TOKEN" ]; then note "_no codegraph config; running FULL_"; exit 0; fi
|
|
# A failed or truncated page must not become a shorter file list: the pipeline would hide
|
|
# gh's exit status behind jq, and a partial list can turn a required lane off. Check the
|
|
# fetch status AND the count against the PR's own changed_files (Codex P1, #15136).
|
|
if ! gh api "repos/$REPO/pulls/$PR/files" --paginate \
|
|
--jq '.[] | {path: .filename, status, patch}' > files.ndjson; then
|
|
note "_could not fetch changed files; running FULL_"; exit 0
|
|
fi
|
|
jq -s . files.ndjson > files.json
|
|
N=$(jq 'length' files.json)
|
|
if [ "$N" -eq 0 ] || { [ -n "$CHANGED" ] && [ "$N" -ne "$CHANGED" ]; }; then
|
|
note "_changed-file list incomplete ($N of ${CHANGED:-?}); running FULL_"; exit 0
|
|
fi
|
|
jq -c --arg b "$BASE_SHA" --arg h "$HEAD_SHA" \
|
|
'{files: ., mode: "safe", lockBaseSha: $b, lockHeadSha: $h}' files.json > body.json
|
|
# curl's status is checked explicitly: a transfer that times out or truncates after a
|
|
# parseable body must fail open, not be honoured (Codex P1, #15136). --fail-with-body
|
|
# also turns HTTP errors into a failure while keeping the error text for the summary.
|
|
RESP=$(curl -sS --fail-with-body -m 45 -H "Authorization: Bearer $TOKEN" \
|
|
-H 'content-type: application/json' --data-binary @body.json "$URL/v1/select"); RC=$?
|
|
if [ "$RC" -ne 0 ] || [ -z "$RESP" ] || ! echo "$RESP" | jq -e '.matrix["playwright-mock"]' >/dev/null 2>&1; then
|
|
note "_codegraph unavailable (curl exit $RC: ${RESP:0:120}); running FULL_"
|
|
exit 0
|
|
fi
|
|
# A lane is skipped only on the JSON boolean false — tested inside jq, because `jq -r`
|
|
# prints the string "false" and the boolean identically (Codex P1, #15136). Anything
|
|
# else (true, null, a string, missing) runs.
|
|
if echo "$RESP" | jq -e '.e2e.fail_open == true' >/dev/null 2>&1; then
|
|
note "_fail-open decision (root/workflow/lockfile change or stale graph): everything runs_"
|
|
fi
|
|
REDIS=$(echo "$RESP" | jq -c '.matrix["playwright-mock"].redis_transport')
|
|
MCP=$(echo "$RESP" | jq -c '.matrix["playwright-mock"].mcp_tool_list_changed')
|
|
REDIS_SKIP=0; MCP_SKIP=0
|
|
echo "$RESP" | jq -e '.matrix["playwright-mock"].redis_transport == false' >/dev/null 2>&1 && REDIS_SKIP=1
|
|
echo "$RESP" | jq -e '.matrix["playwright-mock"].mcp_tool_list_changed == false' >/dev/null 2>&1 && MCP_SKIP=1
|
|
note "| lane | decision |"
|
|
note "|---|---|"
|
|
if [ "$REDIS_SKIP" = 1 ]; then
|
|
INCLUDE=$(echo "$FULL_INCLUDE" | jq -c '.include |= map(select(.suite != "transport"))')
|
|
note "| redis transport | skip (no reach into the stream boundary) |"
|
|
else
|
|
INCLUDE="$FULL_INCLUDE"
|
|
note "| redis transport | run |"
|
|
fi
|
|
if ! echo "$INCLUDE" | jq -e '.include | length >= 6' >/dev/null 2>&1; then
|
|
note "_matrix assembly failed; running FULL_"
|
|
exit 0
|
|
fi
|
|
echo "e2e_include=$INCLUDE" >> "$GITHUB_OUTPUT"
|
|
# Graduated per-spec skips are DARK until the operator arms repo variable
|
|
# CODEGRAPH_E2E_SKIP=on (the election switch — flipped only when the pre-registered
|
|
# resume condition holds). Even then, act only on a well-typed list from a non-fail-open
|
|
# decision: every entry must be a pool spec path, or nothing is skipped. The server
|
|
# already intersects with this PR's skippable tier and applies the streak bars
|
|
# (2x where history-coupled); see codegraph-poc service/graduate.ts.
|
|
SKIP=""
|
|
if [ "$E2E_SKIP_ARMED" = "on" ]; then
|
|
if echo "$RESP" | jq -e '(.e2e.fail_open != true) and (.e2e.graduated | type == "array" and all(.[]?; type == "string" and test("^e2e/specs/mock/[A-Za-z0-9._/-]+\\.spec\\.ts$") and (contains("..") | not)))' >/dev/null 2>&1; then
|
|
SKIP=$(echo "$RESP" | jq -r '[.e2e.graduated[] | sub("^e2e/"; "")] | join(" ")')
|
|
else
|
|
note "_graduated list absent or malformed; no specs skipped_"
|
|
fi
|
|
fi
|
|
echo "e2e_skip=$SKIP" >> "$GITHUB_OUTPUT"
|
|
if [ -n "$SKIP" ]; then
|
|
note "| graduated spec skips | $(echo "$SKIP" | wc -w | tr -d ' ') (armed) |"
|
|
echo "codegraph-e2e-graduated-skips: $SKIP"
|
|
fi
|
|
echo "codegraph-select: redis_transport=$REDIS mcp_tool_list_changed=$MCP matrix_entries=$(echo "$INCLUDE" | jq '.include | length')"
|
|
if [ "$MCP_SKIP" = 1 ]; then
|
|
echo "mcp_run=false" >> "$GITHUB_OUTPUT"
|
|
note "| MCP list_changed | skip (no reach into MCP) |"
|
|
else
|
|
echo "mcp_run=true" >> "$GITHUB_OUTPUT"
|
|
note "| MCP list_changed | run |"
|
|
fi
|
|
echo "decided=true" >> "$GITHUB_OUTPUT"
|
|
note ""
|
|
note "memory shards always run · kill switch: repo variable \`CODEGRAPH_GATING=off\` · full matrix on PR open and nightly"
|
|
exit 0
|
|
|
|
e2e_shards:
|
|
name: e2e (${{ matrix.name }})
|
|
needs: [codegraph_select]
|
|
if: >-
|
|
!cancelled() &&
|
|
(github.event_name == 'schedule' ||
|
|
github.event_name == 'workflow_dispatch' ||
|
|
(github.event_name == 'pull_request' &&
|
|
github.event.pull_request != null &&
|
|
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association) ||
|
|
github.event.pull_request.head.repo.full_name == github.repository)))
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
# Separate runners isolate each shard's user, MongoDB, Redis and fixture state.
|
|
# Keep workers: 1 in Playwright; shared-user cleanup is not worker-safe.
|
|
strategy:
|
|
fail-fast: false
|
|
matrix: >-
|
|
${{
|
|
(github.event_name == 'pull_request' && needs.codegraph_select.outputs.e2e_include != '' &&
|
|
fromJSON(needs.codegraph_select.outputs.e2e_include)) ||
|
|
((github.event_name == 'pull_request' || inputs.coverage == 'pr') &&
|
|
fromJSON('{"include":[{"name":"memory, shard 1/6","stream_store":"memory","redis_image":"","suite":"full","shard":"1/6","artifact":"memory-1-of-6","email_change":"true"},{"name":"memory, shard 2/6","stream_store":"memory","redis_image":"","suite":"full","shard":"2/6","artifact":"memory-2-of-6"},{"name":"memory, shard 3/6","stream_store":"memory","redis_image":"","suite":"full","shard":"3/6","artifact":"memory-3-of-6"},{"name":"memory, shard 4/6","stream_store":"memory","redis_image":"","suite":"full","shard":"4/6","artifact":"memory-4-of-6"},{"name":"memory, shard 5/6","stream_store":"memory","redis_image":"","suite":"full","shard":"5/6","artifact":"memory-5-of-6"},{"name":"memory, shard 6/6","stream_store":"memory","redis_image":"","suite":"full","shard":"6/6","artifact":"memory-6-of-6"},{"name":"redis transport, shard 1/2","stream_store":"redis","redis_image":"redis:7-alpine","suite":"transport","shard":"1/2","artifact":"redis-transport-1-of-2"},{"name":"redis transport, shard 2/2","stream_store":"redis","redis_image":"redis:7-alpine","suite":"transport","shard":"2/2","artifact":"redis-transport-2-of-2"}]}')) ||
|
|
fromJSON('{"include":[{"name":"memory, shard 1/2","stream_store":"memory","redis_image":"","suite":"full","shard":"1/2","artifact":"memory-1-of-2","email_change":"true"},{"name":"memory, shard 2/2","stream_store":"memory","redis_image":"","suite":"full","shard":"2/2","artifact":"memory-2-of-2"},{"name":"redis, shard 1/2","stream_store":"redis","redis_image":"redis:7-alpine","suite":"full","shard":"1/2","artifact":"redis-1-of-2"},{"name":"redis, shard 2/2","stream_store":"redis","redis_image":"redis:7-alpine","suite":"full","shard":"2/2","artifact":"redis-2-of-2"}]}')
|
|
}}
|
|
services:
|
|
redis:
|
|
image: ${{ matrix.redis_image }}
|
|
ports:
|
|
- 6379:6379
|
|
options: >-
|
|
--health-cmd "redis-cli ping"
|
|
--health-interval 5s
|
|
--health-timeout 3s
|
|
--health-retries 10
|
|
env:
|
|
E2E_CHROMIUM_CHANNEL: chrome
|
|
E2E_STREAM_STORE: ${{ matrix.stream_store }}
|
|
REDIS_URI: redis://127.0.0.1:6379
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
|
|
- name: Use Node.js 24.16.0
|
|
uses: actions/setup-node@v5
|
|
with:
|
|
node-version: '24.16.0'
|
|
|
|
- name: Restore node_modules cache
|
|
id: cache-node-modules
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: |
|
|
node_modules
|
|
client/node_modules
|
|
packages/client/node_modules
|
|
packages/chat/node_modules
|
|
packages/data-provider/node_modules
|
|
packages/data-schemas/node_modules
|
|
packages/api/node_modules
|
|
api/node_modules
|
|
key: node-modules-e2e-${{ runner.os }}-24.16.0-${{ hashFiles('package-lock.json') }}
|
|
|
|
- name: Install dependencies
|
|
if: steps.cache-node-modules.outputs.cache-hit != 'true'
|
|
run: npm ci
|
|
|
|
- name: Test Playwright matrix selection
|
|
run: node --test .github/scripts/playwright-matrix.test.mjs
|
|
|
|
- name: Restore data-provider build cache
|
|
id: cache-data-provider
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: packages/data-provider/dist
|
|
key: build-data-provider-${{ runner.os }}-${{ hashFiles('package.json', 'package-lock.json', 'packages/data-provider/src/**', 'packages/data-provider/tsconfig*.json', 'packages/data-provider/tsdown.config.mjs', 'packages/data-provider/package.json') }}
|
|
|
|
- name: Build data-provider
|
|
if: steps.cache-data-provider.outputs.cache-hit != 'true'
|
|
run: npm run build:data-provider
|
|
|
|
- name: Restore data-schemas build cache
|
|
id: cache-data-schemas
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: packages/data-schemas/dist
|
|
key: build-data-schemas-${{ runner.os }}-${{ hashFiles('package.json', 'package-lock.json', 'packages/data-schemas/src/**', 'packages/data-schemas/tsconfig*.json', 'packages/data-schemas/tsdown.config.mjs', 'packages/data-schemas/package.json', 'packages/data-provider/src/**', 'packages/data-provider/tsconfig*.json', 'packages/data-provider/tsdown.config.mjs', 'packages/data-provider/package.json') }}
|
|
|
|
- name: Build data-schemas
|
|
if: steps.cache-data-schemas.outputs.cache-hit != 'true'
|
|
run: npm run build:data-schemas
|
|
|
|
- name: Restore api build cache
|
|
id: cache-api
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: packages/api/dist
|
|
key: build-api-${{ runner.os }}-${{ hashFiles('package.json', 'package-lock.json', 'packages/api/src/**', 'packages/api/tsconfig*.json', 'packages/api/tsdown.config.mjs', 'packages/api/package.json', 'packages/data-provider/src/**', 'packages/data-provider/tsconfig*.json', 'packages/data-provider/tsdown.config.mjs', 'packages/data-provider/package.json', 'packages/data-schemas/src/**', 'packages/data-schemas/tsconfig*.json', 'packages/data-schemas/tsdown.config.mjs', 'packages/data-schemas/package.json') }}
|
|
|
|
- name: Build api
|
|
if: steps.cache-api.outputs.cache-hit != 'true'
|
|
run: npm run build:api
|
|
|
|
- name: Restore client-package build cache
|
|
id: cache-client-package
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: packages/client/dist
|
|
key: build-client-package-${{ runner.os }}-${{ hashFiles('package.json', 'package-lock.json', 'packages/client/src/**', 'packages/client/tsconfig*.json', 'packages/client/tsdown.config.mjs', 'packages/client/package.json', 'packages/data-provider/src/**', 'packages/data-provider/tsconfig*.json', 'packages/data-provider/tsdown.config.mjs', 'packages/data-provider/package.json') }}
|
|
|
|
- name: Build client-package
|
|
if: steps.cache-client-package.outputs.cache-hit != 'true'
|
|
run: npm run build:client-package
|
|
|
|
# About a second, so it is not cached; the chat-package scenarios load the built entries.
|
|
- name: Build chat package
|
|
run: npm run build:chat
|
|
|
|
- name: Restore client app build cache
|
|
id: cache-client-app
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: client/dist
|
|
key: build-client-app-e2e-${{ runner.os }}-${{ hashFiles('package.json', 'package-lock.json', 'client/src/**', 'client/public/**', 'client/index.html', 'client/package.json', 'client/vite.config.*', 'client/tsconfig*.json', 'client/tailwind.config.*', 'client/postcss.config.*', 'packages/client/src/**', 'packages/chat/src/**', 'packages/client/tailwind.preset.cjs', 'packages/client/tsconfig*.json', 'packages/client/tsdown.config.mjs', 'packages/client/package.json', 'packages/data-provider/src/**', 'packages/data-provider/tsconfig*.json', 'packages/data-provider/tsdown.config.mjs', 'packages/data-provider/package.json') }}
|
|
|
|
- name: Build client app
|
|
if: steps.cache-client-app.outputs.cache-hit != 'true'
|
|
run: npm run build:client
|
|
|
|
- name: Verify Chrome is present
|
|
run: google-chrome --version
|
|
|
|
# `video: 'on-first-retry'` needs ffmpeg; without it the first retry dies in
|
|
# browserContext.newPage before the test body runs, so a flaky test loses the
|
|
# retry that would have recovered it.
|
|
#
|
|
# This step used to burn its full 90s bound on every job. Playwright's bundled
|
|
# extractor hangs on Node 24.16.0 (a yauzl/extract-zip regression fixed in
|
|
# Playwright 1.60.0): the 2.3MB download finished in under a second, then
|
|
# extraction stalled and the timeout reaped it, leaving a truncated binary and
|
|
# no INSTALLATION_COMPLETE marker — so ffmpeg was never actually installed and
|
|
# retries never got video. With Playwright bumped past the fix the install
|
|
# takes about a second, and a restored cache skips it outright.
|
|
#
|
|
# The cache is only saved once the binary is verified to run, so a partial
|
|
# extraction can never be promoted into a cache every later job restores.
|
|
# Kept non-fatal: retry video is a debugging aid, not something CI asserts on.
|
|
- name: Resolve Playwright version
|
|
id: playwright-version
|
|
run: |
|
|
version=$(node -p "require('./package-lock.json').packages['node_modules/playwright-core'].version")
|
|
echo "version=${version}" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Restore Playwright ffmpeg cache
|
|
id: cache-ffmpeg
|
|
uses: actions/cache/restore@v5
|
|
with:
|
|
path: ~/.cache/ms-playwright
|
|
key: playwright-ffmpeg-${{ runner.os }}-${{ steps.playwright-version.outputs.version }}
|
|
|
|
- name: Install Playwright ffmpeg (best effort)
|
|
id: install-ffmpeg
|
|
if: steps.cache-ffmpeg.outputs.cache-hit != 'true'
|
|
timeout-minutes: 3
|
|
continue-on-error: true
|
|
run: |
|
|
timeout -k 10 60 npx playwright install ffmpeg
|
|
.github/scripts/verify-playwright-ffmpeg.sh
|
|
|
|
- name: Save Playwright ffmpeg cache
|
|
if: steps.install-ffmpeg.outcome == 'success'
|
|
continue-on-error: true
|
|
uses: actions/cache/save@v5
|
|
with:
|
|
path: ~/.cache/ms-playwright
|
|
key: playwright-ffmpeg-${{ runner.os }}-${{ steps.playwright-version.outputs.version }}
|
|
|
|
# The runner's Chrome is an apt package, so its real library dependencies are
|
|
# already satisfied; all `install-deps` adds here are optional CJK/Thai/Cyrillic
|
|
# font packages (~21MB from azure.archive.ubuntu.com). Nothing in CI asserts on
|
|
# them — visual baselines are opt-in via E2E_VISUAL_SNAPSHOTS — so a stalled
|
|
# Ubuntu mirror must never be able to fail the suite.
|
|
- name: Install optional Playwright font dependencies (best effort)
|
|
timeout-minutes: 4
|
|
continue-on-error: true
|
|
run: .github/scripts/install-playwright-fonts.sh
|
|
|
|
- name: Run full mock-LLM Tier-1 e2e
|
|
if: matrix.suite == 'full'
|
|
env:
|
|
CI: 'true'
|
|
E2E_SKIP: ${{ needs.codegraph_select.outputs.e2e_skip }}
|
|
run: |
|
|
set +e
|
|
# Graduated-spec skipping (dark until repo var CODEGRAPH_E2E_SKIP=on upstream): subtract
|
|
# the earned skips from a run list derived from the tree itself, so an unknown or stale
|
|
# name in the skip list simply matches nothing. If subtraction would drop everything —
|
|
# or drops nothing — run the full shard exactly as before. Skipped specs still execute
|
|
# post-merge in every full-suite vote run, which is the net that catches a wrong skip.
|
|
RUN_ARGS=""
|
|
if [ -n "$E2E_SKIP" ]; then
|
|
KEEP=""; DROP=0
|
|
for spec in $(git ls-files 'e2e/specs/mock/*.spec.ts' 'e2e/specs/mock/**/*.spec.ts' | sed 's|^e2e/||' | sort -u); do
|
|
case "$spec" in *" "*) KEEP="$KEEP $spec"; continue;; esac
|
|
case " $E2E_SKIP " in
|
|
*" $spec "*) DROP=$((DROP+1));;
|
|
*) KEEP="$KEEP $spec";;
|
|
esac
|
|
done
|
|
if [ "$DROP" -gt 0 ] && [ -n "$KEEP" ]; then
|
|
RUN_ARGS="$KEEP"
|
|
echo "codegraph-e2e-skip: dropped $DROP graduated specs from this shard's pool"
|
|
fi
|
|
fi
|
|
set -e
|
|
npx playwright test --config=e2e/playwright.config.mock.ts --shard=${{ matrix.shard }} $RUN_ARGS
|
|
|
|
- name: Run Redis stream transport e2e
|
|
if: matrix.suite == 'transport'
|
|
run: npx playwright test --config=e2e/playwright.config.redis.ts --shard=${{ matrix.shard }}
|
|
env:
|
|
CI: 'true'
|
|
|
|
- name: Run registered-email change e2e
|
|
if: matrix.email_change == 'true'
|
|
run: npx playwright test --config=e2e/playwright.config.email.ts
|
|
env:
|
|
CI: 'true'
|
|
|
|
- name: Run disabled registered-email change e2e
|
|
if: matrix.email_change == 'true'
|
|
run: npx playwright test --config=e2e/playwright.config.email.ts
|
|
env:
|
|
CI: 'true'
|
|
E2E_ALLOW_EMAIL_CHANGE: 'false'
|
|
|
|
- name: Upload Playwright HTML report
|
|
if: ${{ !cancelled() }}
|
|
uses: actions/upload-artifact@v6
|
|
with:
|
|
name: playwright-report-${{ matrix.artifact }}
|
|
path: e2e/playwright-report/**
|
|
retention-days: 7
|
|
if-no-files-found: ignore
|
|
|
|
- name: Upload traces & screenshots
|
|
if: failure()
|
|
uses: actions/upload-artifact@v6
|
|
with:
|
|
name: playwright-test-results-${{ matrix.artifact }}
|
|
path: e2e/specs/.test-results/**
|
|
retention-days: 7
|
|
if-no-files-found: ignore
|
|
|
|
- name: Run first-turn title ownership regressions
|
|
if: ${{ !cancelled() && matrix.stream_store == 'memory' && startsWith(matrix.shard, '1/') }}
|
|
env:
|
|
CI: 'true'
|
|
E2E_TITLE_CONVO: 'true'
|
|
run: >-
|
|
npx playwright test --config=e2e/playwright.config.mock.ts
|
|
conversation-management.spec.ts --grep 'first-turn|final-title polling' --reporter=line
|
|
--output=e2e/specs/.test-results/title-ownership
|
|
|
|
- name: Upload title ownership diagnostics
|
|
if: ${{ always() && matrix.stream_store == 'memory' && startsWith(matrix.shard, '1/') }}
|
|
uses: actions/upload-artifact@v6
|
|
with:
|
|
name: playwright-title-ownership-${{ matrix.artifact }}
|
|
path: e2e/specs/.test-results/title-ownership/**
|
|
retention-days: 7
|
|
if-no-files-found: ignore
|
|
|
|
mcp_tool_list_changed:
|
|
name: MCP list_changed (replica count ${{ matrix.replicas }})
|
|
needs: [codegraph_select]
|
|
if: >-
|
|
!cancelled() &&
|
|
needs.codegraph_select.outputs.mcp_run != 'false' &&
|
|
(github.event_name == 'schedule' ||
|
|
github.event_name == 'workflow_dispatch' ||
|
|
(github.event_name == 'pull_request' &&
|
|
github.event.pull_request != null &&
|
|
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association) ||
|
|
github.event.pull_request.head.repo.full_name == github.repository)))
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
replicas: [1, 2]
|
|
env:
|
|
CI: 'true'
|
|
E2E_CHROMIUM_CHANNEL: chrome
|
|
E2E_MCP_LIST_CHANGED: 'true'
|
|
E2E_REPLICAS: ${{ matrix.replicas }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Use Node.js 24.16.0
|
|
uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '24.16.0'
|
|
|
|
- name: Restore node_modules cache
|
|
id: cache-node-modules
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: |
|
|
node_modules
|
|
client/node_modules
|
|
packages/client/node_modules
|
|
packages/chat/node_modules
|
|
packages/data-provider/node_modules
|
|
packages/data-schemas/node_modules
|
|
packages/api/node_modules
|
|
api/node_modules
|
|
key: node-modules-e2e-${{ runner.os }}-24.16.0-${{ hashFiles('package-lock.json') }}
|
|
|
|
- name: Install dependencies
|
|
if: steps.cache-node-modules.outputs.cache-hit != 'true'
|
|
run: npm ci
|
|
|
|
- name: Build e2e dependencies
|
|
run: npm run e2e:prepare
|
|
|
|
- name: Verify Chrome is present
|
|
run: google-chrome --version
|
|
|
|
# ffmpeg for retry video — see the note in the e2e_shards job.
|
|
- name: Resolve Playwright version
|
|
id: playwright-version
|
|
run: |
|
|
version=$(node -p "require('./package-lock.json').packages['node_modules/playwright-core'].version")
|
|
echo "version=${version}" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Restore Playwright ffmpeg cache
|
|
id: cache-ffmpeg
|
|
uses: actions/cache/restore@v5
|
|
with:
|
|
path: ~/.cache/ms-playwright
|
|
key: playwright-ffmpeg-${{ runner.os }}-${{ steps.playwright-version.outputs.version }}
|
|
|
|
- name: Install Playwright ffmpeg (best effort)
|
|
id: install-ffmpeg
|
|
if: steps.cache-ffmpeg.outputs.cache-hit != 'true'
|
|
timeout-minutes: 3
|
|
continue-on-error: true
|
|
run: |
|
|
timeout -k 10 60 npx playwright install ffmpeg
|
|
.github/scripts/verify-playwright-ffmpeg.sh
|
|
|
|
- name: Save Playwright ffmpeg cache
|
|
if: steps.install-ffmpeg.outcome == 'success'
|
|
continue-on-error: true
|
|
uses: actions/cache/save@v5
|
|
with:
|
|
path: ~/.cache/ms-playwright
|
|
key: playwright-ffmpeg-${{ runner.os }}-${{ steps.playwright-version.outputs.version }}
|
|
|
|
# This job deliberately skips the optional font install: its bounded
|
|
# Playwright apt process can outlive the wrapper on a slow mirror and
|
|
# retain the package-manager lock needed by the required Redis install.
|
|
# The MCP suite does not enable visual snapshot assertions.
|
|
|
|
# Redis is a hard requirement for this job, so this step stays fatal.
|
|
- name: Install Redis runtime dependencies
|
|
timeout-minutes: 5
|
|
run: |
|
|
sudo apt-get -o DPkg::Lock::Timeout=300 update
|
|
sudo apt-get -o DPkg::Lock::Timeout=300 install -y redis-server redis-tools
|
|
|
|
- name: Start standalone Redis and Redis Cluster
|
|
run: |
|
|
redis-server --daemonize yes --port 6379
|
|
redis-cli -p 6379 ping
|
|
chmod +x redis-config/start-cluster.sh redis-config/stop-cluster.sh
|
|
./redis-config/start-cluster.sh
|
|
redis-cli -p 7001 cluster info
|
|
|
|
- name: Test MCP notifications with in-memory cache
|
|
env:
|
|
E2E_STREAM_STORE: memory
|
|
run: >-
|
|
npx playwright test --config=e2e/playwright.config.mock.ts
|
|
mcp-tool-list-changed.spec.ts --retries=0
|
|
|
|
- name: Test MCP notifications with standalone Redis cache
|
|
env:
|
|
E2E_STREAM_STORE: redis
|
|
REDIS_URI: redis://127.0.0.1:6379
|
|
run: >-
|
|
npx playwright test --config=e2e/playwright.config.mock.ts
|
|
mcp-tool-list-changed.spec.ts --retries=0
|
|
|
|
- name: Test MCP notifications with Redis Cluster cache
|
|
env:
|
|
E2E_STREAM_STORE: redis-cluster
|
|
REDIS_URI: redis://127.0.0.1:7001,redis://127.0.0.1:7002,redis://127.0.0.1:7003
|
|
run: >-
|
|
npx playwright test --config=e2e/playwright.config.mock.ts
|
|
mcp-tool-list-changed.spec.ts --retries=0
|
|
|
|
- name: Upload Playwright HTML report
|
|
if: ${{ !cancelled() }}
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: playwright-mcp-list-changed-${{ matrix.replicas }}-replicas
|
|
path: e2e/playwright-report/**
|
|
retention-days: 7
|
|
if-no-files-found: ignore
|
|
|
|
- name: Upload traces & screenshots
|
|
if: failure()
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: playwright-mcp-list-changed-results-${{ matrix.replicas }}-replicas
|
|
path: e2e/specs/.test-results/**
|
|
retention-days: 7
|
|
if-no-files-found: ignore
|
|
|
|
- name: Stop Redis processes
|
|
if: always()
|
|
run: |
|
|
./redis-config/stop-cluster.sh || true
|
|
redis-cli -p 6379 shutdown || true
|
|
|
|
mcp_apps:
|
|
name: MCP Apps (browser and server)
|
|
# Always run the dedicated fixture on eligible PRs, schedule, and manual runs. The regular
|
|
# mock shards skip it unless E2E_MCP_APPS=true; codegraph gating must not omit this lane.
|
|
if: >-
|
|
github.event_name == 'schedule' ||
|
|
github.event_name == 'workflow_dispatch' ||
|
|
(github.event_name == 'pull_request' &&
|
|
github.event.pull_request != null &&
|
|
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association) ||
|
|
github.event.pull_request.head.repo.full_name == github.repository))
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 45
|
|
env:
|
|
CI: 'true'
|
|
E2E_CHROMIUM_CHANNEL: chrome
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
|
|
- name: Use Node.js 24.16.0
|
|
uses: actions/setup-node@v5
|
|
with:
|
|
node-version: '24.16.0'
|
|
|
|
- name: Restore node_modules cache
|
|
id: cache-node-modules
|
|
uses: actions/cache@v5
|
|
with:
|
|
path: |
|
|
node_modules
|
|
client/node_modules
|
|
packages/client/node_modules
|
|
packages/chat/node_modules
|
|
packages/data-provider/node_modules
|
|
packages/data-schemas/node_modules
|
|
packages/api/node_modules
|
|
api/node_modules
|
|
key: node-modules-e2e-${{ runner.os }}-24.16.0-${{ hashFiles('package-lock.json') }}
|
|
|
|
- name: Install dependencies
|
|
if: steps.cache-node-modules.outputs.cache-hit != 'true'
|
|
run: npm ci
|
|
|
|
- name: Verify Chrome is present
|
|
run: google-chrome --version
|
|
|
|
- name: Check MCP Apps result gating and same-repository PR eligibility
|
|
run: node --test e2e/mcp-apps/results.test.mjs e2e/mcp-apps/workflow.test.mjs
|
|
|
|
- name: Check real SDK server fixture
|
|
run: node --test e2e/mcp-apps/server.test.mjs
|
|
|
|
# The runner builds once, then runs all four browser policy phases against one disposable
|
|
# MongoDB. It also rejects a Playwright-successful phase if every test was skipped.
|
|
- name: Run MCP Apps browser fixture (enabled, disabled, omitted, quota)
|
|
run: node e2e/mcp-apps/run.mjs
|
|
|
|
- name: Upload per-phase Playwright reports
|
|
if: ${{ !cancelled() }}
|
|
uses: actions/upload-artifact@v6
|
|
with:
|
|
name: playwright-mcp-apps-reports
|
|
path: |
|
|
e2e/playwright-report/mcp-apps-*/**
|
|
e2e/.generated/mcp-apps-result-*.json
|
|
retention-days: 7
|
|
if-no-files-found: warn
|
|
|
|
- name: Upload MCP Apps traces and screenshots
|
|
if: failure()
|
|
uses: actions/upload-artifact@v6
|
|
with:
|
|
name: playwright-mcp-apps-results
|
|
path: e2e/specs/.test-results/mcp-apps-*/**
|
|
retention-days: 7
|
|
if-no-files-found: ignore
|
|
|
|
e2e:
|
|
name: e2e
|
|
# `!cancelled()`, not `always()`: the gate has to survive a failed shard to report it, but a
|
|
# run superseded by `cancel-in-progress` has nothing to adjudicate. Under `always()` it was
|
|
# still dispatched onto a fresh runner while every other job went `cancelled`, then read
|
|
# `needs.e2e_shards.result == 'cancelled'` and exited 1 — so each superseded push left this
|
|
# required check red instead of cancelled. Matches all three lanes it gates.
|
|
if: >-
|
|
!cancelled() &&
|
|
(github.event_name == 'schedule' ||
|
|
github.event_name == 'workflow_dispatch' ||
|
|
(github.event_name == 'pull_request' &&
|
|
github.event.pull_request != null &&
|
|
(contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association) ||
|
|
github.event.pull_request.head.repo.full_name == github.repository)))
|
|
needs: [codegraph_select, e2e_shards, mcp_tool_list_changed, mcp_apps]
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
# A codegraph-skipped MCP lane reports `skipped`; that is a decision, not a failure.
|
|
- name: Verify every Playwright job passed
|
|
if: >-
|
|
needs.e2e_shards.result != 'success' ||
|
|
needs.mcp_apps.result != 'success' ||
|
|
(needs.mcp_tool_list_changed.result != 'success' &&
|
|
!(needs.mcp_tool_list_changed.result == 'skipped' && needs.codegraph_select.outputs.mcp_run == 'false'))
|
|
run: exit 1
|