🧰 ci: Provision Pinned Redis Binaries Without a Distro Mirror (#18055)

* 🧰 ci: Provision Pinned Redis Binaries Without a Distro Mirror

The MCP list_changed jobs and Cache Integration Tests installed redis-server and redis-tools with apt-get at test time. When the runners could not resolve the Ubuntu mirrors, apt-get update spent 3m20s giving up, the install stalled into the step timeout, and the next step found no redis-cli. The version was whatever the mirror served that day.

A composite action now restores redis-server and redis-cli 8.0.5 from the Actions cache and, on a miss, builds them once from the pinned upstream commit with bounded retries. The step fails with one clear error when Redis cannot be provisioned.

* 🔦 ci: Keep Lighthouse Running When the Ubuntu Mirror Is Unreachable

The Lighthouse job installed its pinned Chromium with `playwright install --with-deps`, which runs apt-get first. With the Ubuntu mirrors unresolvable, apt exited 100 and the job failed before any audit ran.

On the runner every library Chromium links is already present with the preinstalled Chrome; the only packages apt added were nine optional fonts. The browser download stays fatal and keeps the lockfile-pinned build, while the system packages go through the bounded best-effort script the Playwright jobs already use.

* ⏱️ ci: Give the Cold Redis Build Room on a Slow Runner

The first hosted runs built Redis in 49, 58 and 103 seconds. A 180 second cap left the slowest of those less than twice its own time, so a slower runner would have failed a job for being slow. The build may now take 300 seconds and the calling steps eight minutes; an unreachable host still fails within 98 seconds. The ready line reports the fetch and the build separately.
This commit is contained in:
Danny Avila 2026-11-11 10:29:25 -05:00 • committed by GitHub
parent b1591d291b
commit bb3cac9d95
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 703 additions and 36 deletions

1
.github/CODEOWNERS vendored
View file

@ -2,4 +2,5 @@
/.github/CODEOWNERS @danny-avila
/.github/workflows/ @danny-avila
/.github/scripts/ @danny-avila
/.github/actions/ @danny-avila
/.github/MAIN_PROMOTION.md @danny-avila

View file

@ -0,0 +1,39 @@
name: Redis binaries
description: >-
Puts redis-server and redis-cli at the Redis version pinned in pin.json on PATH without a
package manager: restores them from the Actions cache and, when the cache has none for this
pin and runner OS release, builds them once from the pinned upstream commit. Use after Node.js
is set up and before anything starts Redis or calls redis-cli.
runs:
using: composite
steps:
- name: Resolve the pinned Redis build
id: pin
shell: bash
run: node "$GITHUB_ACTION_PATH/provision.mjs" key
# No restore-keys: binaries of another pin or another OS release are never a starting point.
- name: Restore Redis binaries
id: cache
uses: actions/cache/restore@v5
with:
path: ~/.cache/redis-binaries
key: ${{ steps.pin.outputs.key }}
- name: Provision redis-server and redis-cli
shell: bash
env:
REDIS_BINARIES_CACHE_HIT: ${{ steps.cache.outputs.cache-hit }}
run: node "$GITHUB_ACTION_PATH/provision.mjs"
# Saved here rather than in a post-job step: only binaries that were just proven to run are
# promoted into the cache every later job restores, and a failing test run still leaves
# them behind for the next one.
- name: Save Redis binaries
if: steps.cache.outputs.cache-hit != 'true'
continue-on-error: true
uses: actions/cache/save@v5
with:
path: ~/.cache/redis-binaries
key: ${{ steps.cache.outputs.cache-primary-key }}

View file

@ -0,0 +1,6 @@
{
"version": "8.0.5",
"commit": "214a4905c3edb6b69ac9a9c5d769b7d3fb81ed54",
"repository": "https://github.com/redis/redis.git",
"flags": ["BUILD_TLS=no", "USE_SYSTEMD=no"]
}

View file

@ -0,0 +1,316 @@
#!/usr/bin/env node
/**
* Puts `redis-server` and `redis-cli` at the pinned Redis version into
* `~/.cache/redis-binaries/bin`, proves they run, and adds that directory to
* PATH for the steps that follow.
*
* Nothing here talks to a package manager. A job that installed Redis with
* apt needed the distro mirror to resolve at test time, and got whichever
* version the mirror served that day. The binaries are instead restored from
* the Actions cache; when the cache has none that match the pin, they are
* built once from the pinned upstream commit. That fetch goes to the host the
* job's own checkout already depended on, and git only accepts an object
* whose content hashes to the commit asked for, so the pin is the checksum.
*
* `pin.json` holds everything that decides which binaries come out: version,
* commit, repository and make flags. `key` prints the cache key derived from
* it, from this file and from the runner's OS release, because a binary
* linked against one release's libc is not trusted on another.
*
* Exits non-zero, with one `::error` line, when no runnable pinned binaries
* could be put in place: a later step never meets a missing `redis-cli`.
*/
import os from 'node:os';
import path from 'node:path';
import { createHash } from 'node:crypto';
import { fileURLToPath } from 'node:url';
import { spawnSync } from 'node:child_process';
import { setTimeout as sleep } from 'node:timers/promises';
import {
appendFileSync,
chmodSync,
closeSync,
copyFileSync,
existsSync,
mkdirSync,
mkdtempSync,
openSync,
readFileSync,
rmSync,
} from 'node:fs';
const SCRIPT = fileURLToPath(import.meta.url);
const PIN_FILE = path.join(path.dirname(SCRIPT), 'pin.json');
const BINARIES = ['redis-server', 'redis-cli'];
const HOME_CACHE = path.join(os.homedir(), '.cache', 'redis-binaries');
const BIN = path.join(HOME_CACHE, 'bin');
/**
* An unreachable host fails within 30 + 2 + 30 + 6 + 30 s. Hosted runners
* needed 49 to 103 s for the whole cold path, so the build may take about
* three times the slowest of those; fetch and build together stay inside an
* eight-minute step.
*/
const FETCH_TIMEOUT_MS = 30_000;
const FETCH_BACKOFF_MS = [2_000, 6_000];
const FETCH_ATTEMPTS = FETCH_BACKOFF_MS.length + 1;
const BUILD_TIMEOUT_MS = 300_000;
const VERSION_TIMEOUT_MS = 10_000;
const BUILD_LOG_TAIL = 40;
/** A workflow command ends at the first newline, so a multi-line message is folded into one. */
const headline = (error) =>
String(error?.message ?? error)
.split(/\s*\n\s*/)
.filter(Boolean)
.join(' ');
function readPin() {
const pin = JSON.parse(readFileSync(PIN_FILE, 'utf8'));
const valid =
typeof pin.version === 'string' &&
/^[0-9a-f]{40}$/.test(pin.commit) &&
typeof pin.repository === 'string' &&
Array.isArray(pin.flags);
if (!valid) {
throw new Error(`${PIN_FILE} must name a version, a full commit SHA, a repository and flags`);
}
return pin;
}
function fail(error) {
console.error(error);
console.error(`::error title=Redis binaries::${headline(error)}`);
process.exit(1);
}
function loadPin() {
try {
return readPin();
} catch (error) {
return fail(error);
}
}
const PIN = loadPin();
const SHORT_COMMIT = PIN.commit.slice(0, 8);
/** Best effort: a compiler orphaned by a timeout may still be writing into the directory. */
function discard(directory) {
try {
rmSync(directory, { recursive: true, force: true });
} catch {
return;
}
}
/** Runs a command to completion; throws with the reason when it fails, is missing or times out. */
function run(command, args, options = {}) {
const result = spawnSync(command, args, {
encoding: 'utf8',
killSignal: 'SIGKILL',
...options,
});
if (result.status === 0) {
return result.stdout ?? '';
}
const timedOut = result.error?.code === 'ETIMEDOUT';
const reason = timedOut
? `timed out after ${options.timeout / 1000}s`
: result.error?.message ||
result.stderr?.trim() ||
`exit ${result.status ?? result.signal ?? 'unknown'}`;
throw new Error(`${command} ${args[0]}: ${reason}`);
}
function osRelease() {
if (process.platform !== 'linux') {
return `${process.platform}-${os.release().split('.')[0]}`;
}
try {
const release = readFileSync('/etc/os-release', 'utf8');
const field = (name) => release.match(new RegExp(`^${name}="?([^"\\n]+)"?$`, 'm'))?.[1];
return `${field('ID') ?? 'linux'}-${field('VERSION_ID') ?? os.release()}`;
} catch {
return `linux-${os.release()}`;
}
}
function cacheKey() {
const recipe = createHash('sha256')
.update(readFileSync(SCRIPT))
.update(readFileSync(PIN_FILE))
.digest('hex')
.slice(0, 16);
return `redis-binaries-${osRelease()}-${process.arch}-${PIN.version}-${recipe}`;
}
/** The version line of one cached binary; throws unless it runs and is the pinned build. */
function identify(name) {
const binary = path.join(BIN, name);
const line = run(binary, ['--version'], { timeout: VERSION_TIMEOUT_MS }).split('\n', 1)[0].trim();
const match =
name === 'redis-server'
? line.match(/^Redis server v=(\S+) sha=([0-9a-f]{8})/)
: line.match(/^redis-cli (\S+) \(git:([0-9a-f]{8})\)/);
if (match?.[1] !== PIN.version || match?.[2] !== SHORT_COMMIT) {
throw new Error(
`${binary} is not Redis ${PIN.version} built from ${SHORT_COMMIT}: ${line || 'no version line'}`,
);
}
return line;
}
function git(args, directory, timeout) {
return run('git', args, {
cwd: directory,
timeout,
env: { ...process.env, GIT_TERMINAL_PROMPT: '0' },
});
}
/** Each attempt gets its own repository, so a fetch killed by its timeout leaves no lock behind. */
function fetchCommit(root, attempt) {
const directory = path.join(root, `attempt-${attempt}`);
mkdirSync(directory);
git(['init', '--quiet'], directory);
git(
['fetch', '--quiet', '--depth', '1', '--no-tags', PIN.repository, PIN.commit],
directory,
FETCH_TIMEOUT_MS,
);
git(
['-c', 'advice.detachedHead=false', 'checkout', '--quiet', '--detach', 'FETCH_HEAD'],
directory,
);
const head = git(['rev-parse', 'HEAD'], directory).trim();
if (head !== PIN.commit) {
throw new Error(`${PIN.repository} answered with ${head}, not the pinned ${PIN.commit}`);
}
return directory;
}
async function checkout(root, attempt = 1) {
try {
return fetchCommit(root, attempt);
} catch (error) {
if (attempt === FETCH_ATTEMPTS) {
throw new Error(
`Could not fetch Redis ${PIN.version} (${PIN.commit}) from ${PIN.repository} after ${FETCH_ATTEMPTS} attempts: ${headline(error)}`,
);
}
const delay = FETCH_BACKOFF_MS[attempt - 1];
console.log(
`::warning::${headline(error)}. Retrying in ${delay / 1000}s (attempt ${attempt + 1} of ${FETCH_ATTEMPTS}).`,
);
await sleep(delay);
}
return checkout(root, attempt + 1);
}
/** Compiler output goes to a file; only its tail is shown, and only when the build fails. */
function build(directory) {
const log = path.join(directory, 'redis-binaries-build.log');
const output = openSync(log, 'w');
const jobs = String(os.availableParallelism());
try {
run('make', ['-C', 'src', '-j', jobs, ...PIN.flags, ...BINARIES], {
cwd: directory,
timeout: BUILD_TIMEOUT_MS,
stdio: ['ignore', output, output],
});
} catch (error) {
const tail = readFileSync(log, 'utf8').trimEnd().split('\n').slice(-BUILD_LOG_TAIL);
console.log(`::group::Last ${tail.length} lines of the Redis build`);
console.log(tail.join('\n'));
console.log('::endgroup::');
throw new Error(`Building Redis ${PIN.version} failed: ${headline(error)}`);
} finally {
closeSync(output);
}
}
function install(directory) {
discard(BIN);
mkdirSync(BIN, { recursive: true });
for (const name of BINARIES) {
copyFileSync(path.join(directory, 'src', name), path.join(BIN, name));
chmodSync(path.join(BIN, name), 0o755);
}
}
/** Nothing unusable stays behind: a failed build leaves the cache directory without binaries. */
async function buildFromSource() {
const started = Date.now();
const root = mkdtempSync(path.join(process.env.RUNNER_TEMP || os.tmpdir(), 'redis-source-'));
try {
const directory = await checkout(root);
const fetched = Date.now();
build(directory);
install(directory);
const lines = BINARIES.map(identify);
const seconds = (from, to) => Math.round((to - from) / 1000);
const took = `fetch ${seconds(started, fetched)}s, build ${seconds(fetched, Date.now())}s`;
return { lines, source: `built from ${PIN.repository} at ${SHORT_COMMIT}: ${took}` };
} catch (error) {
discard(BIN);
throw error;
} finally {
discard(root);
}
}
async function provision() {
if (!BINARIES.some((name) => existsSync(path.join(BIN, name)))) {
console.log(`No Redis binaries in ${BIN}; building ${PIN.version} from ${SHORT_COMMIT}.`);
return buildFromSource();
}
try {
return { lines: BINARIES.map(identify), source: 'already present' };
} catch (error) {
console.log(`Rebuilding Redis ${PIN.version}: ${headline(error)}`);
return buildFromSource();
}
}
function exportPath() {
if (!process.env.GITHUB_PATH) {
console.log(` GITHUB_PATH is not set; add ${BIN} to PATH yourself`);
return;
}
appendFileSync(process.env.GITHUB_PATH, `${BIN}\n`);
}
async function main() {
const result = await provision();
console.log(`Redis ${PIN.version} ready in ${BIN} (${result.source})`);
for (const line of result.lines) {
console.log(` ${line}`);
}
exportPath();
if (result.source !== 'already present' && process.env.REDIS_BINARIES_CACHE_HIT === 'true') {
console.log(
'::warning::The restored Redis binary cache had no runnable pinned build, and an exact cache hit is never re-saved: every job rebuilds Redis until the cache entry is deleted or the pin changes.',
);
}
}
function printKey() {
const key = cacheKey();
console.log(key);
if (process.env.GITHUB_OUTPUT) {
appendFileSync(process.env.GITHUB_OUTPUT, `key=${key}\n`);
}
}
const command = process.argv[2] ?? 'provision';
const commands = { key: printKey, provision: main };
Promise.resolve()
.then(() => {
if (!Object.hasOwn(commands, command)) {
throw new Error(`Unknown command "${command}"; expected "provision" or "key"`);
}
return commands[command]();
})
.catch(fail);

View file

@ -0,0 +1,301 @@
import os from 'node:os';
import path from 'node:path';
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { fileURLToPath } from 'node:url';
import { spawnSync } from 'node:child_process';
import {
chmodSync,
copyFileSync,
existsSync,
mkdirSync,
mkdtempSync,
readFileSync,
readdirSync,
rmSync,
writeFileSync,
} from 'node:fs';
/**
* Runs the real script, with real git and real make, against a stand-in for
* the Redis repository: a local repository whose `src/Makefile` "builds"
* `redis-server` and `redis-cli` as shell scripts that print the version lines
* the real binaries print, embedding the commit they were built from the way
* the Redis build does. The script reads the pin from the file beside it, so
* each test runs a copy of it next to a pin for that stand-in. Only the
* network is replaced: a `git` first on PATH counts fetches and can fail the
* first few before handing over to the real one.
*/
const SCRIPT = fileURLToPath(new URL('./provision.mjs', import.meta.url));
const VERSION = '9.9.9';
const posixOnly = {
skip: process.platform === 'win32' && 'the stand-in binaries are shell scripts',
};
const MAKEFILE = `
SHA := $(shell git rev-parse --short=8 HEAD)
VERSION := __VERSION__
redis-server:
\t@printf '#!/bin/sh\\necho "Redis server v=%s sha=%s:0 malloc=libc bits=64 build=tls-%s-systemd-%s"\\n' '$(VERSION)' '$(SHA)' '$(BUILD_TLS)' '$(USE_SYSTEMD)' > $@
\t@chmod +x $@
redis-cli:
\t@printf '#!/bin/sh\\necho "redis-cli %s (git:%s)"\\n' '$(VERSION)' '$(SHA)' > $@
\t@chmod +x $@
`;
const BROKEN_MAKEFILE = `
redis-server redis-cli:
\t@echo "cc: fatal error: no input files" >&2
\t@exit 2
`;
const GIT_SHIM = `#!/bin/sh
if [ "$1" = fetch ]; then
echo fetch >> "$FAKE_FETCH_LOG"
if [ "$(wc -l < "$FAKE_FETCH_LOG")" -le "\${FAKE_FETCH_FAILURES:-0}" ]; then
echo "fatal: unable to access 'https://example.invalid/': Could not resolve host" >&2
exit 128
fi
fi
exec "$REAL_GIT" "$@"
`;
const REAL_GIT = spawnSync('sh', ['-c', 'command -v git'], { encoding: 'utf8' }).stdout.trim();
/** The developer's own git configuration (signing, hooks, templates) stays out of the stand-in. */
const HERMETIC_GIT = { ...process.env, GIT_CONFIG_GLOBAL: '/dev/null', GIT_CONFIG_NOSYSTEM: '1' };
function sh(command, args, cwd) {
const result = spawnSync(command, args, { cwd, encoding: 'utf8', env: HERMETIC_GIT });
assert.equal(result.status, 0, `${command} ${args.join(' ')}: ${result.stderr}`);
return result.stdout.trim();
}
function fixture(t, { makefile = MAKEFILE.replace('__VERSION__', VERSION) } = {}) {
const root = mkdtempSync(path.join(os.tmpdir(), 'redis-binaries-'));
t.after(() => rmSync(root, { recursive: true, force: true }));
const home = path.join(root, 'home');
const action = path.join(root, 'action');
const upstream = path.join(root, 'upstream');
const shim = path.join(root, 'shim');
const bin = path.join(home, '.cache', 'redis-binaries', 'bin');
const fetchLog = path.join(root, 'fetches.log');
const githubPath = path.join(root, 'github-path');
const githubOutput = path.join(root, 'github-output');
for (const directory of [home, action, path.join(upstream, 'src'), shim]) {
mkdirSync(directory, { recursive: true });
}
writeFileSync(path.join(upstream, 'src', 'Makefile'), makefile);
sh('git', ['init', '--quiet', '--initial-branch', 'main'], upstream);
sh('git', ['add', '.'], upstream);
sh(
'git',
['-c', 'user.name=fixture', '-c', 'user.email=fixture@example.invalid', 'commit', '-qm', 'x'],
upstream,
);
const commit = sh('git', ['rev-parse', 'HEAD'], upstream);
writeFileSync(path.join(shim, 'git'), GIT_SHIM);
chmodSync(path.join(shim, 'git'), 0o755);
copyFileSync(SCRIPT, path.join(action, 'provision.mjs'));
const pin = (overrides = {}) =>
writeFileSync(
path.join(action, 'pin.json'),
JSON.stringify({
version: VERSION,
commit,
repository: upstream,
flags: ['BUILD_TLS=no', 'USE_SYSTEMD=no'],
...overrides,
}),
);
pin();
const run = (args = [], env = {}) => {
const result = spawnSync(process.execPath, [path.join(action, 'provision.mjs'), ...args], {
cwd: root,
encoding: 'utf8',
env: {
...process.env,
HOME: home,
PATH: `${shim}${path.delimiter}${process.env.PATH}`,
RUNNER_TEMP: root,
REAL_GIT,
FAKE_FETCH_LOG: fetchLog,
GITHUB_PATH: githubPath,
GITHUB_OUTPUT: githubOutput,
REDIS_BINARIES_CACHE_HIT: '',
...env,
},
});
return { status: result.status, output: `${result.stdout}${result.stderr}` };
};
const place = (version, sha) => {
mkdirSync(bin, { recursive: true });
const lines = {
'redis-server': `Redis server v=${version} sha=${sha}:0 malloc=libc bits=64 build=old`,
'redis-cli': `redis-cli ${version} (git:${sha})`,
};
for (const [name, line] of Object.entries(lines)) {
writeFileSync(path.join(bin, name), `#!/bin/sh\necho "${line}"\n`);
chmodSync(path.join(bin, name), 0o755);
}
};
const read = (file) => (existsSync(file) ? readFileSync(file, 'utf8') : '');
const fetches = () => read(fetchLog).split('\n').filter(Boolean).length;
const cached = () => (existsSync(bin) ? readdirSync(bin).sort() : []);
const leftovers = () => readdirSync(root).filter((name) => name.startsWith('redis-source-'));
const exported = () => read(githubPath);
const output = () => read(githubOutput);
return { bin, commit, pin, run, place, fetches, cached, leftovers, exported, output };
}
test('a cold cache is filled by one fetch of the pinned commit and one build', posixOnly, (t) => {
const { bin, commit, run, fetches, cached, leftovers, exported } = fixture(t);
const { status, output } = run();
assert.equal(status, 0, output);
assert.match(
output,
new RegExp(
`Redis 9\\.9\\.9 ready in ${bin} \\(built from .* at ${commit.slice(0, 8)}: fetch \\d+s, build \\d+s\\)`,
),
);
assert.match(output, new RegExp(`Redis server v=9\\.9\\.9 sha=${commit.slice(0, 8)}:0`));
assert.match(output, /build=tls-no-systemd-no/, 'the pinned flags reach make');
assert.match(output, new RegExp(`redis-cli 9\\.9\\.9 \\(git:${commit.slice(0, 8)}\\)`));
assert.match(output, new RegExp(`No Redis binaries in ${bin}; building 9\\.9\\.9 from`));
assert.doesNotMatch(output, /::warning::|::error/);
assert.equal(fetches(), 1);
assert.deepEqual(cached(), ['redis-cli', 'redis-server']);
assert.equal(exported(), `${bin}\n`);
assert.deepEqual(leftovers(), [], 'the source tree is removed once the binaries are in place');
});
test('binaries already in the cache are verified and nothing is fetched', posixOnly, (t) => {
const { bin, run, fetches, exported } = fixture(t);
assert.equal(run().status, 0);
const { status, output } = run([], {
FAKE_FETCH_FAILURES: '99',
REDIS_BINARIES_CACHE_HIT: 'true',
});
assert.equal(status, 0, output);
assert.match(output, /Redis 9\.9\.9 ready in .* \(already present\)/);
assert.doesNotMatch(output, /::warning::/);
assert.equal(fetches(), 1, 'the second run needs no network');
assert.equal(exported(), `${bin}\n${bin}\n`);
});
test('cached binaries of another version or commit are rebuilt, with a warning', posixOnly, (t) => {
const { commit, run, place, fetches } = fixture(t);
for (const [version, sha] of [
['9.9.8', commit.slice(0, 8)],
[VERSION, '0badc0de'],
]) {
place(version, sha);
const { status, output } = run([], { REDIS_BINARIES_CACHE_HIT: 'true' });
assert.equal(status, 0, output);
assert.match(output, /Rebuilding Redis 9\.9\.9: .* is not Redis 9\.9\.9 built from/);
assert.match(output, /\(built from /);
assert.match(output, /::warning::The restored Redis binary cache had no runnable pinned build/);
}
assert.equal(fetches(), 2);
});
test('a fetch that fails once is retried and the build goes ahead', posixOnly, (t) => {
const { run, fetches, cached } = fixture(t);
const { status, output } = run([], { FAKE_FETCH_FAILURES: '1' });
assert.equal(status, 0, output);
assert.match(
output,
/::warning::git fetch: fatal: unable to access .* Retrying in 2s \(attempt 2 of 3\)/,
);
assert.equal(fetches(), 2);
assert.deepEqual(cached(), ['redis-cli', 'redis-server']);
});
test(
'the job fails after three attempts when the pinned commit cannot be fetched',
posixOnly,
(t) => {
const { pin, run, fetches, cached, leftovers, exported } = fixture(t);
const missing = 'f'.repeat(40);
pin({ commit: missing });
const { status, output } = run();
assert.equal(status, 1, output);
assert.match(
output,
new RegExp(
`^::error title=Redis binaries::Could not fetch Redis 9\\.9\\.9 \\(${missing}\\) from .* after 3 attempts: git fetch: `,
'm',
),
);
assert.equal(fetches(), 3, 'bounded, not a loop');
assert.deepEqual(cached(), []);
assert.deepEqual(leftovers(), []);
assert.equal(exported(), '', 'no later step is pointed at binaries that do not exist');
},
);
test('the job fails with the end of the build log when Redis does not compile', posixOnly, (t) => {
const { run, cached, leftovers, exported } = fixture(t, { makefile: BROKEN_MAKEFILE });
const { status, output } = run();
assert.equal(status, 1, output);
assert.match(output, /::group::Last \d+ lines of the Redis build/);
assert.match(output, /cc: fatal error: no input files/);
assert.match(output, /^::error title=Redis binaries::Building Redis 9\.9\.9 failed: make -C: /m);
assert.deepEqual(cached(), []);
assert.deepEqual(leftovers(), []);
assert.equal(exported(), '');
});
test(
'a build that is not the pinned version is rejected and not left in the cache',
posixOnly,
(t) => {
const { run, cached, exported } = fixture(t, {
makefile: MAKEFILE.replace('__VERSION__', '9.9.8'),
});
const { status, output } = run();
assert.equal(status, 1, output);
assert.match(
output,
/^::error title=Redis binaries::.*redis-server is not Redis 9\.9\.9 built from/m,
);
assert.deepEqual(cached(), []);
assert.equal(exported(), '');
},
);
test('the cache key follows the pin and the script, and is stable otherwise', posixOnly, (t) => {
const { commit, pin, run, output } = fixture(t);
const key = () => {
const result = run(['key']);
assert.equal(result.status, 0, result.output);
return result.output.trim();
};
const first = key();
assert.match(first, new RegExp(`^redis-binaries-.+-${process.arch}-9\\.9\\.9-[0-9a-f]{16}$`));
assert.equal(key(), first);
assert.equal(output(), `key=${first}\nkey=${first}\n`);
pin({ flags: ['BUILD_TLS=yes', 'USE_SYSTEMD=no'] });
const flagged = key();
assert.notEqual(flagged, first);
pin({ commit: commit.replace(/.$/, (last) => (last === '0' ? '1' : '0')) });
assert.notEqual(key(), first);
assert.notEqual(key(), flagged);
});
test('a malformed pin or an unknown command fails with one error line', posixOnly, (t) => {
const { pin, run } = fixture(t);
const unknown = run(['install']);
assert.equal(unknown.status, 1, unknown.output);
assert.match(unknown.output, /^::error title=Redis binaries::Unknown command "install"/m);
pin({ commit: 'main' });
const malformed = run();
assert.equal(malformed.status, 1, malformed.output);
assert.match(
malformed.output,
/^::error title=Redis binaries::.*pin\.json must name a version, a full commit SHA/m,
);
});

View file

@ -1,6 +1,7 @@
#!/usr/bin/env bash
#
# Installs Playwright's optional font packages for the `chrome` channel.
# Installs Playwright's optional font packages for the `chrome` channel, or for
# the browser named as the first argument.
#
# The GitHub runner ships Chrome as an apt package, so every library Playwright
# lists is already satisfied by apt itself. The only packages `install-deps` adds
@ -8,6 +9,11 @@
# the one spec that takes screenshots gates the comparison behind
# `E2E_VISUAL_SNAPSHOTS`, which CI never sets.
#
# Lighthouse passes `chromium`: its lockfile-pinned build links the same
# libraries the runner's Chrome already brought in, so on the runner the same
# fonts are all that is missing, and a launch that did lack a library fails in
# the workflow's own browser check rather than here.
#
# Ubuntu's mirrors stall often enough that a hard failure here has repeatedly
# taken down whole e2e runs, so each attempt is capped and a final failure is
# only a warning. The workflow keeps `continue-on-error: true` as a backstop for
@ -15,11 +21,12 @@
set -uo pipefail
readonly BROWSER="${1:-chrome}"
readonly ATTEMPT_TIMEOUT_SECONDS=70
readonly MAX_ATTEMPTS=3
for attempt in $(seq 1 "${MAX_ATTEMPTS}"); do
if timeout "${ATTEMPT_TIMEOUT_SECONDS}" npx playwright install-deps chrome; then
if timeout "${ATTEMPT_TIMEOUT_SECONDS}" npx playwright install-deps "${BROWSER}"; then
exit 0
fi
echo "::warning::playwright install-deps attempt ${attempt}/${MAX_ATTEMPTS} failed or timed out"

View file

@ -20,6 +20,7 @@ on:
- 'packages/data-schemas/package.json'
- 'redis-config/**'
- '.github/workflows/cache-integration-tests.yml'
- '.github/actions/redis-binaries/**'
- '!**.md'
permissions:
@ -44,30 +45,16 @@ jobs:
with:
node-version: '24.22.1'
- name: Install Redis tools
timeout-minutes: 10
run: |
# Same runner apt contention that broke the MCP job in
# playwright-mock.yml: apt-daily/unattended-upgrades hold
# /var/lib/apt/lists/lock at boot. Without a step timeout this hung
# until the job-level one fired, taking the whole leg with it.
sudo systemctl stop apt-daily.service apt-daily-upgrade.service \
unattended-upgrades.service 2>/dev/null || true
sudo systemctl kill --kill-who=all apt-daily.service \
apt-daily-upgrade.service 2>/dev/null || true
- name: Test Redis binary provisioning
run: node --test .github/actions/redis-binaries/provision.test.mjs
apt_with_lock_wait() {
for attempt in $(seq 1 30); do
if sudo apt-get -o DPkg::Lock::Timeout=60 "$@"; then
return 0
fi
echo "apt-get $1 could not take the lock (attempt ${attempt}/30), retrying"
sleep 10
done
return 1
}
apt_with_lock_wait update
apt_with_lock_wait install -y redis-server redis-tools
# The pinned redis-server and redis-cli come from the Actions cache, or
# are built from the pinned commit on a miss. apt is not involved: its
# lock contention and an unreachable Ubuntu mirror have each taken this
# job down before a single test ran.
- name: Install Redis tools
timeout-minutes: 8
uses: ./.github/actions/redis-binaries
- name: Start Single Redis Instance
run: |

View file

@ -77,7 +77,16 @@ jobs:
api/node_modules
key: ${{ steps.cache-node-modules.outputs.cache-primary-key }}
- name: Install lockfile-pinned Chromium
run: npx playwright install --with-deps --no-shell chromium
run: npx playwright install --no-shell chromium
# The audit keeps the lockfile-pinned Chromium build, so its download above stays fatal.
# Its system packages do not: the runner's own Chrome already brings every library
# Chromium links, and what `--with-deps` installed on top were nine optional font packages
# (22MB from the Ubuntu mirror) that no budget reads. With the mirror unreachable, apt
# failed the combined command before any audit ran.
- name: Install optional Chromium system packages (best effort)
timeout-minutes: 4
continue-on-error: true
run: .github/scripts/install-playwright-fonts.sh chromium
- name: Select Lighthouse browser
run: |
chrome=$(node -p "require('@playwright/test').chromium.executablePath()")

View file

@ -518,22 +518,23 @@ jobs:
path: ~/.cache/ms-playwright
key: playwright-ffmpeg-${{ runner.os }}-${{ steps.playwright-version.outputs.version }}
# This job deliberately skips the optional font install: its bounded
# Playwright apt process can outlive the wrapper on a slow mirror and
# retain the package-manager lock needed by the required Redis install.
# The MCP suite does not enable visual snapshot assertions.
# This job deliberately skips the optional font install: the MCP suite
# does not enable visual snapshot assertions, so nothing here needs apt.
# Redis is a hard requirement for this job, so this step stays fatal.
# Redis is a hard requirement for this job, so this step stays fatal. The
# action puts the pinned redis-server and redis-cli on PATH from the
# Actions cache, or builds them from the pinned commit on a miss; no
# distro mirror has to resolve for this job to get Redis.
- name: Install Redis runtime dependencies
timeout-minutes: 5
run: |
sudo apt-get -o DPkg::Lock::Timeout=300 update
sudo apt-get -o DPkg::Lock::Timeout=300 install -y redis-server redis-tools
timeout-minutes: 8
uses: ./.github/actions/redis-binaries
# A daemonized redis-server returns before it opens its port, so the
# first ping waits for it instead of racing it.
- name: Start standalone Redis and Redis Cluster
run: |
redis-server --daemonize yes --port 6379
redis-cli -p 6379 ping
timeout 30 bash -c 'until redis-cli -p 6379 ping; do sleep 0.5; done'
chmod +x redis-config/start-cluster.sh redis-config/stop-cluster.sh
./redis-config/start-cluster.sh
redis-cli -p 7001 cluster info