LibreChat/e2e/playwright.config.redis.ts
Danny Avila c6b7a5ac12
🎲 ci: Fail Playwright Runs When a Test Passes Only on a Retry (#18051)
Playwright reports a test that fails and then passes on a retry as "flaky" and exits 0, so a
regression that fails two attempts in three merges green. Set `failOnFlakyTests` in CI on the
mock profile (inherited by the Redis transport, registered-email and other derived profiles) and
on the base profile. Retry counts are unchanged: the later attempts still record the trace and
video and show whether a failure repeats.

A run failed this way still prints "N flaky" beside a passed count, so add a reporter that names
each retried pass in the job log and the job summary and says that it is what failed the run.
2026-11-11 10:31:06 -05:00

33 lines
1.4 KiB
TypeScript

import { defineConfig } from '@playwright/test';
import mockConfig from './playwright.config.mock';
/** Scenarios that exercise Redis-backed streams or shared cache invalidation. */
export default defineConfig({
...mockConfig,
/**
* Every stream event crosses a real Redis round-trip on this lane, so waits that
* sit comfortably against the in-memory store run much closer to their budget —
* pause/resume and rehydrate paths most of all, since they replay a whole job.
* The suite is serial (`workers: 1`), so this is per-operation latency rather
* than contention, and it is why this lane reports flaky runs the memory shards
* never produce. Give it a larger default assertion budget and one more retry. A
* retried pass still fails CI (`failOnFlakyTests` in the mock profile); the extra
* attempt only shows how often the failure repeats.
*/
retries: process.env.CI ? 3 : 0,
expect: { ...mockConfig.expect, timeout: 20_000 },
testMatch: [
/completion\.spec\.ts/,
/deferred-tools-hitl\.spec\.ts/,
/model-spec-icons\.spec\.ts/,
/steering\.spec\.ts/,
/steering-escalation\.spec\.ts/,
/streaming\.spec\.ts/,
/subagent-activity\.spec\.ts/,
/thread-fold\.spec\.ts/,
/tool-approvals\.spec\.ts/,
/usage\.spec\.ts/,
/scenarios\/redis-stream\.spec\.ts/,
/scenarios\/reset-password\.spec\.ts/,
],
});