`@lhci/cli@0.15.1` pins `lighthouse` to an exact `12.6.1`, so the Lighthouse
lane could never move off a release whose `puppeteer-core` still depends on
`extract-zip` — an advisory with no patched version. That pin, not Lighthouse
itself, was holding 8 of the repository's 16 `npm audit` findings open.
LHCI was only ever a subprocess wrapper here: `audit.ts` shelled out to
`@lhci/cli/src/cli.js` for `collect` and `assert`, and LHCI's own node runner
shells out to `lighthouse/cli/index.js` in turn. Call that CLI directly, run
the three navigations in a loop, and assert the median budgets in TypeScript
next to the assertions `load.spec.ts` already makes.
- `npm audit`: 16 findings (7 high, 1 moderate, 8 low) -> 6 low. The remainder
is the pre-existing `elliptic` chain under `vite-plugin-node-polyfills`,
which has no fixed version and is unrelated to this lane.
- Lighthouse 13 removed `largest-contentful-paint-element` and moved the LCP
node into `lcp-breakdown-insight`. Reading it through a `lcpElement` helper
keeps the "the transcript must be the LCP element" guard working and makes
the next rename fail loudly in one place.
- Reports move to `.lighthouse/` as `lhr-N.report.{json,html}`; cookie
redaction, the API timing table and the desktop/provided-throttling settings
are unchanged. Budgets are now printed as a table immediately before they are
asserted, so the workflow's 80-line failure comment always contains them.
- `CHROME_PATH` and `LIGHTHOUSE_CHROME_FLAGS` are documented: chrome-launcher
prefers the Windows Chrome under WSL, whose debugging port Linux cannot
reach.
|
||
|---|---|---|
| .. | ||
| audit.ts | ||
| librechat.yaml | ||
| load.spec.ts | ||
| README.md | ||
| regression.cjs | ||
Serial database latency Lighthouse CI
Run from the repository root with Node 24 and Chrome installed:
npm ci
E2E_CHROMIUM_CHANNEL=chrome npm run lighthouse
# Reuse the production build:
E2E_CHROMIUM_CHANNEL=chrome npm run lighthouse:run
# Negative control: this MUST exit nonzero with an LCP assertion failure:
E2E_CHROMIUM_CHANNEL=chrome npm run lighthouse:regression
E2E_BASE_URL=http://localhost:3098 selects another local port. Set CHROME_PATH if
chrome-launcher picks the wrong browser — on WSL it prefers the Windows install, whose
debugging port is unreachable from Linux — and LIGHTHOUSE_CHROME_FLAGS to append Chrome
flags such as --no-sandbox. Each run starts a disposable MongoDB and the real Express
server, registers a local user, and seeds a conversation. No model inference is needed.
Do not point this test at a deployed service. Playwright refuses to reuse an existing
server.
The existing benchmarks/mongoose-latency-hook.cjs adds 250 ms per Mongoose
Query/Aggregate execution in the server process. Independent queries can overlap;
serial queries compound. This is a deterministic approximation of remote database
latency, not a replica topology or network emulator. It also delays query-based
writes; native driver calls, bulk operations and cursor batches are outside its
coverage. Use a TCP latency proxy if those paths need coverage.
The runner makes three cold browser navigations to a populated conversation,
using the production client build and real authentication, config, file and message
routes. It uses desktop settings with throttlingMethod: provided so Lighthouse
does not replace the measured server delays with simulated network timing.
Median budgets are LCP 4,500 ms, CLS 0.1, and TBT 500 ms, asserted against
the median of the three runs. These are lab regression budgets, not field web-vitals
percentiles; Lighthouse does not measure INP.
The test also requires the seeded transcript to be the LCP element, so a fast
login page, spinner, or empty shell cannot pass.
When the gate fails
- Read the failed audit's measured median and limit in the budget table the runner prints immediately before asserting.
- Open a
.lighthouse/lhr-*.report.htmlreport. The console also prints API request start/end times. A late request start suggests a browser dependency; a long request suggests server work or serial database reads. - Inspect the relevant path before changing the budget:
| Request / symptom | Code to inspect | Performance change this protects |
|---|---|---|
| Config waits for repeated user lookups | packages/api/src/app/service.ts, api/server/middleware/config/app.js |
#14101 |
| Message authorization and read run serially | api/server/routes/messages.js, packages/api/src/middleware/messageValidation.ts |
#14101 |
| Messages wait for the file map | client/src/data-provider/Messages/queries.ts, client/src/components/Chat/ChatView.tsx |
#14188 |
| Startup repeats auth user reads | api/server/controllers/AuthController.js, packages/api/src/auth/userDocCache.ts |
#14187, #14343, #14747 |
Reuse already-loaded user/config data. Start independent reads together, but keep every read scoped to the authenticated user/tenant and wait for authorization before returning data. Do not raise a threshold to hide added round trips.
Adding another scenario
auditPage({ url, cookies, runs, budgets }) in audit.ts invokes the Lighthouse CLI
once per run, redacts authentication headers, prints API timings and asserts the median
budgets. It returns the reports so each scenario can assert its expected final URL and
LCP content; read the LCP element through the exported lcpElement(report) helper rather
than an audit id, because Lighthouse renames those between majors. load.spec.ts owns
local login state, conversation seeding and transcript assertions; the runner does not
depend on them.
A downstream fork can import this runner from a separate launch spec, supply
cookies from its own authentication fixture and pass its own budgets for
launch thresholds. The database delay hook can be preloaded by that fixture's server
configuration. Keep provider-specific authentication and provisioning in that spec.
For launch flows that navigate across documents, also measure entry-to-ready time
with Playwright: LCP resets on a new document, so final-page LCP alone does not
cover the entire launch. This lane does not exercise OpenID/Redis cache priming.
lighthouse:regression preloads a test-only hook that adds 16 real, sequential
user reads before message retrieval (at least four extra seconds). It uses the
same page and budget assertions, and never modifies production source. Confirm
that largest-contentful-paint fails, rather than treating any process error as
proof. Run the normal command again to restore baseline reports. Reports stay
local or in GitHub job artifacts; session cookie values are redacted before upload.
Same-repository pull requests also receive the last 80 log lines as a failure comment.