* 🧪 test: Read the Artifact Scrim's Blur From the Persistent Scrim "a small blur surface still blurs by four pixels" read the backdrop filter of `#artifact-viewer [role="status"]`.first() once, with a single evaluate. The artifact panel's lazy tabs also render a `role="status"` loading placeholder until their chunk arrives, and the panel re-renders while it opens. A one-shot read that resolves the wrong status, or a node React has just replaced, gets the computed style of a detached element: an empty string instead of `blur(4px)`. Select the always-mounted refresh scrim, the only status that carries `aria-hidden`, and assert its filter with `toHaveCSS`, which re-resolves and retries until the value settles. Untraced, on two CPUs: 11 of 20 runs failed with `Received: ""` before, 20 of 20 pass after. * ⚡ perf: Mount the Mobile Header Menu Only Once a Phone Layout Has Shown It The chat header rendered both of its action surfaces on every load: the desktop buttons and the mobile overflow menu, hidden with `className="hidden"` on wide screens. The hidden menu still mounted its whole tree on desktop, with its bookmark, export, share, temporary-chat and pull-request hooks and its dialog providers. Mount the overflow menu only while the drawer viewport is active, and keep it mounted once it has been shown, so resizing back to desktop does not drop a dialog opened from it. Measured with a React DevTools hook counting component renders over a cold desktop load of a saved conversation (7 warm navigations each, production build): 1,313 component mounts before and 1,242 after, the overflow trigger no longer in the DOM, and the median component re-renders over the load 5,835 before and 5,682 after. `dropped-classes.spec.ts` read the overflow trigger's size on a desktop viewport, where it was only attached; it now reads it at phone width, where the trigger is visible. * 🧲 fix: Settle a Pinned Row's Position on Drop, Not Only on Hover A pinned row reorders in its `hover` callback, which judges the pointer against the hovered row's midpoint using the offset react-dnd last recorded. The HTML5 backend delivers hover once per animation frame, so a release that lands in the same frame as the move across the midpoint is judged against the offset from before it: the swap is refused and the drop keeps the old order. A quick, deliberate drag is discarded. The drop handler now runs the same placement against the final pointer offset before returning its result, so `end` commits the order the pointer actually ended on. Hovering is unchanged. The new regression holds the hover frame and fires dragover/drop at a point past the midpoint and at one short of it. On the previous handler the first case keeps the old order; with this change both cases match the pointer. In a real browser, with a drag that releases without waiting for a hover frame: "a pinned chat cannot be dragged through a pinned model" kept the old order in 4 of 15 untraced runs on the previous handler and 0 of 15 with this one. * 🧲 test: Release a Pinned Drag Without Waiting for a Hover Frame The pinned drag helper waited for an animation frame after each settling move, so react-dnd had delivered a hover at the final position before the release. That kept the reorder scenario green while a release landing in the same frame as its last move was discarded by the app. The drop handler now settles the final pointer position itself, so that wait no longer stands between the scenario and a failure; it only hides the fast-release path. Release straight after the settling moves again, so the scenario covers it. Untraced, on two CPUs, with this helper: "a pinned chat cannot be dragged through a pinned model" failed 4 of 15 runs against the previous drop handler and passed 15 of 15 with the settled drop; the whole pinned-drag-rules file passed 30 of 30. * 🧪 test: Publish the Reaper Fixture's PID Only After It Is Written Two reaper tests wait for a PID file to exist and then read it. The shell creates the file when it opens the redirection and writes the PID a moment later, so a read in between gets an empty file. `Number('')` is 0, and `isGone(0)` reports true because there is no `/proc/0`: "escalates to SIGKILL when the tree ignores SIGTERM" and "sweep reaps a group that outlived a clean root exit" then fail their "still running" assertion. Write the PID to a temporary file and rename it into place, so the file appears only with its content. With a 0.3 s pause forced between the redirection and the write, both tests fail on the old fixture and pass on this one. Also kill each spawned process group after every test, so a failing test does not leave its trap-protected `sleep 30` behind, and read the process state with `ps` where `/proc` does not exist. * 🔦 test: Keep a Measured Trace and a Startup Profile When Lighthouse Fails When the serial-latency audit fails a budget, the uploaded `.lighthouse/` holds the JSON and HTML reports only. They name the slow metric but not the main-thread work behind it, so a total-blocking-time failure cannot be traced to the code that blocked without reproducing it locally. - Run Lighthouse with `--save-assets`, keeping each run's measured `lhr-*-0.trace.json` for Chrome's Performance panel. Its raw DevTools network log is deleted; the trace is parsed and every string value has the session cookie values redacted, so numeric timings and JSON structure survive even for short cookie values. A trace that does not parse (an interrupted run) is deleted rather than uploaded raw. - After a failed audit, navigate once more with the CDP profiler running and save `.lighthouse/startup.cpuprofile`, including three seconds after the transcript appears so late catalog callbacks are sampled. This runs after the measured audits, so it cannot change their numbers, and a profiling error never replaces the original failure. Budgets, runs and the workflow are unchanged; the workflow already uploads `.lighthouse/`. * 🔦 test: Keep Lighthouse Traces Only When the Audit Fails Each saved trace is about 45 MB, so keeping all three on every run turned the always-uploaded `lighthouse` artifact from about 2 MB into about 137 MB for runs that pass and need no diagnosis. Delete the traces once the test's last assertion has passed; a failing run still uploads them with its reports. * 🧪 test: Read the Streaming Shimmer From the Live Footer "an in-flight label sweeps, and high contrast holds both stops opaque" read the stream-elapsed label's styles once through `locator.evaluate`. The label lives in the streaming footer, which re-renders while the reply streams; when the node the locator resolved was replaced before the evaluate ran, the read came from a detached element or failed. Assert the animation name and background clip with `toHaveCSS`, which re-resolves and retries, and read the shimmer's dip alpha by querying the current node inside the evaluate, polling until it settles. Untraced, on two CPUs: 4 of 20 runs failed with `Received: ""` before, 20 of 20 pass after. * 🧪 test: Click the Drawer's New Chat Link, Not Whichever One Is Exposed "returns focus after right-clicking an unopened running menu on a small screen" opens the mobile drawer and clicks `getByRole('link', { name: 'New chat', exact: true })`. On a saved chat the header row also offers a New chat link, and it hides only when the sidebar's expanded state commits. On mobile that commit is deferred: the drawer starts sliding first and the state flips a few frames later inside a transition, which a streaming run keeps pushing back. The drawer's Close button is already visible during the slide, so the test reaches the click while both links are exposed and fails with a strict mode violation. Scope the locator to `#mobile-drawer`, the link the test means. Untraced, on two CPUs: 20 of 20 runs failed on the previous locator and 20 of 20 pass with this one. --------- Co-authored-by: Dustin Healy <54083382+dustinhealy@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| artifacts.mts | ||
| audit.ts | ||
| librechat.yaml | ||
| load.spec.ts | ||
| process.mts | ||
| README.md | ||
| regression.cjs | ||
Serial database latency Lighthouse CI
Run from the repository root with Node 24 and Chrome installed:
npm ci
E2E_CHROMIUM_CHANNEL=chrome npm run lighthouse
# Reuse the production build:
E2E_CHROMIUM_CHANNEL=chrome npm run lighthouse:run
# Negative control: this MUST exit nonzero with an LCP assertion failure:
E2E_CHROMIUM_CHANNEL=chrome npm run lighthouse:regression
E2E_BASE_URL=http://localhost:3098 selects another local port. Set CHROME_PATH if
chrome-launcher picks the wrong browser — on WSL it prefers the Windows install, whose
debugging port is unreachable from Linux — and LIGHTHOUSE_CHROME_FLAGS to append Chrome
flags such as --no-sandbox. Each run starts a disposable MongoDB and the real Express
server, registers a local user, and seeds a conversation. No model inference is needed.
Do not point this test at a deployed service. Playwright refuses to reuse an existing
server.
The existing benchmarks/mongoose-latency-hook.cjs adds 250 ms per Mongoose
Query/Aggregate execution in the server process. Independent queries can overlap;
serial queries compound. This is a deterministic approximation of remote database
latency, not a replica topology or network emulator. It also delays query-based
writes; native driver calls, bulk operations and cursor batches are outside its
coverage. Use a TCP latency proxy if those paths need coverage.
The runner makes three cold browser navigations to a populated conversation,
using the production client build and real authentication, config, file and message
routes. It uses desktop settings with throttlingMethod: provided so Lighthouse
does not replace the measured server delays with simulated network timing.
Median budgets are LCP 4,500 ms, CLS 0.1, and TBT 500 ms, asserted against
the median of the three runs. These are lab regression budgets, not field web-vitals
percentiles; Lighthouse does not measure INP.
The test also requires the seeded transcript to be the LCP element, so a fast
login page, spinner, or empty shell cannot pass.
When the gate fails
- Read the failed audit's measured median and limit in the budget table the runner prints immediately before asserting.
- Open a
.lighthouse/lhr-*.report.htmlreport. The console also prints API request start/end times. A late request start suggests a browser dependency; a long request suggests server work or serial database reads. A failed audit keeps each run's measured.lighthouse/lhr-*-0.trace.jsonfor inspection in Chrome's Performance panel; a passing one discards them. Raw network logs are removed and cookie values redacted before upload. Failed audits also save.lighthouse/startup.cpuprofilefrom a separate navigation to identify expensive startup functions, including three seconds after the transcript appears so later catalog callbacks are sampled. Profiling runs after the measured audits and does not change their metrics or budgets. - Inspect the relevant path before changing the budget:
| Request / symptom | Code to inspect | Performance change this protects |
|---|---|---|
| Config waits for repeated user lookups | packages/api/src/app/service.ts, api/server/middleware/config/app.js |
#14101 |
| Message authorization and read run serially | api/server/routes/messages.js, packages/api/src/middleware/messageValidation.ts |
#14101 |
| Messages wait for the file map | client/src/data-provider/Messages/queries.ts, client/src/components/Chat/ChatView.tsx |
#14188 |
| Startup repeats auth user reads | api/server/controllers/AuthController.js, packages/api/src/auth/userDocCache.ts |
#14187, #14343, #14747 |
Reuse already-loaded user/config data. Start independent reads together, but keep every read scoped to the authenticated user/tenant and wait for authorization before returning data. Do not raise a threshold to hide added round trips.
Adding another scenario
auditPage({ url, cookies, runs, budgets }) in audit.ts invokes the Lighthouse CLI
once per run, redacts authentication headers, prints API timings and asserts the median
budgets. It returns the reports so each scenario can assert its expected final URL and
LCP content; read the LCP element through the exported lcpElement(report) helper rather
than an audit id, because Lighthouse renames those between majors. load.spec.ts owns
local login state, conversation seeding and transcript assertions; the runner does not
depend on them.
A downstream fork can import this runner from a separate launch spec, supply
cookies from its own authentication fixture and pass its own budgets for
launch thresholds. The database delay hook can be preloaded by that fixture's server
configuration. Keep provider-specific authentication and provisioning in that spec.
For launch flows that navigate across documents, also measure entry-to-ready time
with Playwright: LCP resets on a new document, so final-page LCP alone does not
cover the entire launch. This lane does not exercise OpenID/Redis cache priming.
lighthouse:regression preloads a test-only hook that adds 16 real, sequential
user reads before message retrieval (at least four extra seconds). It uses the
same page and budget assertions, and never modifies production source. Confirm
that largest-contentful-paint fails, rather than treating any process error as
proof. Run the normal command again to restore baseline reports. Reports stay
local or in GitHub job artifacts; session cookie values are redacted before upload.
Same-repository pull requests also receive the last 80 log lines as a failure comment.