LibreChat/e2e/lighthouse
Marco Beretta 246e7e5de7
💠 fix: Paint Native Tool Verification Badges (#15862)
* 🪪 style: Give Native Tools a Painted Verified Mark

The native badge on a tool card was lucide's BadgeCheck filled with status-success-strong: green, the same hue the card already uses for selected, and stroked in currentColor, so the white check color also outlined the badge and left a white halo on any surface that is not the card's own white — hover, selection, dark mode.\n\nVerifiedIcon paints the badge instead of stroking it and carries the check as the only stroked path, and status-verified gives provenance its own blue across all four themes.

* 🩹 fix: Hold the Verified Mark at 3:1 on Every Card State

The dark fill was tuned against surface-secondary, but ToolCard repaints to surface-tertiary (#2f2f2f) on hover, where #0b74d4 fell to 2.85:1 — below the non-text floor, and invisible to the spec, which asserted the panel only. The fill moves to #1a7fd8 and the contract now covers every background the card takes. Both of the mark's relationships are graphical objects under WCAG 1.4.11, so the check is held at 3:1 rather than AA: no hue clears 3:1 on the hover surface and 4.5:1 under a white check at once.\n\nA theme that predates the token names status-success-strong and not status-verified, and would otherwise wear LibreChat's stock blue beside its own text-on-status and card surfaces. resolveTheme and the legacy mapColors adapter now carry the mark onto the fill it had before the token existed, following the six derived-token fallbacks already there.

* ✅ test: Exercise the Verified Mark in the Tool Library

Four scenarios in the mock harness, each named by the behavior it observes: the mark a native card wears, its silhouette under the dark hover repaint that the retune fixes, a pre-token deployment palette carrying it on the theme's own success fill, and a third-party MCP server card carrying no mark at all. The contrast assertions read the rendered colors rather than the tokens, so a regression in the component or in theme application fails them.

* 🧪 fix: Teach the Marketplace Dialog Spec About VerifiedIcon

The suite hand-lists the @librechat/client surface it mocks, so ToolCard's new icon resolved to undefined and every render in the file threw.

* 🧪 fix: Reach the Agent Builder on a Narrow Viewport

The scenario run covers mobile emulation, where the side rail is collapsed and the panel opens from the header's Control Panel toggle; the helper only knew the rail. The MCP scenario also minted its access token before the app had loaded, so the refresh request had no origin to resolve against.

* 🧪 fix: Run the Verified-Mark Scenarios on a Touch Viewport

Mobile emulation has no side rail: the panel switcher lives inside a sidebar that starts off-canvas, so the helper opens the sidebar and picks the Agent Builder entry from the switcher's menu. A touch viewport also has no hover state, so the hover scenario measures whatever background the card actually takes there and still holds the mark to 3:1 against it.

* 🩹 fix: Keep Pre-Token Themes on the Fill the Mark Inherited

The first fallback only fired when a theme restated status-success-strong, so a partial palette that customizes surfaces or text-on-status and inherits the green still landed on the stock blue. resolveTheme now derives the omission for any custom theme from the resolved success fill, and the legacy adapter takes the mode's bundled palette from ThemeProvider so it can do the same for a theme that names neither.\n\nThe scenario helper picked its path from a one-second probe for the desktop rail, which turns a slow first paint into the mobile branch; it now decides on viewport width and keeps the suite's full assertion timeout in each branch. The mock project is Desktop Chrome, so a touch scenario asks for a touch context explicitly rather than leaving that layout unexercised in CI.

* 🧪 fix: Match the App's Inclusive Layout Breakpoint

useSidebarState switches on (max-width: 768px), so a 768px-wide context renders the sidebar switcher while the helper's strict comparison went looking for the desktop rail.

* 🩹 fix: Scope the Verified Fallback to Themes That Painted the Mark

Two rounds pulled the same predicate in opposite directions: keying it on a theme restating status-success-strong missed palettes that only repaint the check or the card, and keying it on any custom color at all gave a new partial theme a green mark identical to the selected cue. Both are answered by asking what the theme actually painted — the fill the mark wore, the text-on-status check it carries, or the surfaces ToolCard takes — which is the same shape as the series-scale fallback above it. A theme that repaints none of those keeps the bundled blue.

* 🧪 fix: Wait for the Shell Before Reading the Sidebar State

isVisible() answers immediately, so a slow first paint read as an already-open drawer and the helper then waited on a switcher sealed inside it. It now waits for whichever of the opener and the switcher the shell paints first, which is the same assertion timeout the desktop branch gets.

* 📝 fix: Keep the Token Catalog Entry Catalog-Sized

The entry restated the icon's construction, the contrast rule and the pre-token fallback, all of which live on the token's own doc comment and in semanticTokens.spec.ts; the catalog around it is one line per role.

* 🔦 fix: Let a Crashed Lighthouse Run Be Retried

The audit shells out to Lighthouse three times and fails the lane if any single process exits nonzero, so a NO_NAVSTART — Lighthouse losing the trace of a page load, whose own message is "run Lighthouse again" — sinks a pull request that has nothing wrong with it. That is what happened on this branch: runs 1 and 2 wrote reports, run 3 crashed, and the identical head passed on rerun.\n\nEach run now gets a second attempt before the failure propagates. The repeat is per run, so the median is still taken over three completed reports, and a page that genuinely cannot be traced fails both attempts and still fails the lane.

* 🧪 fix: Read the Drawer State From Its Inert Marker

The readiness wait accepted the panel switcher, which is mounted whether or not the drawer is open — translated off-canvas and inert — so a shell that painted the sidebar before the chat header could satisfy it with the drawer closed and leave the helper clicking an off-viewport control. The drawer's own inert attribute is the state, so the helper waits for the drawer to attach and reads it there, with no timing probe on either side.

* 🧪 fix: Open the Drawer Before Trusting the Restored Form

A narrow context that had already selected the builder restores the form inside the closed drawer, and a translated element passes a visibility check, so the early probe accepted a form sitting under an inert drawer and handed it to the caller. The layout decision now comes first and the drawer is opened before anything inside it is read.
2026-09-12 16:03:03 -04:00
..
audit.ts 💠 fix: Paint Native Tool Verification Badges (#15862) 2026-09-12 16:03:03 -04:00
librechat.yaml 🎈 ci: Gate Page Loads Under Database Latency (#15646) 2026-09-06 12:01:44 -04:00
load.spec.ts 🔦 chore: Drop @lhci/cli and Run Lighthouse 13 Directly (#15687) 2026-09-06 23:20:22 -04:00
README.md 🔦 chore: Drop @lhci/cli and Run Lighthouse 13 Directly (#15687) 2026-09-06 23:20:22 -04:00
regression.cjs 🎈 ci: Gate Page Loads Under Database Latency (#15646) 2026-09-06 12:01:44 -04:00

Serial database latency Lighthouse CI

Run from the repository root with Node 24 and Chrome installed:

npm ci
E2E_CHROMIUM_CHANNEL=chrome npm run lighthouse
# Reuse the production build:
E2E_CHROMIUM_CHANNEL=chrome npm run lighthouse:run
# Negative control: this MUST exit nonzero with an LCP assertion failure:
E2E_CHROMIUM_CHANNEL=chrome npm run lighthouse:regression

E2E_BASE_URL=http://localhost:3098 selects another local port. Set CHROME_PATH if chrome-launcher picks the wrong browser — on WSL it prefers the Windows install, whose debugging port is unreachable from Linux — and LIGHTHOUSE_CHROME_FLAGS to append Chrome flags such as --no-sandbox. Each run starts a disposable MongoDB and the real Express server, registers a local user, and seeds a conversation. No model inference is needed. Do not point this test at a deployed service. Playwright refuses to reuse an existing server.

The existing benchmarks/mongoose-latency-hook.cjs adds 250 ms per Mongoose Query/Aggregate execution in the server process. Independent queries can overlap; serial queries compound. This is a deterministic approximation of remote database latency, not a replica topology or network emulator. It also delays query-based writes; native driver calls, bulk operations and cursor batches are outside its coverage. Use a TCP latency proxy if those paths need coverage.

The runner makes three cold browser navigations to a populated conversation, using the production client build and real authentication, config, file and message routes. It uses desktop settings with throttlingMethod: provided so Lighthouse does not replace the measured server delays with simulated network timing. Median budgets are LCP 4,500 ms, CLS 0.1, and TBT 500 ms, asserted against the median of the three runs. These are lab regression budgets, not field web-vitals percentiles; Lighthouse does not measure INP. The test also requires the seeded transcript to be the LCP element, so a fast login page, spinner, or empty shell cannot pass.

When the gate fails

  1. Read the failed audit's measured median and limit in the budget table the runner prints immediately before asserting.
  2. Open a .lighthouse/lhr-*.report.html report. The console also prints API request start/end times. A late request start suggests a browser dependency; a long request suggests server work or serial database reads.
  3. Inspect the relevant path before changing the budget:
Request / symptom Code to inspect Performance change this protects
Config waits for repeated user lookups packages/api/src/app/service.ts, api/server/middleware/config/app.js #14101
Message authorization and read run serially api/server/routes/messages.js, packages/api/src/middleware/messageValidation.ts #14101
Messages wait for the file map client/src/data-provider/Messages/queries.ts, client/src/components/Chat/ChatView.tsx #14188
Startup repeats auth user reads api/server/controllers/AuthController.js, packages/api/src/auth/userDocCache.ts #14187, #14343, #14747

Reuse already-loaded user/config data. Start independent reads together, but keep every read scoped to the authenticated user/tenant and wait for authorization before returning data. Do not raise a threshold to hide added round trips.

Adding another scenario

auditPage({ url, cookies, runs, budgets }) in audit.ts invokes the Lighthouse CLI once per run, redacts authentication headers, prints API timings and asserts the median budgets. It returns the reports so each scenario can assert its expected final URL and LCP content; read the LCP element through the exported lcpElement(report) helper rather than an audit id, because Lighthouse renames those between majors. load.spec.ts owns local login state, conversation seeding and transcript assertions; the runner does not depend on them.

A downstream fork can import this runner from a separate launch spec, supply cookies from its own authentication fixture and pass its own budgets for launch thresholds. The database delay hook can be preloaded by that fixture's server configuration. Keep provider-specific authentication and provisioning in that spec. For launch flows that navigate across documents, also measure entry-to-ready time with Playwright: LCP resets on a new document, so final-page LCP alone does not cover the entire launch. This lane does not exercise OpenID/Redis cache priming.

lighthouse:regression preloads a test-only hook that adds 16 real, sequential user reads before message retrieval (at least four extra seconds). It uses the same page and budget assertions, and never modifies production source. Confirm that largest-contentful-paint fails, rather than treating any process error as proof. Run the normal command again to restore baseline reports. Reports stay local or in GitHub job artifacts; session cookie values are redacted before upload. Same-repository pull requests also receive the last 80 log lines as a failure comment.