* 🪧 feat: Unify Artifact Triggers as Tool Rows Every chat-side artifact trigger — code-execution output, a model-authored `:::artifact` block, and a mermaid diagram — now renders through one `ArtifactRow` that sits on the tool-row axis instead of three card variants in a wrapped chip strip. The row carries the file type twice: the language's brand glyph where one exists, and a format badge for titles that are not filenames. `kind.rendersPreview` tints the glyph and folds the same fact into the accessible name, so the row says whether opening it yields a rendered view or source to read. The mermaid panel routes every download through its export menu — SVG, PNG and the diagram source — rather than parking a second, unlabelled button beside it. The menu turns the pressed format into its own loading row, defers the synchronous SVG export by a macrotask so the spinner paints before the frame blocks, and no longer disappears on the code tab. A mermaid download keeps its real filename: its content *is* the stored `.mmd`, so it is not suffixed `.preview`. `useArtifactDownload` holds the original-file vs. edited-content decision that both the generic download button and the mermaid menu need. * 🩹 fix: Save the Stored Artifact File, Not Its Cached Text The panel serialized a mermaid diagram from `artifact.content`, which is the backend cache `extractUtf8` truncates at 512 KB, so "Download source" saved a corrupt `.mmd` for any larger stored diagram. An unedited mermaid artifact now takes the same original-file route as every other text-backed artifact, `handleDownload` reports whether bytes were actually delivered, and the export menu turns a failed delivery into its failure status and toast instead of "Diagram download started.". A `text/plain` row announced "Opens as source" while the panel opened it on the rendered markdown preview; it now announces the preview. A pending preview placeholder keeps the row slot it arrived in rather than sinking to the end of the artifact group and jumping up once it resolves. * 🧪 test: Exercise Artifact Rows and Mermaid Exports End to End Seven acceptance scenarios in the mock harness cover what the unit suites cannot: that a message which writes five artifacts shows five trigger rows on one axis, that a row's accessible name matches what the panel actually does with the artifact (rendered preview vs. source, including `text/plain`), that the mermaid export menu offers SVG, PNG and the diagram source on both the preview and the code tab, that "Download source" delivers the stored file and announces a failure when the stored file is unreachable, and that a pending preview placeholder keeps the row slot it arrived in. Each test carries its `@scenario:<id>` tag and is self-contained: fixtures come from the existing fake-model markers, Mongo seed helpers, or page-scoped route stubs, and every seeded conversation is deleted again. * 🏷️ fix: Name a Mermaid Blob After the Preview It Is A share with snapshots disabled strips the download route from a code-execution `.mmd` but keeps the cached `attachment.text`, and that text is what `extractUtf8` truncated at 512 KB. With no route to the stored file the panel can only serialize that cached copy, and the mermaid exception in `getArtifactDownloadFilename` handed those partial bytes over as `flow.mmd` — the stored file's own name, from a menu item labelled "Download source". Mermaid now follows every other file-backed artifact: a serialized blob is named `flow.preview.mmd`, and only the original-file route produces `flow.mmd`. A model-authored diagram has no stored file to be a preview of, so it keeps its own name. * 🎹 test: Reach the Export Menu's First Item From Either Start `supports keyboard export` opened the menu with Enter and pressed ArrowDown once, assuming no item was active yet. CI observed the opposite — the first item already active — so the single ArrowDown walked past SVG onto PNG and the test failed there while passing locally (it failed the same way on the previous head). `Home` lands on the first item from either starting point. * 🔧 test: Fix the Fixtures Three Artifact Scenarios Ran On The first verification run failed five of the seven scenarios on fixture defects, not on product behavior: - `artifact-trigger-rows` drove a fake-model marker (`E2E_ARTIFACT_SHOWCASE_REPLY`) that does not exist, so the reply carried no artifacts and no trigger rows. Both tests now seed the assistant message text through Mongo, the way the passing plain-text scenario does. - `mermaid-source-download` waited for an `ArtifactRow` that a code-execution `.mmd` never renders — it renders an inline diagram whose trigger is "Open as artifact" — and its cached diagram embedded the literal truncation marker, which mermaid cannot parse, so the diagram failed to render at all. The cached copy is now a valid shorter diagram and the stored file keeps the sentinel that proves which bytes were saved. - `pending-artifact-row-slot` gave the tool call's id to only one of its two attachments. Attachments reach a renderer through `mapAttachments` keyed by tool-call id, so the other one rendered nowhere; both now carry the same id and land in the one `artifact-row-group` whose ordering the scenario is about. * 🪜 test: Count Four Authored Rows, Then the Mermaid One\n\nA mermaid fence is not a trigger row while it is inline: it renders the diagram\nwith an "Open as artifact" button and only becomes an ArtifactRow once opened.\nThe axis scenario now asserts the four authored artifacts stack as rows, then\nopens the diagram and asserts its row joins the same column. * 📐 test: Measure the Row Axis in One Layout State\n\nOpening an artifact narrows the chat column, so comparing the mermaid row's x\nagainst a box measured before the panel opened reported an 8px offset that no\nuser sees. All five rows are now measured after the open, and the panel is\nlocated at page level, where it renders \u2014 not inside the message list. * 🧩 fix: Give the Mermaid Row Its Own Filename and Download A code-execution `.mmd` arrives wrapped in a header that prints the filename and a download button, and that header stayed put when the diagram collapsed into its trigger row — so the chat showed `flow.mmd` twice and left the download outside the row that every other file-backed artifact keeps it in. `Mermaid` now takes the file's download action and hands it to `ArtifactRow`, and reports whether it is currently rendered as that row; `ToolMermaidArtifact` drops its header while the row owns both. Opening and closing the diagram therefore moves the filename and the download between exactly one place each. * 🔖 test: Match a File-Backed Diagram's Own Trigger Id\n\nThe row ownership scenario looked for a \ trigger id, which\nonly a model-authored fence gets; a code-execution diagram's row carries the\nattachment's artifact id. * 📱 test: Dismiss the Artifact Sheet Before Using the Row\n\nOn a phone the artifact panel opens as a sheet over the message list, so the\nrow's download button sits behind it. The scenario now closes the panel with its\nown control first, which also shows the trigger stays a row after closing. * 🔔 fix: Raise One Toast per Failed Source Download\n\n\ already toasts "Error downloading file" before reporting a\nfailed delivery, and the export menu then raised its own — two notifications for\none press, on every ordinary 4xx/5xx, because React Query's refetch resolves\nwith an error result rather than throwing.\n\nA reported failure now only updates the menu's live region, which keeps the\nexport's own phrase for a screen reader; a thrown task still raises the toast,\nsince nothing else has spoken for it. * 🔕 fix: Let the Download Layer Own Every Failure It Reports The previous round split the failure notification by cause and left a hole in the other direction: `useAttachmentLink` toasted only its empty-response branch, so a thrown fetch — and the artifact hook's own blob failure — returned `false` with nobody having spoken, and the export menu's live-region-only announcement reached no sighted user. One rule instead of two branches: whichever layer returns `false` has already told the user. `useAttachmentLink` toasts its thrown branch as well, and `useArtifactDownload` toasts when the blob path throws or there is nothing to serialize. The mermaid export menu keeps announcing the export's own phrase in its live region, so one press still raises exactly one toast — this time for every failure, not just the polite ones. * 🪄 fix: Hold the Row Slot and the Frame Before Blocking Work Two fixes on the same theme — a row or a control that moves at exactly the moment it is meant to stay put. Writing salience-sorted resolved entries back into their arrival slots still moved a pending row once it resolved: the placeholder joined the sort and could pass an empty artifact that had been sitting above it. `attachmentSalience` reads only `bytes`, which resolution does not change, so one sort over the whole panel row — pending entries included — is stable across the transition and is also less code. The export deferral used `setTimeout(…, 0)`, which only moves the blocking SVG work to another task; the timer can still run before the browser paints the loading row, which is the whole point of deferring it. It now waits for an animation frame and then a task, falling back to a timer when the tab is hidden and serves no frames. * 🏷️ fix: Keep the Recovered Filename on the Mermaid Row\n\n\ turns a sandbox-generated \ back into\n\, but that name lived only in the wrapper header. The moment the\ndiagram collapsed into its trigger row, the row title and its download button's\naccessible name fell back to \ — the internal filename.\n\n\ now takes the caller's display name for the row, and\n\ passes the one it already computed. * 🗂️ fix: Open Each Artifact on the Tab Its Row Promises A code-only artifact forces the panel onto Code, and that constraint was written back into the panel's single `activeTab`. The next HTML, Markdown, plain-text, React or diagram artifact then opened on Code while its trigger row announced "Opens as a rendered preview" — the row was telling the truth about the artifact and a lie about the click. The constrained tab no longer leaks: the panel resets to the preview tab when a different, unconstrained artifact is opened, so a manual Code choice lasts as long as that artifact stays open and never follows the user to the next one. `displayedTab` already derived the forced tab for rendering, so nothing else depended on the write-back. |
||
|---|---|---|
| .. | ||
| benchmarks | ||
| benchmarks-mobile-chat | ||
| benchmarks-navigation | ||
| benchmarks-reasoning | ||
| benchmarks-tree | ||
| bombadil | ||
| byom | ||
| config | ||
| fixtures | ||
| lighthouse | ||
| perf | ||
| recordings | ||
| setup | ||
| specs | ||
| client-build.test.mjs | ||
| config.local.example.ts | ||
| jestSetup.js | ||
| playwright.config.a11y.ts | ||
| playwright.config.benchmark.ts | ||
| playwright.config.bombadil.ts | ||
| playwright.config.lighthouse.ts | ||
| playwright.config.local.ts | ||
| playwright.config.mermaid.ts | ||
| playwright.config.mobile-chat-perf.ts | ||
| playwright.config.mock.ts | ||
| playwright.config.navigation-perf.ts | ||
| playwright.config.real.ts | ||
| playwright.config.reasoning-perf.ts | ||
| playwright.config.redis.ts | ||
| playwright.config.tree-parity.ts | ||
| playwright.config.tree-perf-prod.ts | ||
| playwright.config.tree-perf.ts | ||
| playwright.config.ts | ||
| README.md | ||
| types.ts | ||
LibreChat e2e
The mock e2e profile is the safest default for generated tests. It starts LibreChat with e2e/config/librechat.e2e.yaml, injects an in-process fake LLM (via LIBRECHAT_TEST_RUN_HOOK), creates an authenticated e2e user, and avoids real provider credentials.
Stream Stores and Shards
The mock profile uses the in-memory generation stream store by default. To exercise the same browser scenarios through a real Redis job store and pub/sub transport, start Redis on port 6379 and run:
npm run e2e:mock:redis
Memory mode explicitly disables Redis. Redis mode defaults to database 15 with a LibreChatE2E key prefix, and fails closed: the test server pings Redis and verifies that the generation job manager did not silently fall back to memory. Override REDIS_URI or E2E_REDIS_KEY_PREFIX when needed.
Pull request CI runs the complete mock suite in memory mode across three shards, plus a focused Redis transport suite. The Redis suite covers streaming fidelity, steering, interrupts, resumptions, HITL approvals, completion, thread folding, model icons, and usage:
npx playwright test --config=e2e/playwright.config.mock.ts --shard=1/3
npm run e2e:mock:redis:transport
The nightly schedule and manual workflow dispatch run the complete mock suite in both stream modes across two shards per mode. Every shard keeps one worker so tests do not contend for its authenticated user and database.
Property-based browser testing
Bombadil explores randomized sequences across the core chat loop, message branches, parallel multi-conversation responses, model changes, reloads, and sidebar conversation lifecycle operations:
npm run e2e:bombadil
Set BOMBADIL_TIME_LIMIT for longer local or scheduled runs. Failures leave a
reproducible trace under e2e/.generated/bombadil-output; rerun it with:
BOMBADIL_REPRODUCE=e2e/.generated/bombadil-output npm run e2e:bombadil:run
Reproducing a real violation is expected to fail the Playwright test. Before a
new run overwrites the active output, the harness archives it under
e2e/.generated/bombadil-history/. Reproduction can diverge when streaming
timing changes; Bombadil reports that explicitly.
The harness uses the credential-free mock-LLM profile, so exploration never sends billable provider requests.
CI runs the broad property exploration for five minutes in the non-blocking
Bombadil Property Exploration workflow. If a property fails, download the
bombadil-reproduction-* artifact into
e2e/.generated/bombadil-output/, then reproduce it locally:
BOMBADIL_REPRODUCE=e2e/.generated/bombadil-output npm run e2e:bombadil:run
The accompanying bombadil-diagnostics-* artifact contains the captured CI log,
Playwright HTML report, and Playwright test results. A Bombadil failure produces
a workflow warning but does not block merge.
The default instruments inline JavaScript only because instrumenting LibreChat's
full Vite bundle can exceed Bombadil's driver timeout during stateful runs. Set
BOMBADIL_INSTRUMENT_JAVASCRIPT=files,inline for shorter coverage-guided
experiments.
The branch reload, fork submission, model/conversation, HITL pause/resume, and mid-run steering lifecycle properties can be run independently:
npm run e2e:bombadil:branch-reload
npm run e2e:bombadil:fork-lifecycle
npm run e2e:bombadil:model-lifecycle
npm run e2e:bombadil:hitl
npm run e2e:bombadil:steering
These focused commands are diagnostic properties: they exit nonzero when they
reproduce a product invariant violation. Reproduce a focused trace with its
matching :run script and output directory, for example:
BOMBADIL_REPRODUCE=e2e/.generated/bombadil-output-hitl npm run e2e:bombadil:hitl:run
HITL drives a real ask_user_question checkpoint through the answer/resume
controller, reloads while the question is paused, answers it once, and reloads
the completed conversation. Steering submits an in-flight steer during a slow
MCP-backed run, checks that it moves exactly once from the composer anchor into
the response at the tool boundary, and reloads the applied state. The model
lifecycle property is the passing control. The branch reload and fork
properties preserve their minimal failing traces.
Recording Tests
Use Playwright codegen when you want to turn an exploratory browser session into a draft test:
npm run e2e:record
That command builds the app, starts the LibreChat test server (with an in-process fake LLM) when needed, writes e2e/storageState.json, and opens Playwright codegen at /c/new. The npm script uses http://localhost:3333 so it does not collide with a normal dev server on 3080. Raw recordings are written to e2e/recordings/ and ignored by git.
For a real local LibreChat config instead of the mock profile:
npm run e2e:record:local
Useful direct options:
node e2e/setup/record.js --url=http://localhost:3080/c/new
node e2e/setup/record.js --profile=local --no-output
node e2e/setup/record.js --auth-only
node e2e/setup/record.js --output=e2e/recordings/settings-draft.spec.ts
LLM-Assisted Loop
- Start
npm run e2e:record. - Let the LLM use Computer Use to operate the headed Playwright browser.
- Stop codegen after the workflow is captured.
- Move the useful parts from
e2e/recordings/into a committed spec undere2e/specs/mock/. - Replace brittle generated selectors with role, label, text, or
data-testidlocators. - Add assertions that prove the behavior, not just the clicked path.
- Run the finished spec with
npm run e2e:mock -- <spec name>.
Generated recordings are a draft, not the final test. The committed version should use the shared helpers in e2e/specs/mock/helpers.ts where possible, wait on network or visible UI state instead of fixed sleeps, and keep test data deterministic.