mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-08-27 04:07:05 +00:00
📄 feat: Rich File Artifact Previews for DOCX, CSV, XLSX, PPTX (#12934)
* 📄 feat: Rich File Artifact Previews for DOCX, CSV, XLSX, PPTX Render office files emitted by tools as interactive previews in the artifact panel instead of raw extracted text. The backend produces a sanitized HTML document via mammoth (DOCX), SheetJS (CSV/XLSX/XLS/ODS), or yauzl-based slide extraction (PPTX) and ships it through the existing SSE attachment payload; the client routes it through the Sandpack `static` template's `index.html` slot — no new browser deps, no client-side blob fetch, no React renderer components. * 🔐 fix: Restrict data: URLs to <img> in office HTML sanitizer Codex review on #12934 caught that `data:` lived in the global `allowedSchemes`, which meant a smuggled `<a href="data:text/html, <script>...</script>">` would survive sanitization. The Sandpack iframe sandbox does not gate `target="_blank"` navigations, so a click would open attacker-controlled HTML in a new tab. Scope `data:` to `<img src>` only via `allowedSchemesByTag` (mammoth inlines DOCX images as base64 `data:image/...` URIs — that path still works). Add a regression suite (`sanitizeOfficeHtml security`) with 8 cases covering: <script> stripping, event-handler removal, javascript:/data: rejection on anchors, data:image preservation in <img>, http/https/mailto allowance, target=_blank rel=noopener enforcement, and <iframe> stripping. * 🔧 fix: Route extensionless office files by MIME alone Codex review on #12934 caught that the office-render gate in `extractCodeArtifactText` only fired when the extension was in `OFFICE_HTML_EXTENSIONS` or the category was `document`/`pptx`. A tool emitting `data` with `text/csv` (no extension) classifies as `utf8-text`, so the gate was skipped and raw CSV text shipped to the client — but the client routes by MIME to the SPREADSHEET bucket expecting a full HTML document, so the panel rendered broken text. Extract a shared `officeHtmlBucket(name, mime)` predicate from `html.ts` (returns the bucket name or null). Both `bufferToOfficeHtml` (the dispatcher) and the upstream gate in `extract.ts` now go through this single source of truth, so they can never drift apart again. The predicate already mirrors the dispatcher's extension/MIME logic (extension wins; MIME is the fallback for extensionless inputs). Adds: - 14 cases for the new `officeHtmlBucket` predicate covering the positive paths (each bucket via extension OR MIME) and the negative paths (txt, py, json, jpg, pdf, zip, odt, plain noext). - A direct regression test in `extract.spec.ts` for the Codex catch: `data` with `text/csv` + utf8-text category routes through the office HTML producer. - Parameterized cases for extensionless DOCX/XLSX/XLS/ODS/PPTX files identified by MIME alone. * 🛡️ fix: Enforce extension-wins precedence in officeHtmlBucket Codex review on #12934 caught that the predicate's if-chain interleaved extension and MIME checks for each bucket — e.g. CSV's branch was `ext === 'csv' || CSV_MIME_PATTERN.test(mimeType)`. A `deck.pptx` shipped with `text/csv` (sandboxed tools sometimes ship generic MIMEs) matched the CSV branch BEFORE the PPTX extension branch was reached, so a binary PPTX would have been handed to `csvToHtml` to parse as text — yielding garbage or a parse exception. Restructure to a strict two-pass dispatch: an exhaustive extension table first (one lookup, all known extensions), then MIME-only fallback for extensionless / unknown-ext inputs. The doc comment's "extension wins" claim is now actually enforced by the implementation. Add 7 regression cases covering the conflicting-MIME footgun for each bucket: deck.pptx + text/csv → pptx; workbook.xlsx + text/csv → spreadsheet; legacy.xls + pptx-MIME → spreadsheet; report.docx + text/csv → docx; data.csv + docx-MIME → csv; etc. * 🛡️ fix: Reject zip-bomb office files before in-process parsing (SEC) Addresses pre-existing availability vulnerability validated by SEC review (Codex finding 275344c5...) and made worse by this PR's HTML rendering path. A sub-1MiB compressed XLSX/DOCX/PPTX (highly compressed run-of-zeros) inflates to 200+ MiB of XML when handed to mammoth/xlsx — blocking the Node event loop for 10+ seconds and spiking RSS to ~1 GiB. The existing 8s `withTimeout` wrapper uses `Promise.race`, which can only return early; it cannot interrupt synchronous parser CPU/RAM consumption. PoC ran an authenticated execute_code call to OOM the API process. Add `assertSafeZipSize(buffer)` — a yauzl-based pre-flight that streams every entry with mid-inflate byte counting and bails on either a per-entry or total decompressed-size cap. Mid-inflate counting cannot be bypassed by falsifying the central directory's `uncompressedSize` field (the technique the PoC used). Defaults: 25 MiB per entry, 100 MiB total — generous headroom for legitimate image-heavy office files, well below the attack profile. Hook the check into every path that hands a buffer to mammoth/xlsx /yauzl: - New HTML producers (`wordDocToHtml`, `excelSheetToHtml`, `pptxToSlideListHtml`) — added by this PR - Legacy RAG text extractors (`wordDocToText`, `excelSheetToText` in `crud.ts`) — pre-existing path, also vulnerable Errors propagate as a tag-distinct `ZipBombError` so callers can distinguish a refused bomb from generic parse failures. The outer `extractCodeArtifactText` swallows the error and returns null, falling back to the regular download UI. `.xls` (BIFF/CFB binary, not ZIP) is detected by magic bytes and skipped — yauzl would reject it as malformed anyway. Adds 15 tests: - `zipSafety.spec.ts` (9): benign passes, per-entry cap, total cap, ZipBombError type-tagging, malformed-zip distinction, directory- entry handling, named-error surfacing, and the SEC-PoC pattern (sub-1 MiB compressed → 50 MiB inflated rejected on default caps). - `html.spec.ts` zip-bomb suite (5): each producer rejects a bomb; dispatcher propagates correctly; legitimate fixtures still render. - `extract.spec.ts` (1): outer extractor swallows ZipBombError and returns null so the download UI fallback fires. * 🧹 fix: Normalize MIME parameters; add legacy CSV MIME variant Two related Codex catches on PR #12934 — both about MIME-routing inconsistencies between backend and client that would cause extensionless CSV files to render as broken (raw text under an HTML slot) or skip the artifact panel entirely. P2 — backend MIME normalization: `officeHtmlBucket` matched MIME strings exactly, so a real-world `text/csv; charset=utf-8` Content-Type slipped through and the backend returned raw CSV text. The client's `baseMime` helper strips parameters before its own MIME lookup, so it routed the same file to the SPREADSHEET bucket expecting an HTML body that never arrived. Mirror the client's normalization on the backend (strip everything from `;` onward, lowercase) before bucket matching. P3 — client legacy CSV MIME: Backend's `CSV_MIME_PATTERN` accepts three variants (`text/csv`, `application/csv`, `text/comma-separated-values`); the client's `MIME_TO_TOOL_ARTIFACT_TYPE` only had the first two. An extensionless file with `text/comma-separated-values` would have backend HTML produced but the client would skip the artifact panel entirely. Add the missing variant. Tests: - 9 new parameterized-MIME cases on backend covering charset/ boundary/case variants for every bucket. - 1 new client routing case for `text/comma-separated-values`. * 🩹 fix: Try office HTML before short-circuiting on category=other Codex review on #12934 caught that the early `category === 'other'` return short-circuited before `hasOfficeHtmlPath` was checked. The classifier returns 'other' for inputs the new dispatcher can still route — extensionless `application/csv` (CSV MIMEs aren't in the classifier's text-MIME set and don't start with `text/`), and extensionless office MIMEs with parameters like `application/vnd... spreadsheetml.sheet; charset=binary` (the classifier's `isDocumentMime` exact-matches these MIMEs without parameter normalization). Both would route correctly through `officeHtmlBucket` but never reached it. Move the office-HTML attempt above the 'other' early return, and drop the `|| category === 'document' || category === 'pptx'` shortcut now that `hasOfficeHtmlPath` covers the same surface (with parameter normalization) and a wider one. ODT still routes through `extractDocument` unchanged — `hasOfficeHtmlPath` returns false for it and the `category === 'document'` branch below handles it. Adds 3 regression tests: - extensionless `application/csv` + category='other' → office HTML - extensionless parameterized office MIME + category='other' → office HTML - defense check: actual binary 'other' (image/jpeg) still returns null without invoking the office producer * 🛡️ fix: Office types are HTML-or-null (no text fallback → XSS) Codex P1 review on #12934 caught that when `renderOfficeHtml` failed (timeout, malformed file, zip-bomb rejection) for an office type, the extractor fell through to `extractDocument` and returned plain text. The client routes by extension/MIME to the office preview buckets and feeds `attachment.text` straight into the Sandpack iframe's `index.html`. A spreadsheet cell or document body containing the literal string `<script>alert(1)</script>` would have been injected as executable markup — direct XSS. The contract for office types is now HTML-or-null with no text fallback. Failed render returns null, the client's empty-text gate keeps the artifact off the panel, and the file falls back to the regular download UI (matching what PPTX already did). PDF and ODT still go through `extractDocument` because the client routes them to PLAIN_TEXT (which the markdown viewer escapes) or no artifact at all, so plain text is safe there. Test reshuffle: - `document` describe block now uses ODT/PDF for the legacy parseDocument-path tests (DOCX/XLSX/XLS/ODS bypass that path). - New "does NOT call parseDocument for office HTML types" test locks in the SEC contract for all four office HTML buckets. - "falls back to ..." tests rewritten as "returns null when ..." with explicit `parseDocumentCalls.length === 0` assertions to prove no text leaks back to the client. - New XSS regression test for the XLSX failure path. - Mock parseDocument failure-name match relaxed to `includes()` so ODT-named tests can use the same trigger. * 🧽 chore: Address follow-up review findings on PR #12934 Wraps up the 10-finding follow-up review. Two MAJOR + four MINOR + two NIT addressed; one NIT skipped after verifying it was a misread of the package.json structure. MAJOR - #1: Rewrite `renderOfficeHtml` JSDoc to document the HTML-or-null contract explicitly. The pre-fix doc described a text-fallback path that was the original XSS vector (commit b06f08a). A future maintainer trusting the stale doc could reintroduce the fallback. - #2: Replace byte-truncation of office HTML with a small "preview too large" banner document. Cutting at a UTF-8 boundary lands mid-tag (`<table><tr><td>con\n…[truncated]`) and ships malformed markup to the iframe — unpredictable rendering, occasional broken layouts on DOCX with embedded images / wide spreadsheets. MINOR - #4: Wrap `readSlidesFromZip`'s `zipfile.close()` in try/catch so a close-time exception (mid-flight stream) doesn't replace the original error. Mirrors the defensive pattern in zipSafety.ts. - #5: Refactor PPTX extraction to use `yauzl.fromBuffer` directly, eliminating the temp-file write/unlink the safety pre-flight already proved unnecessary. Removes 4 unused imports (os, path, fs/promises, randomUUID). - #6: Extract `isPreviewOnlyArtifact(type)` to `client/src/utils/ artifacts.ts` so the membership check is unit-testable without mounting the full Artifacts component (Recoil + Sandpack + media query). 15 new test cases covering positive types, negative types, null/undefined, and unknown strings. NIT - #3: Remove dead `stripColorStyles` / `COLOR_PROPERTY_PATTERN` — unused (sanitizer's `allowedStyles` config handles color implicitly). - #7: Remove dead `!_lc_csv_label` worksheet property write. - #9: Remove no-op `exclusiveFilter: () => false` sanitize-html config. - #10: Type-narrow `PREVIEW_ONLY_ARTIFACT_TYPES` to `ReadonlySet<ToolArtifactType>` so the membership table is compile-time checked against the enum. SKIPPED - #8: Reviewer flagged `sanitize-html` as duplicated in devDeps and dependencies. The package has no `dependencies` section — only `devDependencies` and `peerDependencies`. Existing convention (mammoth, xlsx, yauzl, pdfjs-dist) is to appear in BOTH. Removing the devDep entry would break local test runs. Tests: packages/api 4406/4406, client artifacts 128/128. * 🪞 chore: Fix isPreviewOnlyArtifact test description parameter order Follow-up review nit on PR #12934. Jest's `it.each` substitutes `%s` positionally, and the table rows were `[type, expected]` while the description template read `'returns %s for type %s'` — outputting "returns application/vnd.librechat.docx-preview for type true" instead of the intended "type ... returns true". Reorder the template to match the column order. Test runner output now reads naturally: "type application/vnd.librechat.docx-preview returns true". Pure cosmetic — runtime behavior unchanged. * ✨ feat: Improve DOCX rendering and surface filename in panel header Two UX improvements based on hands-on use of the office preview pipeline. DOCX rendering — mammoth strips the navy banners, cell shading, and column layouts that direct-formatted docs apply (python-docx-style output is a common case). The flat `<p><strong>X</strong></p>` and bare `<table><tr><td>` it emits looks washed out next to the source. Three targeted compensations: - Style map promotes `Title`, `Subtitle`, `Heading 1` thru `Heading 6`, and `Quote` paragraphs to their semantic HTML equivalents (mammoth's default only handles Heading 1-6, missing Title/Subtitle/Quote). - Extra CSS scoped to `.lc-docx` gives the first table row sticky- looking header styling regardless of `<thead>` (mammoth never emits `<thead>`), adds zebra striping, and treats the python-docx `<p><strong>X</strong></p>` section-heading idiom as a pseudo-h2 with a thin accent left border so document structure survives the round trip. Headings get a left accent or underline so they read as headings instead of just bold paragraphs. - Sanitizer's `allowedAttributes` opens `class` on the heading and block tags the styleMap and CSS heuristics rely on. `<script>`, event handlers, javascript: URLs, etc. are still stripped — the existing security regression suite catches any drift. Panel header — `Artifacts.tsx` showed a generic "Preview" pill for preview-only artifacts. Single-tab Radio is a no-op; surfacing the document filename there gives the user something useful in the chrome without taking real estate. `displayFilename` handles the sandbox dotfile suffix the upload pipeline applies. Tests: html.spec.ts +1 (new CSS-emission lock), 71/71. Backend files suite 428/428. Client 308/308. * ✨ feat: High-fidelity DOCX preview via docx-preview in iframe Switch the default DOCX render path from server-side mammoth → flat HTML to client-side `docx-preview` loaded inside the Sandpack iframe. Mammoth becomes the fallback for files above the cap. Why --- The Sandpack iframe is a real browser DOM. Server-side rendering ceiling for DOCX→HTML is well below the source's visual fidelity — mammoth strips cell shading, run colors, banners, and column layouts because Word's layout model doesn't fit HTML's flow model. Pushing the render into the iframe lifts that ceiling without paying the server-side cost of jsdom or LibreOffice. What ---- - New `wordDocToHtmlViaCdn(buffer)` builds a self-contained HTML doc that embeds the binary as base64 and lets `docx-preview@0.3.7` render it on load. CSS preserves dark/light mode handoff via `prefers-color-scheme`. Bootstrap script falls back to a "preview unavailable, please download" message if the CDN is unreachable or the parse throws. - `docx-preview` and its `jszip` peer dep are pinned to specific versions on jsdelivr with SRI sha384 integrity hashes and `crossorigin="anonymous"`. Refresh: re-fetch the file, run `openssl dgst -sha384 -binary FILE | openssl base64 -A`. - CSP locked down on the iframe: `default-src 'none'`, scripts only from jsdelivr (no eval), `connect-src 'none'` so a parser bug in docx-preview can't be turned into exfiltration of the embedded document, `base-uri 'none'`, `form-action 'none'`. Defense in depth on top of the Sandpack cross-origin sandbox. - `wordDocToHtml` dispatches by size: ≤ 350 KB binary → CDN path (high fidelity), larger → mammoth fallback (preserves the size cap on `attachment.text`). 350 KB chosen so worst-case base64-inflated output (~478 KB) plus wrapper overhead (~5 KB) fits under MAX_TEXT_CACHE_BYTES (512 KB) with 40 KB headroom. - Internal renderers exported as `_internal` for tests. Public API unchanged — callers still go through `wordDocToHtml`. PPTX intentionally NOT switched ------------------------------- Surveyed the available client-side PPTX libraries: - `pptx-preview@1.0.7` ships an ESM-only main entry plus a 1.36 MB UMD that references `require("stream"/"events"/"buffer"/"util")` — bundled for Node, not browser-clean. Could work but the runtime references to undefined Node globals are a fragility risk worth more validation than this PR can absorb. - `pptxjs` is jQuery-era, requires four separate UMD scripts in a specific order, less actively maintained. - The honest answer for PPTX is the LibreOffice sidecar (DOCX/XLSX/ PPTX → PDF → PDF.js), which is the architecture every major product (Google Drive, Claude.ai, ChatGPT) effectively uses and the only path to ~5/5 fidelity for arbitrary user decks. PPTX stays on the existing slide-list extraction for now. Open a follow-up issue for the LibreOffice/Gotenberg sidecar. Tests ----- - 6 new in CDN-rendered describe block: wrapper structure, base64 round-trip, SRI integrity + crossorigin, CSP locks (connect-src/eval/base-uri/form-action), fallback message wiring, size-threshold lock. - Adjusted 2 existing tests that asserted on mammoth-path artifacts (literal document text in `<article class="lc-docx">`) — those assertions move to the mammoth-fallback test that calls `_internal.wordDocToHtmlViaMammoth` directly. Dispatcher tests now assert CDN-path signatures instead. packages/api files: 434/434 ✅, full unit suite 4473/4473 ✅. * 🧷 fix: Address Codex P1 (MIME aliases) + P2 (CDN dependency) Two follow-up review findings on PR #12934, both real. P1 — Spreadsheet MIME aliases on client ---------------------------------------- Backend's `officeHtmlBucket` uses the broad `excelMimeTypes` regex from `librechat-data-provider` (covers `application/x-ms-excel`, `application/x-msexcel`, `application/msexcel`, `application/x-excel`, `application/x-dos_ms_excel`, `application/xls`, `application/x-xls`, plus the canonical sheet MIMEs). The client's exact-match `MIME_TO_TOOL_ARTIFACT_TYPE` only had three of those, so an extensionless XLS upload with a legacy MIME would have backend HTML produced but the client would fail to route the artifact at all — preview chip never registers. Fix: import the same regex on the client and add it as a fallback in `detectArtifactTypeFromFile` after the exact-match map miss. Stays in lock-step with the backend automatically. 7 new test cases — one per legacy alias. P2 — Hard CDN dependency on jsdelivr ------------------------------------- Air-gapped / corporate-filtered networks where jsdelivr is unreachable would see DOCX previews permanently degrade to "Preview unavailable" because the iframe could never load the renderer scripts. Mammoth was sitting right there on the server but the dispatcher always preferred the CDN path for files under 350 KB. Fix: `OFFICE_PREVIEW_DISABLE_CDN` env var. When truthy (`1`, `true`, `yes`, case-insensitive, whitespace-trimmed), `wordDocToHtml` short-circuits to the mammoth path regardless of file size. Operators on filtered networks set the env var; default behavior is unchanged. Read at function-call time (not module load) so jest can flip it in `beforeEach` without `jest.resetModules()`. The cost is one property access per render. 12 new test cases: env-unset uses CDN (default), all five truthy forms force mammoth, five non-truthy forms (`false`/`0`/`no`/empty/ arbitrary string) leave CDN active. Tests ----- packages/api/src/files: 446/446 ✅ (was 434, +12 from env-var matrix). client artifact suites: 235/235 ✅ (was 228, +7 from MIME aliases). * ✨ feat: High-fidelity PPTX preview via pptx-preview in iframe Mirrors the DOCX CDN architecture for PPTX: small files (≤350 KB binary) embed as base64 and render via `pptx-preview` loaded from jsdelivr inside the Sandpack iframe. Larger files and air-gapped deployments fall back to the existing slide-list extraction. Why --- PPTX is the format where the gap between LibreChat's preview and Claude.ai-style previews was most visible (slide-list of bullet points vs. rendered slide layouts). LibreOffice → PDF → PDF.js is still the eventual gold-standard answer for PPTX fidelity, but client-side rendering inside the Sandpack iframe gets us a meaningful intermediate step (~1.5/5 → ~3.5/5) without a sidecar. What ---- - `pptx-preview@1.0.7` (ISC license, ~1.36 MB UMD bundle that includes its echarts/lodash/uuid/jszip/tslib deps inline). Pinned to a specific version on jsdelivr with SHA-384 SRI and `crossorigin="anonymous"`. - `buildPptxCdnDocument` mirrors the DOCX wrapper: same CSP locks (`default-src 'none'`, `connect-src 'none'`, no eval, no base/form tampering), same `id="lc-doc-data"` base64 slot, same fallback message wiring (`typeof pptxPreview === 'undefined'` → "Preview unavailable"). - New public `pptxToHtml(buffer)` dispatcher; `bufferToOfficeHtml` switches its `'pptx'` case to call it. `pptxToSlideListHtml` stays exported as the slide-list-only path (still hit by tests directly and by the dispatcher fallback). - `OFFICE_PREVIEW_DISABLE_CDN=true` env-var hatch applies to PPTX too — air-gapped operators get the slide-list path. Same env-var read at call time, same matrix of truthy values (`1` / `true` / `yes` / case-insensitive / whitespace-trimmed). - `_internal` re-exports moved to after the PPTX section since the PPTX internals live further down in the file. Adds `pptxToHtmlViaCdn`, `MAX_PPTX_CDN_BINARY_BYTES`, `PPTX_PREVIEW_CDN`. Honest caveats -------------- - The 1.36 MB UMD bundle has `require("stream"/"events"/"buffer"/ "util")` references in its outer wrapper. Those are bundled-dep artifacts (likely from `tslib` / Node-shim transforms) and don't appear to execute on the browser code paths, but I haven't done manual e2e on a wide range of decks. If a class of files turns up that breaks rendering, the iframe-side fallback message catches it and operators have `OFFICE_PREVIEW_DISABLE_CDN=true` as the bail. - First-render CDN fetch is ~1.36 MB (browser-cached after). - PPTX with embedded media easily exceeds the 350 KB binary cap; those files take the slide-list path. Lifting the cap is a follow-up (tied to the broader self-hosting work). Tests ----- 11 new in two new describe blocks: - `pptxToHtml dispatcher`: routing predicate (small → CDN, env-set → slide-list). - `CDN-rendered path`: base64 round-trip, SRI integrity + crossorigin, CSP locks (connect/eval/base/form), fallback message, size-threshold lock at 350 KB. - `OFFICE_PREVIEW_DISABLE_CDN escape hatch`: env-var matrix for truthy values. packages/api/src/files: 457/457 ✅ (was 446, +11). * 🪟 fix: DOCX preview fills the artifact panel width docx-preview defaults to rendering at the document's native page width (8.5in for letter, 21cm for A4). In a wide artifact panel that left whitespace on either side; in a narrow one it forced horizontal scroll. Two changes: - Pass `ignoreWidth: true` to `docx.renderAsync` so the library skips the document's pageSize width and uses its container's width. - Defensive CSS overrides on `.docx-wrapper` and `.docx-wrapper > section.docx` in case a future library version regresses on the option, plus `padding: 0` on the wrapper to drop the page-edge whitespace docx-preview otherwise reserves. `renderHeaders`/`renderFooters`/etc. stay enabled — those still appear in the rendered output, just inside a container that fills the panel instead of a fixed-width "page." Tests unchanged (100/100); manual e2e ahead of merge. * 🩹 fix: PPTX black screen — allow blob: workers + harden bootstrap Manual e2e of the PPTX CDN renderer surfaced a black screen with "Could not establish connection. Receiving end does not exist." unhandled-rejection — characteristic of a Web Worker that couldn't start. Root cause: pptx-preview's bundled echarts dep spins up Web Workers via blob: URLs for chart rendering. Our CSP had `default-src 'none'` and no `worker-src`, so workers fell back to default → blocked. The async failure deep inside echarts didn't surface through the outer `previewer.preview()` promise, so my bootstrap's `.catch` never fired, the loading state was removed, and the iframe sat with the body background showing through (dark navy in dark mode = "black screen"). Three changes: - Add `worker-src blob:` to the PPTX CSP. Allows blob:-only worker creation without permitting arbitrary worker URLs. - Bootstrap: window-level `unhandledrejection` and `error` listeners so rejections from inside bundled-dep async pipelines surface as the user-facing "Preview unavailable" fallback instead of going silent. - Bootstrap: 8-second timeout that checks `container.children.length` — if the renderer hasn't appended anything visible by then, assume silent failure and show the fallback. Also wipe `container.innerHTML` when showing the fallback so a partial render doesn't compete with the message. DOCX wrapper unchanged: docx-preview doesn't use workers, so the worker-src directive doesn't apply, and the existing fallback path already covers its failure modes. Tests ----- - Existing PPTX CSP test now also asserts `worker-src blob:` is present. - Existing fallback-message test extended to cover the new unhandledrejection/error/timeout listeners. packages/api/src/files: 467/467 ✅. * 🔒 fix: gate office HTML routing on backend trust flag (textFormat) Codex P1 review on PR #12934: routing .docx/.csv/.xlsx/.xls/.ods/.pptx into the office preview buckets assumed `attachment.text` was already sanitized full-document HTML, but that guarantee only existed for the new code-output extractor path. Existing stored attachments and other non-code paths can still carry plain extracted text — `useArtifactProps` would then inject that as `index.html` inside the Sandpack iframe. Adds a `textFormat: 'html' | 'text' | null` trust flag persisted on the file record by the code-output extractor, surfaced over the SSE attachment payload and the TFile API type. The client's routing in `detectArtifactTypeFromFile` requires `textFormat === 'html'` before landing on an office HTML bucket; everything else (legacy attachments, RAG-extracted plain text from `parseDocument`, explicitly-marked 'text' entries) falls back to the PLAIN_TEXT bucket where the markdown viewer escapes content rather than executing it. Tests: new `getExtractedTextFormat` helper has 14 cases covering all office paths, legacy XLS MIME aliases, parseDocument fallthroughs, and null-input. Client `artifacts.test.ts` adds three security-gate tests proving downgrade behavior for missing/null/'text' textFormat, plus a `fileToArtifact` test that legacy office attachments without the flag end up in PLAIN_TEXT with their content escaped. * 🌐 fix: air-gapped DOCX preview — embed mammoth fallback in CDN doc Codex P2 review on PR #12934: the CDN-rendered DOCX path always pulled docx-preview + jszip from cdn.jsdelivr.net. Air-gapped or corporate- filtered networks where jsdelivr is blocked would degrade to a static "Preview unavailable" message even though the server already had a local mammoth renderer that could produce readable output. Now the dispatcher renders mammoth first and embeds the sanitized output inside the CDN document as a hidden `#lc-fallback` block. The iframe's existing `typeof docx === 'undefined'` check (which fires when the CDN scripts can't load) un-hides the fallback so the user sees a real preview. CDN-success path is unchanged: high-fidelity docx-preview output owns the viewport, mammoth fallback stays hidden. Two new safeguards in the dispatcher: - Size budget: if base64(binary) + mammoth body + wrapper > 512 KB (the `attachment.text` cache cap), drop to mammoth-only so a giant document still renders. The `OFFICE_HTML_OUTPUT_CAP` constant mirrors `MAX_TEXT_CACHE_BYTES` from extract.ts (separate constant to avoid a circular import; pinned by a unit test). - `lc-render` is hidden when fallback shows so the empty padded slot doesn't sit above the mammoth content. Tests: existing CDN-path tests updated for the new `wordDocToHtmlViaCdn(buffer, mammothBody)` signature; new test for the embedded fallback structure (`#lc-fallback`, mammoth body content, "High-fidelity renderer unavailable" notice, render-slot hide); new constant pin and per-fixture cap-respect assertion. * 🧪 feat: LibreOffice → PDF preview path (POC, opt-in via env) Per the plan-mode discussion: prove out a LibreOffice subprocess pipeline as an alternative to the docx-preview / pptx-preview CDN renderers. LibreOffice handles every office format Microsoft and LibreOffice itself can open (DOCX, PPTX, XLSX, ODT, ODP, ODS, RTF, many more), produces a PDF, and the host browser's built-in PDF viewer renders it inside the Sandpack iframe via a `data:` URI. No client-side JS dependency, no CDN dependency, true high fidelity for any feature LibreOffice supports. Off by default. Operators opt in by setting both: - `OFFICE_PREVIEW_LIBREOFFICE=true` - LibreOffice (`soffice` or `libreoffice`) on the server's `$PATH` When either is missing, the dispatcher falls through to the existing CDN/mammoth/slide-list pipeline so a misconfiguration doesn't break previews. Hardening (`packages/api/src/files/documents/libreoffice.ts`): - Fresh subprocess per call with isolated temp dir, stripped env (PATH/HOME/TMPDIR only), and `-env:UserInstallation` so concurrent conversions can't collide on shared `~/.config/libreoffice` locks - 30-second wall-time cap; SIGKILL on timeout - 50 MB PDF output cap to bound disk pressure - 512 KB output cap on the wrapped HTML so the SSE/cache contract stays intact (base64 inflates ~33%, effective PDF cap ~380 KB) - Macros disabled by default flags (`--norestore --invisible --nodefault --nofirststartwizard --nolockcheck`) - Tag-distinct `LibreOfficeUnavailableError` / `LibreOfficeConversionError` so callers can swallow appropriately Iframe wrapper (`buildPdfEmbedDocument`): - Native browser PDF viewer via `<iframe src="data:application/pdf; base64,...">` — works in Chrome, Edge, Safari, Firefox - CSP locks the iframe to `default-src 'none'; frame-src data:; connect-src 'none'; script-src 'unsafe-inline'` — no outbound network, no eval, no external scripts - `#view=FitH` for first-paint sizing - 4-second heuristic timer that swaps to a "Preview unavailable" fallback when the browser's PDF viewer is disabled (kiosk mode, Brave Shields, etc.) Wired into `wordDocToHtml` and `pptxToHtml` as the first branch — returns null when disabled / unavailable / oversized so the existing pipeline takes over. XLSX intentionally NOT routed through this path: SheetJS's HTML output is already excellent for spreadsheets (sortable, sticky headers) and PDF rendering of sheets is awkward. Tests (`libreoffice.spec.ts`, 30 cases — 25 always run, 5 conditional on the binary): env-gating parser semantics matching `OFFICE_PREVIEW_DISABLE_CDN`, fallthrough contract (never throws, returns null on any failure), CSP lock-down, fallback structure, binary probe caching + missing-binary path, error tagging, and integration tests that engage when `soffice`/`libreoffice` is on PATH (DOCX→PDF, PPTX→PDF, output-cap fallthrough). Integration tests skip cleanly on bare CI. * 🩹 fix: CI — preserve legacy download path for empty-text office attachments Two regressions surfaced after the textFormat security gate landed. 1. **Client** (`LogContent.test.tsx` "falls back to the legacy download branch for an office file with no extracted text"): When the security gate downgraded an office type without `textFormat: 'html'` to PLAIN_TEXT, the lenient empty-text gate on PLAIN_TEXT then accepted a missing `text` field and rendered a half-empty panel card. The historical contract is "office type + no text → legacy download UI"; the downgrade should only fire when there's actual plain text that needs safe-escaping. Fix in `detectArtifactTypeFromFile`: short-circuit to null when the office type lands in the security-gate branch with no text. The PLAIN_TEXT downgrade still fires for legacy attachments that DO carry plain text. 2. **API** (`process.spec.js` + `process-traversal.spec.js`): the `@librechat/api` mocks didn't expose `getExtractedTextFormat`, so `processCodeOutput` called `undefined(...)` → TypeError → tests got undefined results. Added the helper to both mocks with a faithful default (returns 'text' for non-null extractor output, null otherwise). Tests: new regression in `artifacts.test.ts` pinning the empty-text + no-textFormat → null contract for all four office types (.docx/.csv/.xlsx/.pptx), so a future refactor can't silently re-introduce the half-empty card. * 🩹 fix: PPTX slides scale to fit panel width (no horizontal scroll) Manual e2e on PR #12934: pptx-preview rendered slides at their native init dimensions (960×540 default). The artifact panel is much narrower than that, so the iframe got a horizontal scrollbar and only a corner of each slide showed at any time — the user had to drag-scroll across each slide to read it. Fix: keep pptx-preview's init at 960×540 so its internal layout math stays correct, then post-process each rendered slide: - Cache the slide's native width/height on its dataset BEFORE applying any transform (so subsequent re-fits don't measure the already-transformed box). - Wrap the slide in `.lc-slide-wrap` with explicit width/height set inline to the scaled dimensions; the wrap shrinks the layout space the slide occupies. - Apply `transform: scale(panel_width / 960)` to the slide itself with `transform-origin: top left` so the rendered output shrinks from the top-left corner into the wrap. - Cap the scale at 1.0 so small slides don't upscale and get blurry. Streaming + resize: - `MutationObserver` watches the container for slide insertions so streaming renders get scaled on arrival rather than waiting for the entire `previewer.preview` promise to settle. - `ResizeObserver` re-fits all wrapped slides when the iframe resizes (panel drag, window resize). Tests: new "bootstrap wraps + scales each slide" lock in the wrap class, scale computation, observer setup, and native-size caching so a future refactor can't silently re-introduce the overflow. * 🩹 fix: PPTX wrap+scale runs after preview, not during streaming Manual e2e on PR #12934: regenerated PPTX showed "Preview unavailable" in the iframe. Root cause: the MutationObserver I added in the previous commit fired during pptx-preview's render and moved slides out from under the library's references. pptx-preview's async pipeline raised an unhandled rejection, the iframe's window-level listener caught it, and the fallback message replaced the partial render. Fix: drop the MutationObserver. Apply the wrap+scale ONCE in a `finalize` step that runs: - On `previewer.preview().then` (the happy path) - On the 8-second timeout safety net IF the container has children (silent-failure path — pptx-preview emitted slides but never resolved its outer promise) To prevent the user from seeing an unscaled flash while pptx-preview renders into the 960px-wide canvas, the container is set to `visibility: hidden` at init and only revealed inside `finalize` after wrap+scale completes. Resize handling stays via `ResizeObserver` on `document.body`, installed AFTER the wrap pass so it doesn't fire during the wrap itself. Tests: regression assertion now also locks in: - `container.style.visibility = 'hidden' / 'visible'` (the flash- prevention contract) - Absence of MutationObserver (the bug we just removed — must NOT creep back in via a future "let's scale during streaming" idea) * 🩹 fix: PPTX slides fill panel width (drop upscale cap, per-slide scale) Manual e2e on PR #12934: slides rendered correctly but didn't fill the artifact panel — whitespace on either side. Two issues: 1. The scale was capped at `Math.min(1, available / SLIDE_W)`. On panels wider than 960px, the cap clamped the scale to 1.0 and slides rendered at native size with whitespace on the sides instead of stretching. 2. The scale was computed against the constant `SLIDE_W = 960`, but pptx-preview can emit slides whose `offsetWidth` differs from the init param if the source PPTX has a non-16:9 layout. Per-slide division of `available / nativeW` handles that case. Fix: replace `computeScale()` with two helpers — `availableWidth()` returns the panel content-box width and `scaleFor(nativeW)` returns the per-slide scale. No upscale cap. The slide content is rendered by pptx-preview against its 960×540 canvas using vector text / canvas — scaling up to e.g. 1500px doesn't visibly degrade quality. Tests: regression now also asserts: - `availableWidth()` and `scaleFor()` exist by name - The exact scale formula `availableWidth() / (nativeW || SLIDE_W)` - Negative assertion that `Math.min(1, ...)` is NOT present, so a future "let's add an upscale cap" rewrite can't silently re-introduce the whitespace. * 🩹 fix: PPTX preview fills panel height (no white gap below slides) Manual e2e on PR #12934: PPTX preview filled the panel width but left empty space below the last slide. DOCX didn't have this issue because its content (mammoth-rendered HTML) flows naturally and either fits exactly or overflows; PPTX slides are fixed-aspect 16:9 and don't grow with the panel. Two changes: 1. **Body fills the iframe viewport** — `html, body { min-height: 100vh }` plus `body { display: flex; flex-direction: column }` and `#lc-render { flex: 1 0 auto }`. The dark theme bg now fills the iframe even when total slide content is shorter than the panel, so a single-slide deck never reveals a "white below" gap. 2. **Per-slide scale honors viewport height** — `scaleFor(nativeW, nativeH)` now returns `min(width-fit, height-fit)` (largest factor that fits without overflowing either dimension). On a tall artifact panel with a short deck, slides grow up to the full panel height instead of staying at the width-bound size. Existing height-fit was always considered correct conceptually but the previous implementation only used width-fit, leaving half the viewport unused per slide. Tests: regression now also asserts `availableHeight()`, the `Math.min(sw, sh)` formula, and `min-height: 100vh` are in the bootstrap. Negative assertion for the old `Math.min(1, ...)` upscale cap remains. * 🩹 fix: revert body flex on PPTX bootstrap (caused black-screen render) Manual e2e regression on PR #12934: the previous commit added `body { display: flex; flex-direction: column }` plus `#lc-render { flex: 1 0 auto }` to fill the panel height. Side effect: pptx-preview's internal layout assumes block flow on its ancestor elements; making body a flex container caused slides to render as solid-black rectangles (sized correctly, but with no visible content inside). Fix: keep just `html, body { min-height: 100vh }` for the bg-fill effect — that alone gives empty space below short decks the dark theme bg without changing flow. Drop the body-flex and the `#lc-render { flex: 1 0 auto }` directives. The height-aware `scaleFor(nativeW, nativeH)` from the same commit stays — it doesn't interact with pptx-preview's layout, just chooses a per-slide scale. Each slide still grows to fit the viewport contain-style. Negative-assertion added to the regression test: `body { display: flex }` must NOT appear in the bootstrap, so a future "let's flex the body to make height work" rewrite can't silently re-introduce this. (Note: the user also flagged DOCX theming as faint body text; I'm leaving that for now per their note that it may be pre-existing. Not addressed in this commit.) * 🩹 fix: revert PPTX height-fill changes; lock DOCX CDN to light scheme Two fixes for separate manual e2e regressions on PR #12934. **1. PPTX black screen (single slide rendering as solid black).** The previous fix removed `body { display: flex }` thinking that was the sole cause, but the regression persisted. Bisecting against the last known-good commit (4e2d538b0, width-fit only), the actual culprit is the COMBINATION of: - `min-height: 100vh` on html/body - `availableHeight()` reading viewport-derived dimensions - `Math.min(sw, sh)` height-aware scale pptx-preview's CSS injection step interacts unpredictably with these. Reverting to width-only `scaleFor(nativeW)` and dropping the viewport min-height restores reliable rendering. Vertical empty space below short decks now shows the body's bg color (`var(--bg)`) which still matches the panel theme — that's an acceptable trade-off vs. the black-screen regression. Negative assertions added: `Math.min(sw, sh)`, `availableHeight`, `min-height: 100vh`, `body { display: flex }` must NOT appear in the bootstrap. So a future "let's fill height" rewrite has to demonstrate it doesn't break pptx-preview before it can land. **2. DOCX body text rendering as faint / translucent grey.** docx-preview emits page-style rendering with white pages and the docs native text colors. The CDN doc declared `color-scheme: light dark`, so on OS dark mode the iframes inheritable `--fg` resolved to `#e5e7eb` (light grey). docx-preview body text (no explicit color in the source DOCX) inherited that light-grey on the white page bg → barely-visible "translucent" rendering. Fix: declare `color-scheme: light` only in `buildDocxCdnDocument`, drop the dark-mode `@media` override. docx-preview is a light-mode- only renderer; matching that produces correct contrast regardless of OS theme. The mammoth-only `wrapAsDocument` path is unaffected — it owns its own bg + text colors and continues to respect the users OS scheme. New regression test pins the lock: CDN doc must contain `color-scheme: light`, must NOT contain `color-scheme: light dark`, must NOT contain `prefers-color-scheme: dark`. * 🩹 fix: relax connect-src to allow sourcemap fetches (silence CSP noise) Manual e2e on PR #12934: every time DevTools is open while viewing a DOCX or PPTX preview, the console fills with CSP violations like: Connecting to 'https://cdn.jsdelivr.net/npm/docx-preview@0.3.7/ dist/docx-preview.min.js.map' violates the following Content Security Policy directive: "connect-src 'none'". The request has been blocked. The actual rendering isn't affected (sourcemap fetches happen AFTER the script has already loaded and executed via `script-src`), but the noise is enough to make people suspect a real problem and distracts from useful console output. Fix: relax `connect-src` from `'none'` to `'self' https://cdn. jsdelivr.net` in both DOCX and PPTX CDN docs. This allows: - Same-origin fetches (sandpack-static-server) — covers any bundler-embedded sourcemaps + same-origin runtime fetches the renderer might make - jsdelivr fetches — covers sourcemaps from the CDN where we loaded the script Exfiltration risk stays minimal: the iframe is cross-origin to LibreChat so an attacker can't read application data anyway, and neither 'self' (sandpack-static-server) nor jsdelivr is a useful target for exfiltrating slide content to a host the attacker controls. Tests updated: assertions for `connect-src 'none'` swapped to `connect-src 'self' https://cdn.jsdelivr.net` for both DOCX + PPTX CDN docs. Added negative assertion for wildcard `*` in connect-src so a future "let's allow everything" rewrite can't widen the exfiltration surface. * 🩹 fix: surface PPTX/DOCX fallback reason (inline + console) Manual e2e on PR #12934: "Preview unavailable" appears in the iframe with no way to know what actually failed. The reason was tucked into the fallback element's `title` attribute (hover-only tooltip) — easy to miss and impossible to copy/paste. Now surfaces three ways: 1. Visible inline via a `<details>` element with the reason in monospace, folded so the friendly message stays primary but the diagnostic is one click away in the iframe itself. 2. `title` attribute (preserved) for hover tooltip. 3. `console.error('[pptx-preview] fallback fired:', reason)` so DevTools shows it in red — also the only reliable way to see the reason if the iframe is detached / re-mounted. DOCX gets the same console mirror (as `console.warn` since the fallback there is "high-fidelity unavailable, showing simplified preview" — informational, not error). The DOCX fallback already displays the mammoth-rendered content visibly, so no `<details>` needed there. Tests: regression assertions pin the diagnostic surfacing — the `<details>` element, the `title` write, and the `console.error` call must all be present in the bootstrap. * 🩹 fix: PPTX CDN embeds slide-list fallback + detects empty renders Manual e2e + DOM inspection on PR #12934: pptx-preview silently produces empty `.pptx-preview-wrapper` placeholders for pptxgenjs- generated decks. The library parses the file enough to create the 960×540 host element with a black bg, then fails to populate it. The outer Promise resolves "successfully" — no throw, no rejection, the bootstrap thinks rendering succeeded — and the user sees a black rectangle with no content and no fallback message. Fix mirrors the DOCX mammoth-fallback pattern from commit0c0b0ce88: 1. **Server side**: `pptxToHtml` now renders the slide-list body (`<ol class="lc-pptx-list">...`) via the new `renderPptxSlidesBody` helper, then embeds it inside the CDN doc via the new `buildPptxCdnDocument(base64, slideListFallbackBody)` signature. Combined-doc size budget mirrors the DOCX pattern: if the CDN doc would exceed `OFFICE_HTML_OUTPUT_CAP` (512 KB), drop to slide-list only. 2. **Iframe bootstrap**: new `hasRenderedContent()` check after `wrapSlides()` walks each `.lc-slide-wrap` looking for actual child content inside pptx-preview's emitted slide nodes. If every wrap is empty, fires `showFallback('renderer-produced-empty- wrappers ...')` which reveals the embedded slide-list view instead of the previous static "Preview unavailable" message. 3. **CSS**: slide-list rules extracted to `PPTX_SLIDE_LIST_CSS` constant so they can be inlined into both the standalone slide- list document AND the CDN doc's `<style>` block (CSP `style-src` is `'unsafe-inline'` only — no external sheets). `renderPptxSlidesHtml` now delegates to `renderPptxSlidesBody` wrapped in `wrapAsDocument` — single source of truth for the slide markup. Tests (506 passing, +1 vs before): existing `pptxToHtmlViaCdn` call sites updated for the new fallback-body argument; new regression test pins `hasRenderedContent`, the `renderer-produced-empty-wrappers` reason string, the embedded fallback structure, and the inlined slide-list CSS. * fix: Detect Empty PPTX Preview Slides * 🩹 fix: LibreOffice PDF embed uses blob: URL (Chrome blocks data: PDFs) Manual e2e on PR #12934: enabling `OFFICE_PREVIEW_LIBREOFFICE=true` on a host with `soffice` installed surfaced "This page has been blocked by Chrome" inside the PDF preview iframe. Root cause: Chrome blocks `data:application/pdf;base64,...` navigations inside sandboxed iframes (anti-phishing measure since Chrome 76, see crbug.com/863001). The Sandpack iframe IS sandboxed (its `sandbox="..."` attribute lacks `allow-top-navigation` for data: URLs specifically), so when our inner `<iframe src="data: application/pdf;...">` tries to navigate, Chrome's interstitial fires and renders the "blocked" message. Fix: switch from `data:` URL to `blob:` URL. The bootstrap now: 1. Reads the base64 payload from a `<script type="application/ octet-stream;base64">` data block (same pattern as the DOCX and PPTX wrappers). 2. Decodes via `atob` + `Uint8Array.from`. 3. Creates a `Blob` with `type: 'application/pdf'`. 4. `URL.createObjectURL(blob)` produces a same-origin blob: URL. 5. Sets `pdfFrame.src = url + '#view=FitH'` — Chrome treats blob: URLs as legitimate navigation and serves the built-in PDF viewer. CSP updated: `frame-src blob:` (was `frame-src data:`). `data:` is now explicitly NOT allowed in `frame-src` since Chrome would block it anyway in our context — keeping it would be misleading documentation. Bonus: failure paths now log to `console.error` with a `[libreoffice-pdf]` prefix so DevTools surfaces blob-creation failures and PDF-viewer load timeouts in red. Tests updated: - "emits a complete sandboxed HTML document" now asserts the data-block + blob URL construction (not the old data: URL). - New CSP test "allows blob: in frame-src (NOT data:)" with both positive and negative assertions to lock in the change. - Integration test for `tryLibreOfficePreview` updated to look for the data block + `URL.createObjectURL` instead of the data: URL. - Large-payload test now verifies the data block round-trip rather than data: URL escaping (base64 alphabet has no characters that break out of `<script>` anyway). * 🩹 fix: LibreOffice PDF embed renders via pdf.js (Chrome blocks blob: PDFs too) Manual e2e on PR #12934 round 2: switching from `data:` to `blob:` URLs (commitd90f26c11) didn't fix the "This page has been blocked by Chrome" interstitial. Chrome blocks BOTH data: AND blob: PDF navigations inside sandboxed iframes — the built-in PDF viewer requires a top-level browsing context. The Sandpack host iframe is sandboxed, so neither approach works. Fix: switch from native browser PDF viewer to pdf.js (Mozilla's pdfjs-dist) loaded from CDN. pdf.js renders to `<canvas>` which works in any context — no plugin, no privileged viewer, no top-level requirement. ~1 MB CDN load is acceptable for a path that's already opt-in via `OFFICE_PREVIEW_LIBREOFFICE=true`. Implementation: - Pin pdf.js v3.11.174 (single-file UMD; v4+ uses ES modules which complicate the load + SRI flow) - Worker URL pointed at the same jsdelivr origin; CSP `worker-src https://cdn.jsdelivr.net blob:` allows it - DPR-aware canvas rendering: scale based on `panelWidth / page.viewport.width * devicePixelRatio` so retina displays get crisp pixels - Sequential page rendering (Promise chain) so a many-slide PDF doesn't spawn N parallel render jobs - 15 s timeout safety net (was 4 s for the native viewer; pdf.js with DPR=2 on a many-page PDF can take longer) CSP changes: - Added `script-src https://cdn.jsdelivr.net 'unsafe-inline'` (was inline-only) - Added `worker-src https://cdn.jsdelivr.net blob:` - Removed `frame-src` entirely (no nested iframes) - Removed `object-src` (no `<object>`/`<embed>` either) Same diagnostic surfacing as the other CDN paths: failure reasons shown via `<details>` disclosure inline + `console.error` to DevTools. Tests updated: PDF.js script presence, GlobalWorkerOptions setup, canvas render path, all the new failure detection paths. Negative assertions for both `data:application/pdf` and `blob:...application /pdf` so a future "let's just try the native viewer again" rewrite can't silently re-introduce the Chrome block. SRI hashes intentionally omitted (unlike docx-preview / pptx- preview) — operator opted in by setting the env flag and trusts the LibreOffice render pipeline. Worth adding once the path is proven in production. * 🧹 cleanup: trim unused _internal exports + stale JSDoc references After the LibreOffice + pdf.js path proved out, swept the office HTML modules for dead code and stale documentation. **Unused `_internal` exports removed (`html.ts`):** - `renderMammothBody` — only called within the file (by `wordDocToHtmlViaMammoth` and `wordDocToHtml`), never imported by tests. - `DOCX_PREVIEW_CDN` — internal config constant, never referenced. - `PPTX_PREVIEW_CDN` — same, never referenced. The remaining `_internal` surface (`wordDocToHtmlViaCdn`, `wordDocToHtmlViaMammoth`, `pptxToHtmlViaCdn`, `MAX_DOCX_CDN_BINARY_BYTES`, `MAX_PPTX_CDN_BINARY_BYTES`, `OFFICE_HTML_OUTPUT_CAP`) is all actively used by the spec file. **Stale JSDoc fixed (`libreoffice.ts`):** Module-level header still claimed we "embed the PDF as a base64 data:application/pdf URI" and "rely on the host browser's built-in PDF viewer". Both untrue after the pdf.js switch in commitb2cc81ad8. Updated to: - Describe the actual pipeline: PPTX → soffice → PDF → pdf.js → canvas - Document the dead-end iterations (data: blocked, blob: also blocked, pdf.js works) so future readers don't re-discover the same Chrome PDF-viewer-in-sandboxed-iframe limitation - Drop "(POC)" tag — the path is production-quality, just opt-in - Adjust disk footprint estimate (250-350 MB with `--no-install-recommends` is more accurate than the 500 MB original) No production code changes; tests still 505 passing. * ✨ feat: per-format LibreOffice opt-in (env value accepts format list) Manual e2e on PR #12934: enabling `OFFICE_PREVIEW_LIBREOFFICE=true` forces both DOCX and PPTX through the LibreOffice path. DOCX renders ~instantly via docx-preview and rarely needs the LibreOffice treatment; paying the ~2-3 s cold-start there hurts UX without adding much. Solution: extend the env var to accept three forms: - Truthy (`true`/`1`/`yes`): all formats — backwards compatible with the previous behavior - Falsy (`false`/`0`/`no`/empty/unset): no formats — default - Comma-separated list (`pptx`, `pptx,docx`): just those formats Practical guidance documented in the module header: most operators will set `OFFICE_PREVIEW_LIBREOFFICE=pptx` — pptx-preview chokes on pptxgenjs decks and the slide-list fallback loses formatting, so LibreOffice is the only path that produces a faithful PPTX preview. DOCX is well-served by docx-preview's existing CDN renderer. API: - New `isLibreOfficeEnabledFor(format)` is the per-format gate, used by `tryLibreOfficePreview` to short-circuit before doing work. - Existing `isLibreOfficeEnabled()` retained for "any format enabled" diagnostic checks (returns true if at least one format is opted in). - Internal `parseLibreOfficeEnablement` returns `'all' | Set | null` — keeps the gate future-proof: adding a new format to the LibreOffice route doesnt require operators to re-enumerate their env value. Edge cases handled: - Whitespace-tolerant: ` pptx , docx ` works - Case-insensitive on both env value AND format name - Empty list entries dropped: `pptx, ,docx` enables pptx + docx - Empty string treated as unset (not as a valid empty list) Tests: 21 new cases pinning the parse semantics + per-format gate (`pptx` env vs `docx` lookup → false, etc.). Existing `isLibreOfficeEnabled` tests retained but renamed to clarify the "any format" semantic. Total file tests: 526 passing (+21 vs before). * 🔒 fix: officeHtmlBucket only does MIME fallback when extension is empty Codex P2 review on PR #12934: the server's `officeHtmlBucket` falls back to MIME whenever the extension isn't an OFFICE extension. The client's `detectArtifactTypeFromFile` is stricter — it routes by extension first for ANY known extension (`.txt` → PLAIN_TEXT, `.md` → MARKDOWN, `.py` → CODE, etc.), only falling back to MIME when the extension is unknown. Mismatch case: `notes.txt` shipped with `Content-Type: application/ vnd.openxmlformats-officedocument.wordprocessingml.document`. Server runs `officeHtmlBucket` → extension `.txt` not office → MIME fallback → 'docx' → produces full HTML, sets `textFormat: 'html'`. Client routes by extension to PLAIN_TEXT (extension wins), markdown viewer escapes the HTML, user sees raw `<html>...` markup instead of the rendered preview. Fix: server only falls back to MIME when extension is genuinely empty (extensionless filename). Symmetric with the client's "extension wins for any known extension" semantic — neither will mis-route. Trade-off: a true DOCX renamed to `myfile.bin` with the canonical DOCX MIME no longer routes through office HTML on the server. The client would have routed to the office bucket via MIME, then the security gate (`textFormat !== 'html'`) would have downgraded to PLAIN_TEXT anyway. So the user-visible outcome is the same (raw bytes via PLAIN_TEXT) — the new behavior just avoids producing HTML that the client would never use. Long-term fix: share the extension routing table in data-provider so both server and client query the same source of truth. Out of scope for this PR. Tests: new 8-case `it.each` block in `officeHtmlBucket predicate` locks in the contract — `.txt`/`.md`/`.json`/`.py`/`.html`/`.css` + office MIME → null, and `.bin`/`.dat` + office MIME → null too. Existing extension-wins tests still pass unchanged. Total file tests: 534 (+8 vs before).
This commit is contained in:
parent
5683706af5
commit
f20419d0b7
28 changed files with 5244 additions and 143 deletions
|
|
@ -105,6 +105,7 @@
|
|||
"passport-local": "^1.0.0",
|
||||
"pdfjs-dist": "^5.4.624",
|
||||
"rate-limit-redis": "^4.2.0",
|
||||
"sanitize-html": "^2.13.0",
|
||||
"sharp": "^0.33.5",
|
||||
"traverse": "^0.6.7",
|
||||
"ua-parser-js": "^1.0.36",
|
||||
|
|
@ -116,6 +117,7 @@
|
|||
"zod": "^3.22.4"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/sanitize-html": "^2.13.0",
|
||||
"jest": "^30.2.0",
|
||||
"mongodb-memory-server": "^11.0.1",
|
||||
"nodemon": "^3.0.3",
|
||||
|
|
|
|||
|
|
@ -824,6 +824,19 @@ function createResponsesToolEndCallback({ req, res, tracker, artifactPromises })
|
|||
width: fileMetadata.width,
|
||||
height: fileMetadata.height,
|
||||
tool_call_id: output.tool_call_id,
|
||||
/* Inline text / sanitized HTML preview from
|
||||
* `extractCodeArtifactText` — drives the file artifact panel's
|
||||
* rich preview for DOCX/XLSX/CSV/PPTX. Pass null explicitly
|
||||
* (rather than undefined) so the wire format is stable for the
|
||||
* empty-text gate on the client. */
|
||||
text: fileMetadata.text ?? null,
|
||||
/* Trust signal so the client can route .docx/.csv/.xlsx/
|
||||
* .pptx attachments to the office HTML buckets only when
|
||||
* the backend produced sanitized full-document HTML.
|
||||
* RAG-uploaded plain text from mammoth.extractRawText
|
||||
* arrives without this flag and must NOT be injected as
|
||||
* HTML — Codex P1 review on PR #12934. */
|
||||
textFormat: fileMetadata.textFormat ?? null,
|
||||
};
|
||||
writeResponsesAttachment(res, tracker, attachment, metadata);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -27,6 +27,11 @@ jest.mock('@librechat/api', () => {
|
|||
createAxiosInstance: jest.fn(() => mockAxios),
|
||||
classifyCodeArtifact: jest.fn(() => 'other'),
|
||||
extractCodeArtifactText: jest.fn(async () => null),
|
||||
/* `processCodeOutput` calls this to derive the trust flag persisted
|
||||
* on `IMongoFile.textFormat` — Codex P1 review on PR #12934. The
|
||||
* mock returns null in lockstep with the null `text` above so
|
||||
* downstream consumers don't see a phantom format. */
|
||||
getExtractedTextFormat: jest.fn(() => null),
|
||||
codeServerHttpAgent: new http.Agent({ keepAlive: false }),
|
||||
codeServerHttpsAgent: new https.Agent({ keepAlive: false }),
|
||||
};
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ const {
|
|||
codeServerHttpAgent,
|
||||
codeServerHttpsAgent,
|
||||
extractCodeArtifactText,
|
||||
getExtractedTextFormat,
|
||||
} = require('@librechat/api');
|
||||
const {
|
||||
Tools,
|
||||
|
|
@ -271,6 +272,15 @@ const processCodeOutput = async ({
|
|||
const leafName = path.basename(safeName);
|
||||
const category = classifyCodeArtifact(leafName, mimeType);
|
||||
const text = await extractCodeArtifactText(buffer, leafName, mimeType, category);
|
||||
/* `textFormat` accompanies `text` so the client can gate
|
||||
* office-HTML-bucket routing on a trusted signal — clients MUST
|
||||
* NOT inject `text` into the iframe as HTML unless `textFormat ===
|
||||
* 'html'`. RAG-uploaded `.docx` etc. arrive with plain text from
|
||||
* mammoth.extractRawText and would otherwise be hijacked by the
|
||||
* extension-based office routing into the HTML-injection path
|
||||
* (Codex P1 review on PR #12934). null on extract failure — the
|
||||
* client treats absence as 'text' for safety. */
|
||||
const textFormat = getExtractedTextFormat(leafName, mimeType, text);
|
||||
|
||||
const file = {
|
||||
file_id,
|
||||
|
|
@ -293,6 +303,7 @@ const processCodeOutput = async ({
|
|||
// text — `createFile` uses findOneAndUpdate with $set semantics, which
|
||||
// would otherwise leave a stale value behind.
|
||||
text: text ?? null,
|
||||
textFormat: textFormat ?? null,
|
||||
};
|
||||
|
||||
await createFile(file, true);
|
||||
|
|
|
|||
|
|
@ -43,6 +43,7 @@ mockAxios.isAxiosError = jest.fn(() => false);
|
|||
|
||||
const mockClassifyCodeArtifact = jest.fn(() => 'other');
|
||||
const mockExtractCodeArtifactText = jest.fn(async () => null);
|
||||
const mockGetExtractedTextFormat = jest.fn((_name, _mime, text) => (text == null ? null : 'text'));
|
||||
jest.mock('@librechat/api', () => {
|
||||
const http = require('http');
|
||||
const https = require('https');
|
||||
|
|
@ -63,6 +64,15 @@ jest.mock('@librechat/api', () => {
|
|||
*/
|
||||
classifyCodeArtifact: (...args) => mockClassifyCodeArtifact(...args),
|
||||
extractCodeArtifactText: (...args) => mockExtractCodeArtifactText(...args),
|
||||
/* `processCodeOutput` derives the `textFormat` trust flag for
|
||||
* `IMongoFile` from this helper — Codex P1 review on PR #12934.
|
||||
* The mock returns 'text' for non-null extractor output and null
|
||||
* otherwise so the downstream `file.textFormat` field is set to
|
||||
* a believable shape without modeling the office-HTML branch
|
||||
* (the dispatcher under test isn't exercising that path). Per-
|
||||
* test overrides via `mockGetExtractedTextFormat.mockReturnValue`
|
||||
* if a case needs to assert the 'html' value. */
|
||||
getExtractedTextFormat: (...args) => mockGetExtractedTextFormat(...args),
|
||||
codeServerHttpAgent: new http.Agent({ keepAlive: false }),
|
||||
codeServerHttpsAgent: new https.Agent({ keepAlive: false }),
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
import { useRef, useState, useEffect, useCallback } from 'react';
|
||||
import { useRef, useState, useEffect, useCallback, useMemo } from 'react';
|
||||
import copy from 'copy-to-clipboard';
|
||||
import * as Tabs from '@radix-ui/react-tabs';
|
||||
import { Code, Play, RefreshCw, X } from 'lucide-react';
|
||||
|
|
@ -11,6 +11,8 @@ import useArtifacts from '~/hooks/Artifacts/useArtifacts';
|
|||
import DownloadArtifact from './DownloadArtifact';
|
||||
import ArtifactVersion from './ArtifactVersion';
|
||||
import ArtifactTabs from './ArtifactTabs';
|
||||
import { isPreviewOnlyArtifact } from '~/utils/artifacts';
|
||||
import { displayFilename } from '~/components/Chat/Messages/Content/Parts/attachmentTypes';
|
||||
import { useLocalize } from '~/hooks';
|
||||
import { cn } from '~/utils';
|
||||
import store from '~/store';
|
||||
|
|
@ -37,18 +39,21 @@ export default function Artifacts() {
|
|||
const setArtifactsVisible = useSetRecoilState(store.artifactsVisibility);
|
||||
const resetCurrentArtifactId = useResetRecoilState(store.currentArtifactId);
|
||||
|
||||
const tabOptions = [
|
||||
{
|
||||
value: 'code',
|
||||
label: localize('com_ui_code'),
|
||||
icon: <Code className="size-4" />,
|
||||
},
|
||||
{
|
||||
value: 'preview',
|
||||
label: localize('com_ui_preview'),
|
||||
icon: <Play className="size-4" />,
|
||||
},
|
||||
];
|
||||
const allTabOptions = useMemo(
|
||||
() => [
|
||||
{
|
||||
value: 'code',
|
||||
label: localize('com_ui_code'),
|
||||
icon: <Code className="size-4" />,
|
||||
},
|
||||
{
|
||||
value: 'preview',
|
||||
label: localize('com_ui_preview'),
|
||||
icon: <Play className="size-4" />,
|
||||
},
|
||||
],
|
||||
[localize],
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
setIsMounted(true);
|
||||
|
|
@ -88,6 +93,33 @@ export default function Artifacts() {
|
|||
setCurrentArtifactId,
|
||||
} = useArtifacts();
|
||||
|
||||
/* Office artifacts (DOCX/SPREADSHEET/PRESENTATION) have no source view —
|
||||
* the underlying file is binary and the "code" tab would display the
|
||||
* generated HTML blob, which isn't useful. Filter the tab options and
|
||||
* snap the active tab when the user lands on an office artifact while
|
||||
* the code tab is selected. */
|
||||
const isPreviewOnly = isPreviewOnlyArtifact(currentArtifact?.type);
|
||||
const tabOptions = useMemo(() => {
|
||||
if (!isPreviewOnly) {
|
||||
return allTabOptions;
|
||||
}
|
||||
/* When only the preview tab is shown, the generic "Preview" label is
|
||||
* a no-op pill — surface the document filename there instead. The
|
||||
* Play icon stays as a visual cue for "rendered preview". `displayFilename`
|
||||
* handles the sandbox dotfile suffix the upload pipeline applies. */
|
||||
const filename = displayFilename(currentArtifact?.title);
|
||||
const previewTab = allTabOptions.find((opt) => opt.value === 'preview');
|
||||
if (!previewTab) {
|
||||
return allTabOptions;
|
||||
}
|
||||
return [filename ? { ...previewTab, label: filename } : previewTab];
|
||||
}, [allTabOptions, isPreviewOnly, currentArtifact?.title]);
|
||||
useEffect(() => {
|
||||
if (isPreviewOnly && activeTab === 'code') {
|
||||
setActiveTab('preview');
|
||||
}
|
||||
}, [isPreviewOnly, activeTab, setActiveTab]);
|
||||
|
||||
const handleCopyArtifact = useCallback(() => {
|
||||
const content = currentArtifact?.content ?? '';
|
||||
if (!content) {
|
||||
|
|
|
|||
|
|
@ -183,15 +183,40 @@ describe('Attachment routing for tool artifacts', () => {
|
|||
expect(screen.queryByText('com_ui_artifact_click')).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('falls through to the inline <pre> for unsupported text types (CSV)', () => {
|
||||
const csv = baseAttachment({
|
||||
filename: 'data.csv',
|
||||
text: 'a,b,c\n1,2,3',
|
||||
it('falls through to the inline <pre> for unsupported text types (JSON)', () => {
|
||||
/* CSV used to fall through here, but now routes to the SPREADSHEET
|
||||
* preview bucket. JSON is still inline-only (no dedicated viewer
|
||||
* yet); use it as the canonical "unrouted text" example. */
|
||||
const json = baseAttachment({
|
||||
filename: 'data.json',
|
||||
type: 'application/json',
|
||||
text: '{"a":1,"b":2}',
|
||||
} as Partial<TAttachment>);
|
||||
const { container } = renderWith(<Attachment attachment={csv} />);
|
||||
const { container } = renderWith(<Attachment attachment={json} />);
|
||||
expect(container.querySelector('pre')).not.toBeNull();
|
||||
expect(screen.queryByTestId('mermaid-render')).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it.each([
|
||||
['data.csv', 'text/csv'],
|
||||
['workbook.xlsx', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'],
|
||||
['report.docx', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'],
|
||||
['deck.pptx', 'application/vnd.openxmlformats-officedocument.presentationml.presentation'],
|
||||
])('routes %s through the office preview panel (panel artifact)', (filename, type) => {
|
||||
const att = baseAttachment({
|
||||
file_id: `office-${filename}`,
|
||||
filename,
|
||||
type,
|
||||
text: '<!DOCTYPE html><body><table><tr><td>x</td></tr></table></body>',
|
||||
} as Partial<TAttachment>);
|
||||
renderWith(<Attachment attachment={att} />);
|
||||
expect(screen.getByText(filename)).toBeInTheDocument();
|
||||
/* Auto-pressed open button (streaming + non-CODE bucket) — same UX as
|
||||
* the HTML panel artifact above. */
|
||||
expect(screen.getByRole('button', { pressed: true })).toBeInTheDocument();
|
||||
const downloadPattern = new RegExp(`com_ui_download.*${filename.replace('.', '\\.')}`, 'i');
|
||||
expect(screen.getByRole('button', { name: downloadPattern })).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
|
||||
describe('ToolArtifactCard click behaviour', () => {
|
||||
|
|
@ -645,8 +670,9 @@ describe('AttachmentGroup routing', () => {
|
|||
} as Partial<TAttachment>),
|
||||
baseAttachment({
|
||||
file_id: 'c',
|
||||
filename: 'data.csv',
|
||||
text: 'a,b,c\n1,2,3',
|
||||
filename: 'data.json',
|
||||
type: 'application/json',
|
||||
text: '{"a":1}',
|
||||
} as Partial<TAttachment>),
|
||||
baseAttachment({
|
||||
file_id: 'd',
|
||||
|
|
@ -661,7 +687,7 @@ describe('AttachmentGroup routing', () => {
|
|||
expect(screen.getByText('index.html')).toBeInTheDocument();
|
||||
// Mermaid render
|
||||
expect(screen.getByTestId('mermaid-render')).toBeInTheDocument();
|
||||
// Inline text fallback for CSV
|
||||
// Inline text fallback for JSON (CSV now goes to SPREADSHEET)
|
||||
expect(container.querySelector('pre')).not.toBeNull();
|
||||
// FileContainer for the plain zip (and potentially others)
|
||||
expect(screen.getAllByTestId('file-container').length).toBeGreaterThan(0);
|
||||
|
|
|
|||
|
|
@ -103,13 +103,17 @@ describe('LogContent attachment routing', () => {
|
|||
expect(screen.queryByRole('button', { pressed: true })).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('routes text-bearing CSV through inline <pre>, not the panel', () => {
|
||||
const csv = baseAttachment({
|
||||
it('routes text-bearing JSON through inline <pre>, not the panel', () => {
|
||||
/* CSV used to fall through here, but now routes to the SPREADSHEET
|
||||
* preview bucket. JSON has no dedicated viewer yet, so it remains
|
||||
* the canonical "unrouted text" example. */
|
||||
const json = baseAttachment({
|
||||
file_id: 'c',
|
||||
filename: 'data.csv',
|
||||
text: 'a,b,c\n1,2,3',
|
||||
filename: 'data.json',
|
||||
type: 'application/json',
|
||||
text: '{"a":1,"b":2}',
|
||||
} as Partial<TAttachment>);
|
||||
const { container } = renderWith(<LogContent output="" attachments={[csv]} />);
|
||||
const { container } = renderWith(<LogContent output="" attachments={[json]} />);
|
||||
expect(container.querySelector('pre')).not.toBeNull();
|
||||
expect(screen.queryByRole('button', { pressed: true })).not.toBeInTheDocument();
|
||||
expect(screen.queryByTestId('mermaid-render')).not.toBeInTheDocument();
|
||||
|
|
@ -125,28 +129,47 @@ describe('LogContent attachment routing', () => {
|
|||
expect(screen.getByTestId('log-link')).toHaveAttribute('data-filename', 'archive.zip');
|
||||
});
|
||||
|
||||
it('renders a panel card for pptx with empty text using the placeholder', () => {
|
||||
// pptx text extraction is deferred; the artifact still routes through
|
||||
// the panel and gets the localized placeholder content. Here the
|
||||
// localize mock returns the key, so we assert by class/structure.
|
||||
it('renders a panel card for a pptx with backend-rendered HTML in text', () => {
|
||||
/* PPTX (and DOCX/XLSX/CSV) now route through the office preview
|
||||
* bucket with a strict empty-text gate — the artifact only registers
|
||||
* once the backend's `bufferToOfficeHtml` has produced the slide-list
|
||||
* HTML and shipped it via `attachment.text`. */
|
||||
const pptx = baseAttachment({
|
||||
file_id: 'e',
|
||||
filename: 'slides.pptx',
|
||||
text: undefined as unknown as string,
|
||||
type: 'application/vnd.openxmlformats-officedocument.presentationml.presentation',
|
||||
text: '<!DOCTYPE html><body><ol><li>Slide 1</li></ol></body>',
|
||||
} as Partial<TAttachment>);
|
||||
renderWith(<LogContent output="" attachments={[pptx]} />);
|
||||
expect(screen.getByText('slides.pptx')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('falls back to the legacy download branch for an office file with no extracted text', () => {
|
||||
/* Empty `text` for an office type fails the strict gate, so the
|
||||
* artifact stays unregistered and the file flows to the download
|
||||
* fallback (LogLink), not a half-rendered panel card. */
|
||||
const pptx = baseAttachment({
|
||||
file_id: 'e2',
|
||||
filename: 'slides.pptx',
|
||||
type: 'application/vnd.openxmlformats-officedocument.presentationml.presentation',
|
||||
text: undefined as unknown as string,
|
||||
} as Partial<TAttachment>);
|
||||
renderWith(<LogContent output="" attachments={[pptx]} />);
|
||||
expect(screen.queryByRole('button', { pressed: true })).not.toBeInTheDocument();
|
||||
expect(screen.getByTestId('log-link')).toHaveAttribute('data-filename', 'slides.pptx');
|
||||
});
|
||||
|
||||
it('routes an expired panel-eligible attachment through the legacy expired path', () => {
|
||||
// Without this gate, an expired pptx/html/etc. would render as a
|
||||
// clickable artifact card backed by a dead download link. The
|
||||
// legacy "download expired" message must win for any panel-eligible
|
||||
// entry whose `expiresAt` is in the past.
|
||||
/* Without this gate, an expired pptx/html/etc. with extracted HTML
|
||||
* would render as a clickable artifact card backed by a dead
|
||||
* download link. The legacy "download expired" message must win for
|
||||
* any panel-eligible entry whose `expiresAt` is in the past — even
|
||||
* when the office HTML is present. */
|
||||
const expired = baseAttachment({
|
||||
file_id: 'x-expired',
|
||||
filename: 'slides.pptx',
|
||||
text: undefined as unknown as string,
|
||||
type: 'application/vnd.openxmlformats-officedocument.presentationml.presentation',
|
||||
text: '<!DOCTYPE html><body><ol><li>Slide 1</li></ol></body>',
|
||||
expiresAt: Date.now() - 60_000,
|
||||
} as Partial<TAttachment>);
|
||||
renderWith(<LogContent output="" attachments={[expired]} />);
|
||||
|
|
@ -183,8 +206,13 @@ describe('LogContent attachment routing', () => {
|
|||
} as Partial<TAttachment>),
|
||||
baseAttachment({
|
||||
file_id: 't',
|
||||
filename: 'notes.csv',
|
||||
text: 'a,b\n1,2',
|
||||
/* JSON stays on the inline `<pre>` rendering path. CSV used to
|
||||
* live here too but now routes to the SPREADSHEET preview
|
||||
* bucket, so it would no longer satisfy the "inline pre" check
|
||||
* below. */
|
||||
filename: 'notes.json',
|
||||
type: 'application/json',
|
||||
text: '{"a":1,"b":2}',
|
||||
} as Partial<TAttachment>),
|
||||
] as TAttachment[];
|
||||
const { container } = renderWith(
|
||||
|
|
@ -194,7 +222,7 @@ describe('LogContent attachment routing', () => {
|
|||
expect(screen.getByText('index.html')).toBeInTheDocument();
|
||||
// mermaid render present
|
||||
expect(screen.getByTestId('mermaid-render')).toBeInTheDocument();
|
||||
// CSV inline pre present
|
||||
// JSON inline pre present
|
||||
expect(container.querySelector('pre')).not.toBeNull();
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -56,10 +56,15 @@ jest.mock('~/utils', () => ({
|
|||
const textAttachment = (overrides: Partial<TAttachment> = {}): TAttachment =>
|
||||
({
|
||||
file_id: 'file-1',
|
||||
filename: 'output.csv',
|
||||
filepath: '/files/output.csv',
|
||||
type: 'text/csv',
|
||||
text: 'a,b,c\n1,2,3',
|
||||
/* JSON stays on the inline `<pre>` rendering path. CSV used to live
|
||||
* here too but now routes through the SPREADSHEET artifact panel
|
||||
* (Recoil-bound), so a CSV fixture would force every test in this
|
||||
* file to add a `RecoilRoot` wrapper. JSON has the same shape (text-
|
||||
* bearing, downloadable, expandable) without the panel coupling. */
|
||||
filename: 'output.json',
|
||||
filepath: '/files/output.json',
|
||||
type: 'application/json',
|
||||
text: '{"a":1,"b":2,"c":3}',
|
||||
...overrides,
|
||||
}) as TAttachment;
|
||||
|
||||
|
|
@ -102,7 +107,7 @@ describe('TextAttachment (via Attachment default export)', () => {
|
|||
const { container } = render(<Attachment attachment={textAttachment()} />);
|
||||
const pre = container.querySelector('pre');
|
||||
expect(pre).not.toBeNull();
|
||||
expect(pre!.textContent).toBe('a,b,c\n1,2,3');
|
||||
expect(pre!.textContent).toBe('{"a":1,"b":2,"c":3}');
|
||||
});
|
||||
|
||||
it('renders a download chip when filepath is present', () => {
|
||||
|
|
@ -163,10 +168,11 @@ describe('AttachmentGroup', () => {
|
|||
});
|
||||
|
||||
it('routes text-bearing attachments through the text rendering path', () => {
|
||||
// `.csv` is text-bearing but not artifact-eligible (CSV gets a
|
||||
// dedicated viewer in a follow-up), so it falls through to the
|
||||
// inline <pre> renderer rather than the side panel card.
|
||||
const attachments = [textAttachment({ file_id: 'a', filename: 'a.csv' })] as TAttachment[];
|
||||
/* `.json` is text-bearing but not artifact-eligible (JSON has no
|
||||
* dedicated viewer yet), so it falls through to the inline <pre>
|
||||
* renderer rather than the side panel card. CSV used to live here
|
||||
* too but now routes through the SPREADSHEET artifact panel. */
|
||||
const attachments = [textAttachment({ file_id: 'a', filename: 'a.json' })] as TAttachment[];
|
||||
const { container } = render(<AttachmentGroup attachments={attachments} />);
|
||||
expect(container.querySelector('pre')).not.toBeNull();
|
||||
});
|
||||
|
|
|
|||
|
|
@ -133,9 +133,13 @@ describe('artifactTypeForAttachment', () => {
|
|||
});
|
||||
|
||||
it('returns null for unsupported extensions', () => {
|
||||
/* CSV / DOCX / XLSX / PPTX now route through the office preview
|
||||
* buckets (rich HTML preview); use a binary type with no preview
|
||||
* pipeline instead. */
|
||||
const attachment = baseAttachment({
|
||||
filename: 'data.csv',
|
||||
text: 'a,b,c',
|
||||
filename: 'photo.jpg',
|
||||
type: 'image/jpeg',
|
||||
text: undefined,
|
||||
} as Partial<TAttachment>);
|
||||
expect(artifactTypeForAttachment(attachment)).toBeNull();
|
||||
});
|
||||
|
|
|
|||
|
|
@ -45,6 +45,21 @@ export default function useArtifactProps({ artifact }: { artifact: Artifact }) {
|
|||
return ['content.md', getMarkdownFiles(artifact.content ?? '')];
|
||||
}
|
||||
|
||||
/* Office preview buckets (DOCX/SPREADSHEET/PRESENTATION): the backend
|
||||
* already produced a complete sanitized HTML document via
|
||||
* `bufferToOfficeHtml` and shipped it as `attachment.text`. Hand it
|
||||
* to the Sandpack `static` template's `index.html` slot directly —
|
||||
* no wrapping, no transformation, no client-side parsing libs. The
|
||||
* empty-text gate in `detectArtifactTypeFromFile` guarantees we
|
||||
* never reach this branch with an empty content payload. */
|
||||
if (
|
||||
type === TOOL_ARTIFACT_TYPES.DOCX ||
|
||||
type === TOOL_ARTIFACT_TYPES.SPREADSHEET ||
|
||||
type === TOOL_ARTIFACT_TYPES.PRESENTATION
|
||||
) {
|
||||
return ['index.html', { 'index.html': artifact.content ?? '' }];
|
||||
}
|
||||
|
||||
const fileKey = getArtifactFilename(artifact.type ?? '', artifact.language);
|
||||
const files = removeNullishValues({
|
||||
[fileKey]: artifact.content,
|
||||
|
|
|
|||
|
|
@ -2,9 +2,11 @@ import {
|
|||
buildSandpackOptions,
|
||||
detectArtifactTypeFromFile,
|
||||
fileToArtifact,
|
||||
isPreviewOnlyArtifact,
|
||||
languageForFilename,
|
||||
TOOL_ARTIFACT_TYPES,
|
||||
} from '../artifacts';
|
||||
import type { ToolArtifactType } from '../artifacts';
|
||||
|
||||
const TAILWIND_CDN = 'https://cdn.tailwindcss.com/3.4.17#tailwind.js';
|
||||
|
||||
|
|
@ -55,35 +57,195 @@ describe('detectArtifactTypeFromFile', () => {
|
|||
['flow.mmd', TOOL_ARTIFACT_TYPES.MERMAID],
|
||||
['flow.mermaid', TOOL_ARTIFACT_TYPES.MERMAID],
|
||||
['readme.txt', TOOL_ARTIFACT_TYPES.PLAIN_TEXT],
|
||||
['report.docx', TOOL_ARTIFACT_TYPES.PLAIN_TEXT],
|
||||
['notes.odt', TOOL_ARTIFACT_TYPES.PLAIN_TEXT],
|
||||
['slides.pptx', TOOL_ARTIFACT_TYPES.PLAIN_TEXT],
|
||||
['report.docx', TOOL_ARTIFACT_TYPES.DOCX],
|
||||
['data.csv', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
['workbook.xlsx', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
['legacy.xls', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
['sheet.ods', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
['slides.pptx', TOOL_ARTIFACT_TYPES.PRESENTATION],
|
||||
])('classifies %s by extension', (filename, expected) => {
|
||||
expect(detectArtifactTypeFromFile({ filename, type: '', text: 'content' })).toBe(expected);
|
||||
/* Office types require `textFormat: 'html'` to route to their HTML
|
||||
* preview buckets — the security gate added for Codex P1 review on
|
||||
* PR #12934. Plain-text/markdown/code/etc. don't take that path so
|
||||
* they pass through unchanged. */
|
||||
const isOfficeBucket = (
|
||||
[
|
||||
TOOL_ARTIFACT_TYPES.DOCX,
|
||||
TOOL_ARTIFACT_TYPES.SPREADSHEET,
|
||||
TOOL_ARTIFACT_TYPES.PRESENTATION,
|
||||
] as ToolArtifactType[]
|
||||
).includes(expected);
|
||||
const textFormat = isOfficeBucket ? ('html' as const) : undefined;
|
||||
expect(detectArtifactTypeFromFile({ filename, type: '', text: 'content', textFormat })).toBe(
|
||||
expected,
|
||||
);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['index.html', TOOL_ARTIFACT_TYPES.HTML],
|
||||
['App.tsx', TOOL_ARTIFACT_TYPES.REACT],
|
||||
['flow.mmd', TOOL_ARTIFACT_TYPES.MERMAID],
|
||||
])('returns null when %s has no text (%s viewer needs real content)', (filename) => {
|
||||
expect(detectArtifactTypeFromFile({ filename, type: '', text: '' })).toBeNull();
|
||||
expect(detectArtifactTypeFromFile({ filename, type: '', text: undefined })).toBeNull();
|
||||
});
|
||||
['index.html', TOOL_ARTIFACT_TYPES.HTML, undefined],
|
||||
['App.tsx', TOOL_ARTIFACT_TYPES.REACT, undefined],
|
||||
['flow.mmd', TOOL_ARTIFACT_TYPES.MERMAID, undefined],
|
||||
/* Office preview buckets need server-rendered HTML in `text` to render
|
||||
* — the empty-text gate keeps the artifact off the panel until the
|
||||
* backend's `bufferToOfficeHtml` finishes. The `textFormat: 'html'`
|
||||
* trust flag is required for the routing to even land on the office
|
||||
* bucket; without it, the security gate downgrades to PLAIN_TEXT
|
||||
* (Codex P1 review). The strict empty-text gate then returns null. */
|
||||
['report.docx', TOOL_ARTIFACT_TYPES.DOCX, 'html' as const],
|
||||
['data.csv', TOOL_ARTIFACT_TYPES.SPREADSHEET, 'html' as const],
|
||||
['workbook.xlsx', TOOL_ARTIFACT_TYPES.SPREADSHEET, 'html' as const],
|
||||
['slides.pptx', TOOL_ARTIFACT_TYPES.PRESENTATION, 'html' as const],
|
||||
])(
|
||||
'returns null when %s has no text (renderer needs real content)',
|
||||
(filename, _expected, textFormat) => {
|
||||
expect(detectArtifactTypeFromFile({ filename, type: '', text: '', textFormat })).toBeNull();
|
||||
expect(
|
||||
detectArtifactTypeFromFile({ filename, type: '', text: undefined, textFormat }),
|
||||
).toBeNull();
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
['readme.txt', TOOL_ARTIFACT_TYPES.PLAIN_TEXT],
|
||||
['slides.pptx', TOOL_ARTIFACT_TYPES.PLAIN_TEXT],
|
||||
['report.docx', TOOL_ARTIFACT_TYPES.PLAIN_TEXT],
|
||||
['notes.md', TOOL_ARTIFACT_TYPES.MARKDOWN],
|
||||
['notes.odt', TOOL_ARTIFACT_TYPES.PLAIN_TEXT],
|
||||
])(
|
||||
'still routes %s through the panel without text (deferred-extraction case)',
|
||||
'still routes %s through the panel without text (deferred-extraction case for plain-text/markdown)',
|
||||
(filename, expected) => {
|
||||
expect(detectArtifactTypeFromFile({ filename, type: '', text: '' })).toBe(expected);
|
||||
expect(detectArtifactTypeFromFile({ filename, type: '', text: undefined })).toBe(expected);
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
[
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
TOOL_ARTIFACT_TYPES.DOCX,
|
||||
],
|
||||
[
|
||||
'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
|
||||
TOOL_ARTIFACT_TYPES.SPREADSHEET,
|
||||
],
|
||||
['application/vnd.ms-excel', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
['application/vnd.oasis.opendocument.spreadsheet', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
['text/csv', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
['application/csv', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
/* Legacy CSV MIME variant — backend's `CSV_MIME_PATTERN` accepts
|
||||
* it, so the client must too or extensionless CSVs with this MIME
|
||||
* would be skipped despite the backend producing valid HTML.
|
||||
* Regression for Codex P3 review on PR #12934. */
|
||||
['text/comma-separated-values', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
/* Legacy XLS MIME aliases — backend's `excelMimeTypes` regex
|
||||
* accepts the full set used by older browsers/servers; the client
|
||||
* mirrors via the same regex so an extensionless XLS with any of
|
||||
* these legacy MIMEs gets routed to the spreadsheet bucket and
|
||||
* the artifact actually shows up on the panel. Regression for
|
||||
* Codex P1 review on PR #12934. */
|
||||
['application/x-ms-excel', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
['application/x-msexcel', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
['application/msexcel', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
['application/x-excel', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
['application/x-dos_ms_excel', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
['application/xls', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
['application/x-xls', TOOL_ARTIFACT_TYPES.SPREADSHEET],
|
||||
[
|
||||
'application/vnd.openxmlformats-officedocument.presentationml.presentation',
|
||||
TOOL_ARTIFACT_TYPES.PRESENTATION,
|
||||
],
|
||||
])('routes office MIME %s to its preview bucket when extension is missing', (mime, expected) => {
|
||||
/* `textFormat: 'html'` is required so the security gate (Codex P1 on
|
||||
* PR #12934) lets routing proceed to the office HTML bucket — without
|
||||
* it, the gate would downgrade to PLAIN_TEXT to keep RAG-extracted
|
||||
* plain-text from being injected as HTML. */
|
||||
expect(
|
||||
detectArtifactTypeFromFile({
|
||||
filename: 'noext',
|
||||
type: mime,
|
||||
text: '<html>x</html>',
|
||||
textFormat: 'html',
|
||||
}),
|
||||
).toBe(expected);
|
||||
});
|
||||
|
||||
/* Codex P1 SECURITY: office HTML routing requires the backend's explicit
|
||||
* `textFormat: 'html'` trust flag. Without it (e.g. RAG-uploaded `.docx`
|
||||
* with mammoth.extractRawText plain text in `attachment.text`, or any
|
||||
* legacy attachment from before this flag existed), routing falls back
|
||||
* to PLAIN_TEXT so the markdown viewer escapes content rather than
|
||||
* letting useArtifactProps inject the text as `index.html`. */
|
||||
it('downgrades office types to PLAIN_TEXT when textFormat is missing (legacy attachments)', () => {
|
||||
expect(
|
||||
detectArtifactTypeFromFile({
|
||||
filename: 'report.docx',
|
||||
type: '',
|
||||
text: 'Plain text mammoth extracted from a docx — must NOT render as HTML.',
|
||||
}),
|
||||
).toBe(TOOL_ARTIFACT_TYPES.PLAIN_TEXT);
|
||||
expect(
|
||||
detectArtifactTypeFromFile({
|
||||
filename: 'data.csv',
|
||||
type: 'text/csv',
|
||||
text: 'col1,col2\n1,2',
|
||||
}),
|
||||
).toBe(TOOL_ARTIFACT_TYPES.PLAIN_TEXT);
|
||||
expect(
|
||||
detectArtifactTypeFromFile({
|
||||
filename: 'slides.pptx',
|
||||
type: '',
|
||||
text: 'Slide 1: Intro\nSlide 2: Outro',
|
||||
}),
|
||||
).toBe(TOOL_ARTIFACT_TYPES.PLAIN_TEXT);
|
||||
});
|
||||
|
||||
it('downgrades office types to PLAIN_TEXT when textFormat is "text" (explicit non-HTML)', () => {
|
||||
/* The backend marks RAG/text-extraction output as `textFormat: 'text'`
|
||||
* to make the trust contract explicit. Either no flag or 'text' both
|
||||
* route to PLAIN_TEXT — only 'html' unlocks the office HTML bucket. */
|
||||
expect(
|
||||
detectArtifactTypeFromFile({
|
||||
filename: 'report.docx',
|
||||
type: '',
|
||||
text: '<script>alert(1)</script>',
|
||||
textFormat: 'text',
|
||||
}),
|
||||
).toBe(TOOL_ARTIFACT_TYPES.PLAIN_TEXT);
|
||||
});
|
||||
|
||||
it('downgrades office types to PLAIN_TEXT when textFormat is null (DB legacy)', () => {
|
||||
/* Mongoose returns `null` for fields the document was saved without
|
||||
* — covers attachments persisted before the textFormat field existed. */
|
||||
expect(
|
||||
detectArtifactTypeFromFile({
|
||||
filename: 'workbook.xlsx',
|
||||
type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
|
||||
text: '<table><tr><td>1</td></tr></table>',
|
||||
textFormat: null,
|
||||
}),
|
||||
).toBe(TOOL_ARTIFACT_TYPES.PLAIN_TEXT);
|
||||
});
|
||||
|
||||
/* Regression: an office attachment with no `text` AND no `textFormat`
|
||||
* must NOT downgrade to PLAIN_TEXT (which has the lenient empty-text
|
||||
* gate and would render as a half-empty panel card). The historical
|
||||
* contract for office types with missing extraction is "fall through
|
||||
* to the legacy download UI"; the security gate's empty-text exception
|
||||
* preserves that. CI regression on PR #12934 — `LogContent.test.tsx`
|
||||
* "falls back to the legacy download branch for an office file with
|
||||
* no extracted text" was the canary. */
|
||||
it.each([
|
||||
['report.docx', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'],
|
||||
['data.csv', 'text/csv'],
|
||||
['workbook.xlsx', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'],
|
||||
['slides.pptx', 'application/vnd.openxmlformats-officedocument.presentationml.presentation'],
|
||||
])(
|
||||
'%s with no text and no textFormat returns null (preserves legacy download path)',
|
||||
(filename, type) => {
|
||||
expect(detectArtifactTypeFromFile({ filename, type, text: '' })).toBeNull();
|
||||
expect(detectArtifactTypeFromFile({ filename, type, text: undefined })).toBeNull();
|
||||
},
|
||||
);
|
||||
|
||||
it('falls back to MIME when the extension is unknown', () => {
|
||||
expect(detectArtifactTypeFromFile({ filename: 'noext', type: 'text/html', text: 'x' })).toBe(
|
||||
TOOL_ARTIFACT_TYPES.HTML,
|
||||
|
|
@ -114,12 +276,14 @@ describe('detectArtifactTypeFromFile', () => {
|
|||
});
|
||||
|
||||
it('returns null for unsupported types', () => {
|
||||
expect(
|
||||
detectArtifactTypeFromFile({ filename: 'output.csv', type: 'text/csv', text: 'a,b' }),
|
||||
).toBeNull();
|
||||
/* PDFs have no rich preview path on the client (the artifact panel
|
||||
* doesn't host a PDF viewer); they fall back to the download UI. */
|
||||
expect(
|
||||
detectArtifactTypeFromFile({ filename: 'doc.pdf', type: 'application/pdf', text: 'x' }),
|
||||
).toBeNull();
|
||||
expect(
|
||||
detectArtifactTypeFromFile({ filename: 'photo.jpg', type: 'image/jpeg', text: 'binary' }),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it('does not route bare text/plain MIME without a recognized extension', () => {
|
||||
|
|
@ -177,12 +341,11 @@ describe('detectArtifactTypeFromFile', () => {
|
|||
);
|
||||
});
|
||||
|
||||
it('does NOT route data formats to CODE (CSV / JSON / YAML / TOML / XML)', () => {
|
||||
it('does NOT route data formats to CODE (JSON / YAML / TOML / XML)', () => {
|
||||
/* These get dedicated viewers in follow-ups; for now they fall
|
||||
* through to inline rendering (return null). */
|
||||
expect(
|
||||
detectArtifactTypeFromFile({ filename: 'data.csv', type: 'text/csv', text: 'a,b' }),
|
||||
).toBeNull();
|
||||
* through to inline rendering (return null). CSV is the exception:
|
||||
* it routes to the SPREADSHEET preview bucket — covered separately
|
||||
* in the "classifies %s by extension" suite above. */
|
||||
expect(
|
||||
detectArtifactTypeFromFile({ filename: 'data.json', type: 'application/json', text: '{}' }),
|
||||
).toBeNull();
|
||||
|
|
@ -397,7 +560,10 @@ describe('fileToArtifact', () => {
|
|||
});
|
||||
|
||||
it('returns null for unsupported types so callers can fall through', () => {
|
||||
expect(fileToArtifact({ ...baseFile, filename: 'data.csv', type: 'text/csv' })).toBeNull();
|
||||
expect(fileToArtifact({ ...baseFile, filename: 'photo.jpg', type: 'image/jpeg' })).toBeNull();
|
||||
expect(
|
||||
fileToArtifact({ ...baseFile, filename: 'doc.pdf', type: 'application/pdf' }),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
/* End-to-end test for the CODE bucket. The classification path is
|
||||
|
|
@ -502,13 +668,18 @@ describe('fileToArtifact', () => {
|
|||
});
|
||||
|
||||
it('uses the caller-provided placeholder when a deferred-extraction file has no text', () => {
|
||||
// Backend extractor returns null for pptx (deferred). Client sees
|
||||
// `text === null` and substitutes the localized placeholder.
|
||||
/* Plain-text and markdown remain on the lenient empty-text gate so the
|
||||
* artifact card can render a "preparing preview…" placeholder while
|
||||
* extraction is in flight. (Office preview buckets — DOCX, SPREADSHEET,
|
||||
* PRESENTATION — use the strict gate instead: their renderers need
|
||||
* server-rendered HTML, so the artifact stays unregistered until the
|
||||
* `text` field arrives. See the strict-gate test in the
|
||||
* `detectArtifactTypeFromFile` suite.) */
|
||||
const artifact = fileToArtifact(
|
||||
{
|
||||
...baseFile,
|
||||
filename: 'slides.pptx',
|
||||
type: 'application/vnd.openxmlformats-officedocument.presentationml.presentation',
|
||||
filename: 'notes.txt',
|
||||
type: 'text/plain',
|
||||
text: null as unknown as string,
|
||||
},
|
||||
{ placeholder: '_Coming soon_' },
|
||||
|
|
@ -521,13 +692,115 @@ describe('fileToArtifact', () => {
|
|||
it('falls back to empty content when no placeholder is supplied and text is missing', () => {
|
||||
const artifact = fileToArtifact({
|
||||
...baseFile,
|
||||
filename: 'slides.pptx',
|
||||
type: 'application/vnd.openxmlformats-officedocument.presentationml.presentation',
|
||||
filename: 'notes.txt',
|
||||
type: 'text/plain',
|
||||
text: undefined,
|
||||
});
|
||||
expect(artifact).not.toBeNull();
|
||||
expect(artifact!.content).toBe('');
|
||||
});
|
||||
|
||||
it('returns null for office preview buckets without text (strict gate)', () => {
|
||||
/* `textFormat: 'html'` is required for routing to land on the office
|
||||
* bucket in the first place; without it the security gate downgrades
|
||||
* to PLAIN_TEXT (which has the lenient empty-text gate). The strict
|
||||
* empty-text gate then fires and the artifact stays unregistered. */
|
||||
expect(
|
||||
fileToArtifact({
|
||||
...baseFile,
|
||||
filename: 'slides.pptx',
|
||||
type: 'application/vnd.openxmlformats-officedocument.presentationml.presentation',
|
||||
text: undefined,
|
||||
textFormat: 'html',
|
||||
}),
|
||||
).toBeNull();
|
||||
expect(
|
||||
fileToArtifact({
|
||||
...baseFile,
|
||||
filename: 'report.docx',
|
||||
type: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
text: '',
|
||||
textFormat: 'html',
|
||||
}),
|
||||
).toBeNull();
|
||||
expect(
|
||||
fileToArtifact({
|
||||
...baseFile,
|
||||
filename: 'data.csv',
|
||||
type: 'text/csv',
|
||||
text: undefined,
|
||||
textFormat: 'html',
|
||||
}),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it('builds a SPREADSHEET artifact for csv/xlsx with backend-rendered HTML in text', () => {
|
||||
const csv = fileToArtifact({
|
||||
...baseFile,
|
||||
filename: 'data.csv',
|
||||
type: 'text/csv',
|
||||
text: '<!DOCTYPE html><table><tr><td>1</td></tr></table>',
|
||||
textFormat: 'html',
|
||||
});
|
||||
expect(csv).not.toBeNull();
|
||||
expect(csv!.type).toBe(TOOL_ARTIFACT_TYPES.SPREADSHEET);
|
||||
expect(csv!.title).toBe('data.csv');
|
||||
expect(csv!.content).toContain('<table>');
|
||||
|
||||
const xlsx = fileToArtifact({
|
||||
...baseFile,
|
||||
filename: 'workbook.xlsx',
|
||||
type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
|
||||
text: '<!DOCTYPE html><body>sheet</body>',
|
||||
textFormat: 'html',
|
||||
});
|
||||
expect(xlsx).not.toBeNull();
|
||||
expect(xlsx!.type).toBe(TOOL_ARTIFACT_TYPES.SPREADSHEET);
|
||||
});
|
||||
|
||||
it('builds a DOCX artifact for .docx with backend-rendered HTML in text', () => {
|
||||
const artifact = fileToArtifact({
|
||||
...baseFile,
|
||||
filename: 'report.docx',
|
||||
type: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
text: '<!DOCTYPE html><body><p>hello</p></body>',
|
||||
textFormat: 'html',
|
||||
});
|
||||
expect(artifact).not.toBeNull();
|
||||
expect(artifact!.type).toBe(TOOL_ARTIFACT_TYPES.DOCX);
|
||||
});
|
||||
|
||||
it('builds a PRESENTATION artifact for .pptx with backend-rendered HTML in text', () => {
|
||||
const artifact = fileToArtifact({
|
||||
...baseFile,
|
||||
filename: 'deck.pptx',
|
||||
type: 'application/vnd.openxmlformats-officedocument.presentationml.presentation',
|
||||
text: '<!DOCTYPE html><body><ol><li>Slide 1</li></ol></body>',
|
||||
textFormat: 'html',
|
||||
});
|
||||
expect(artifact).not.toBeNull();
|
||||
expect(artifact!.type).toBe(TOOL_ARTIFACT_TYPES.PRESENTATION);
|
||||
});
|
||||
|
||||
/* Codex P1 SECURITY companion: legacy/RAG path where `textFormat` is
|
||||
* missing — the security gate downgrades to PLAIN_TEXT, which has the
|
||||
* lenient empty-text gate. The text round-trips into the markdown
|
||||
* viewer as escaped content rather than being injected as HTML. */
|
||||
it('downgrades to a PLAIN_TEXT artifact when an office file has text but no textFormat flag', () => {
|
||||
const artifact = fileToArtifact({
|
||||
...baseFile,
|
||||
filename: 'report.docx',
|
||||
type: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
/* This is what mammoth.extractRawText returns for a RAG upload —
|
||||
* not sanitized HTML, just the document's flowed text. The
|
||||
* security gate ensures it's never injected as HTML. */
|
||||
text: 'Document body text from extractRawText. <script>alert(1)</script>',
|
||||
});
|
||||
expect(artifact).not.toBeNull();
|
||||
expect(artifact!.type).toBe(TOOL_ARTIFACT_TYPES.PLAIN_TEXT);
|
||||
expect(artifact!.content).toContain('<script>');
|
||||
});
|
||||
|
||||
it('preserves an empty string as legitimate content (does not fall through to placeholder)', () => {
|
||||
// A user can write a 0-byte `.md` or `.txt`; that's a valid artifact
|
||||
// with empty content, not "extraction unavailable."
|
||||
|
|
@ -585,3 +858,33 @@ describe('fileToArtifact', () => {
|
|||
expect(artifact!.id).toBe('tool-artifact-index.html');
|
||||
});
|
||||
});
|
||||
|
||||
describe('isPreviewOnlyArtifact', () => {
|
||||
/* The Artifacts panel hides the "code" tab and snaps `activeTab` to
|
||||
* 'preview' when the current artifact is preview-only — i.e. the
|
||||
* underlying file is binary and the generated HTML blob isn't a
|
||||
* useful "code" view. Regression for review finding #6 on PR #12934.
|
||||
* Without this test, removing a type from the predicate (or adding a
|
||||
* non-office type) would silently leave users seeing the raw HTML
|
||||
* blob in the code tab. */
|
||||
it.each([
|
||||
[TOOL_ARTIFACT_TYPES.DOCX, true],
|
||||
[TOOL_ARTIFACT_TYPES.SPREADSHEET, true],
|
||||
[TOOL_ARTIFACT_TYPES.PRESENTATION, true],
|
||||
[TOOL_ARTIFACT_TYPES.HTML, false],
|
||||
[TOOL_ARTIFACT_TYPES.REACT, false],
|
||||
[TOOL_ARTIFACT_TYPES.MARKDOWN, false],
|
||||
[TOOL_ARTIFACT_TYPES.MERMAID, false],
|
||||
[TOOL_ARTIFACT_TYPES.CODE, false],
|
||||
[TOOL_ARTIFACT_TYPES.PLAIN_TEXT, false],
|
||||
])('type %s returns %s', (type, expected) => {
|
||||
expect(isPreviewOnlyArtifact(type)).toBe(expected);
|
||||
});
|
||||
|
||||
it.each([[null], [undefined], [''], ['application/pdf'], ['text/plain'], ['some/random-type']])(
|
||||
'returns false for non-artifact type %s',
|
||||
(type) => {
|
||||
expect(isPreviewOnlyArtifact(type)).toBe(false);
|
||||
},
|
||||
);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
import dedent from 'dedent';
|
||||
import { shadcnComponents } from 'librechat-data-provider';
|
||||
import { excelMimeTypes, shadcnComponents } from 'librechat-data-provider';
|
||||
import type {
|
||||
SandpackProviderProps,
|
||||
SandpackPredefinedTemplate,
|
||||
|
|
@ -12,6 +12,13 @@ const artifactFilename = {
|
|||
'application/vnd.ant.react': 'App.tsx',
|
||||
'text/html': 'index.html',
|
||||
'application/vnd.code-html': 'index.html',
|
||||
/* Office preview buckets — the backend produces a complete sanitized
|
||||
* `index.html` document (head + body) and ships it via `attachment.text`.
|
||||
* The Sandpack `static` template loads it as-is. See
|
||||
* `packages/api/src/files/documents/html.ts`. */
|
||||
'application/vnd.librechat.docx-preview': 'index.html',
|
||||
'application/vnd.librechat.spreadsheet-preview': 'index.html',
|
||||
'application/vnd.librechat.presentation-preview': 'index.html',
|
||||
// mermaid and markdown types are handled separately in useArtifactProps.ts
|
||||
default: 'index.html',
|
||||
// 'css': 'css',
|
||||
|
|
@ -44,6 +51,11 @@ const artifactTemplate: Record<
|
|||
'text/markdown': 'static',
|
||||
'text/md': 'static',
|
||||
'text/plain': 'static',
|
||||
/* Office preview buckets ride the same static pipeline — the backend
|
||||
* already sanitized the HTML, so we just hand it to Sandpack. */
|
||||
'application/vnd.librechat.docx-preview': 'static',
|
||||
'application/vnd.librechat.spreadsheet-preview': 'static',
|
||||
'application/vnd.librechat.presentation-preview': 'static',
|
||||
default: 'static',
|
||||
// 'css': 'css',
|
||||
// 'javascript': 'js',
|
||||
|
|
@ -137,6 +149,11 @@ const dependenciesMap: Record<
|
|||
'text/markdown': {},
|
||||
'text/md': {},
|
||||
'text/plain': {},
|
||||
/* Office preview HTML is fully self-contained (CSS-only sheet tabs, no
|
||||
* JS), so no Sandpack-side packages are needed. */
|
||||
'application/vnd.librechat.docx-preview': {},
|
||||
'application/vnd.librechat.spreadsheet-preview': {},
|
||||
'application/vnd.librechat.presentation-preview': {},
|
||||
default: standardDependencies,
|
||||
};
|
||||
|
||||
|
|
@ -265,10 +282,44 @@ export const TOOL_ARTIFACT_TYPES = {
|
|||
MERMAID: 'application/vnd.mermaid',
|
||||
PLAIN_TEXT: 'text/plain',
|
||||
CODE: 'application/vnd.code',
|
||||
/* Office-format rich previews. The backend renders the binary file as a
|
||||
* complete sanitized HTML document and ships it via `attachment.text`;
|
||||
* the client routes these types through the Sandpack `static` template's
|
||||
* `index.html` slot. The values are synthetic LibreChat-internal MIMEs
|
||||
* — they don't appear on disk or in HTTP headers, only on the artifact
|
||||
* object — so they can't collide with the canonical office MIMEs that
|
||||
* the routing maps key off of. */
|
||||
DOCX: 'application/vnd.librechat.docx-preview',
|
||||
SPREADSHEET: 'application/vnd.librechat.spreadsheet-preview',
|
||||
PRESENTATION: 'application/vnd.librechat.presentation-preview',
|
||||
} as const;
|
||||
|
||||
export type ToolArtifactType = (typeof TOOL_ARTIFACT_TYPES)[keyof typeof TOOL_ARTIFACT_TYPES];
|
||||
|
||||
/**
|
||||
* Artifact types whose preview is server-rendered HTML — there's no
|
||||
* source for a "code" view because the underlying file is binary, and
|
||||
* showing the generated HTML blob in a code editor would just be
|
||||
* noise. Used by the artifacts panel to hide the code tab and snap
|
||||
* the active tab to "preview" when one of these is selected.
|
||||
*
|
||||
* Exposed via a predicate (rather than the bare set) so callers can't
|
||||
* accidentally widen the set and so the membership check is unit-
|
||||
* testable without mounting the full Artifacts component.
|
||||
*/
|
||||
const PREVIEW_ONLY_ARTIFACT_TYPES: ReadonlySet<ToolArtifactType> = new Set([
|
||||
TOOL_ARTIFACT_TYPES.DOCX,
|
||||
TOOL_ARTIFACT_TYPES.SPREADSHEET,
|
||||
TOOL_ARTIFACT_TYPES.PRESENTATION,
|
||||
]);
|
||||
|
||||
export function isPreviewOnlyArtifact(type: string | null | undefined): boolean {
|
||||
if (type == null) {
|
||||
return false;
|
||||
}
|
||||
return PREVIEW_ONLY_ARTIFACT_TYPES.has(type as ToolArtifactType);
|
||||
}
|
||||
|
||||
/**
|
||||
* Extension → fenced-code-block language hint for the CODE bucket. The
|
||||
* key is the lowercased file extension (no dot); the value is the
|
||||
|
|
@ -501,13 +552,19 @@ const EXTENSION_TO_TOOL_ARTIFACT_TYPE: Record<string, ToolArtifactType> = {
|
|||
mdx: TOOL_ARTIFACT_TYPES.MARKDOWN,
|
||||
mmd: TOOL_ARTIFACT_TYPES.MERMAID,
|
||||
mermaid: TOOL_ARTIFACT_TYPES.MERMAID,
|
||||
// Plain text + office documents fall through to the markdown-style
|
||||
// viewer until dedicated renderers land. `pptx` is wired up here so
|
||||
// the routing fires as soon as backend text extraction is added.
|
||||
txt: TOOL_ARTIFACT_TYPES.PLAIN_TEXT,
|
||||
docx: TOOL_ARTIFACT_TYPES.PLAIN_TEXT,
|
||||
// ODT has no rich HTML producer — it stays on the markdown-text path.
|
||||
odt: TOOL_ARTIFACT_TYPES.PLAIN_TEXT,
|
||||
pptx: TOOL_ARTIFACT_TYPES.PLAIN_TEXT,
|
||||
/* Office formats with rich HTML previews. The backend's
|
||||
* `extractCodeArtifactText` path produces a complete sanitized HTML
|
||||
* document via `bufferToOfficeHtml` and ships it through
|
||||
* `attachment.text`. */
|
||||
docx: TOOL_ARTIFACT_TYPES.DOCX,
|
||||
csv: TOOL_ARTIFACT_TYPES.SPREADSHEET,
|
||||
xlsx: TOOL_ARTIFACT_TYPES.SPREADSHEET,
|
||||
xls: TOOL_ARTIFACT_TYPES.SPREADSHEET,
|
||||
ods: TOOL_ARTIFACT_TYPES.SPREADSHEET,
|
||||
pptx: TOOL_ARTIFACT_TYPES.PRESENTATION,
|
||||
};
|
||||
|
||||
/* Append every entry in `CODE_EXTENSION_TO_LANGUAGE` to the routing map
|
||||
|
|
@ -563,14 +620,28 @@ const MIME_TO_TOOL_ARTIFACT_TYPE: Record<string, ToolArtifactType> = {
|
|||
'text/x-lua': TOOL_ARTIFACT_TYPES.CODE,
|
||||
'text/x-swift': TOOL_ARTIFACT_TYPES.CODE,
|
||||
'text/css': TOOL_ARTIFACT_TYPES.CODE,
|
||||
// Office MIME types fall through to the plain-text bucket here too —
|
||||
// matches the extension map so a file whose extension was stripped
|
||||
// somewhere upstream still routes to the panel.
|
||||
// Office MIME types — route to the rich HTML preview buckets when the
|
||||
// canonical MIME is present. ODT remains on the plain-text path (no
|
||||
// dedicated HTML producer). These complement the extension map for
|
||||
// extensionless filenames.
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document':
|
||||
TOOL_ARTIFACT_TYPES.PLAIN_TEXT,
|
||||
TOOL_ARTIFACT_TYPES.DOCX,
|
||||
'application/vnd.oasis.opendocument.text': TOOL_ARTIFACT_TYPES.PLAIN_TEXT,
|
||||
'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet':
|
||||
TOOL_ARTIFACT_TYPES.SPREADSHEET,
|
||||
'application/vnd.ms-excel': TOOL_ARTIFACT_TYPES.SPREADSHEET,
|
||||
'application/vnd.oasis.opendocument.spreadsheet': TOOL_ARTIFACT_TYPES.SPREADSHEET,
|
||||
'text/csv': TOOL_ARTIFACT_TYPES.SPREADSHEET,
|
||||
'application/csv': TOOL_ARTIFACT_TYPES.SPREADSHEET,
|
||||
/* `text/comma-separated-values` is a legacy CSV MIME variant — rare
|
||||
* in modern HTTP traffic but still emitted by some sandboxes. Kept
|
||||
* in lock-step with the backend's `CSV_MIME_PATTERN` in
|
||||
* `packages/api/src/files/documents/html.ts` so an extensionless CSV
|
||||
* with this MIME doesn't slip through the client routing while the
|
||||
* backend has already produced full HTML for it. */
|
||||
'text/comma-separated-values': TOOL_ARTIFACT_TYPES.SPREADSHEET,
|
||||
'application/vnd.openxmlformats-officedocument.presentationml.presentation':
|
||||
TOOL_ARTIFACT_TYPES.PLAIN_TEXT,
|
||||
TOOL_ARTIFACT_TYPES.PRESENTATION,
|
||||
// Note: bare `text/plain` is NOT mapped here. The extension map handles
|
||||
// `.txt` explicitly; routing every unrecognized-extension `text/plain`
|
||||
// file (extensionless scripts, .env, etc.) through the panel would be a
|
||||
|
|
@ -591,8 +662,21 @@ const MIME_TO_TOOL_ARTIFACT_TYPE: Record<string, ToolArtifactType> = {
|
|||
* and Mermaid buckets still require real content because their viewers
|
||||
* (sandpack / mermaid.js) error on empty input.
|
||||
*/
|
||||
/**
|
||||
* Office preview buckets the backend MUST mark as `textFormat: 'html'`
|
||||
* before the client will inject `attachment.text` as `index.html`.
|
||||
* Routing to these buckets without the trust flag would let plain
|
||||
* text from RAG-uploaded `.docx` etc. (mammoth.extractRawText output)
|
||||
* be rendered as HTML — Codex P1 review on PR #12934.
|
||||
*/
|
||||
const OFFICE_HTML_BUCKETS: ReadonlySet<ToolArtifactType> = new Set([
|
||||
TOOL_ARTIFACT_TYPES.DOCX,
|
||||
TOOL_ARTIFACT_TYPES.SPREADSHEET,
|
||||
TOOL_ARTIFACT_TYPES.PRESENTATION,
|
||||
]);
|
||||
|
||||
export function detectArtifactTypeFromFile(
|
||||
attachment: Partial<Pick<TFile, 'filename' | 'type' | 'text'>>,
|
||||
attachment: Partial<Pick<TFile, 'filename' | 'type' | 'text' | 'textFormat'>>,
|
||||
): ToolArtifactType | null {
|
||||
/* Compute the basename once and reuse it across the extension AND
|
||||
* bare-name lookups. Both `extensionOf(filename)` and
|
||||
|
|
@ -608,17 +692,60 @@ export function detectArtifactTypeFromFile(
|
|||
const byBareName = byExtension
|
||||
? undefined
|
||||
: EXTENSION_TO_TOOL_ARTIFACT_TYPE[bareNameFromBasename(base)];
|
||||
const type =
|
||||
byExtension ?? byBareName ?? MIME_TO_TOOL_ARTIFACT_TYPE[baseMime(attachment.type)] ?? null;
|
||||
/* Exact-match MIME lookup first; for the spreadsheet bucket the
|
||||
* backend's `officeHtmlBucket` accepts the broad `excelMimeTypes`
|
||||
* regex (covers `application/x-ms-excel`, `application/x-xls`,
|
||||
* `application/msexcel`, `application/x-dos_ms_excel`, etc.). The
|
||||
* client must accept the same set or extensionless XLS uploads with
|
||||
* legacy MIMEs would have backend HTML produced but never get
|
||||
* routed/registered on the panel. */
|
||||
const normalizedMime = baseMime(attachment.type);
|
||||
const byMime =
|
||||
MIME_TO_TOOL_ARTIFACT_TYPE[normalizedMime] ??
|
||||
(excelMimeTypes.test(normalizedMime) ? TOOL_ARTIFACT_TYPES.SPREADSHEET : undefined);
|
||||
const type = byExtension ?? byBareName ?? byMime ?? null;
|
||||
if (type == null) {
|
||||
return null;
|
||||
}
|
||||
/* SECURITY GATE: office HTML buckets inject `attachment.text` into
|
||||
* the iframe via `index.html`. Plain text from a RAG-extracted
|
||||
* .docx (mammoth output) would become executable markup here.
|
||||
* Require the backend's explicit `textFormat: 'html'` trust signal
|
||||
* before allowing this routing — fall back to PLAIN_TEXT (markdown
|
||||
* viewer, escapes content) for everything else.
|
||||
*
|
||||
* Legacy attachments stored before this field existed have
|
||||
* `textFormat === undefined`; they may have HTML in `text` from the
|
||||
* pre-flag period and the safe default ('text' bucket) is OK for
|
||||
* those — they were rendering correctly before this flag, and the
|
||||
* markdown viewer escaping HTML is a safety upgrade, not a
|
||||
* regression. Codex P1 review on PR #12934.
|
||||
*
|
||||
* Empty-text exception: if the office attachment has NO text at all,
|
||||
* downgrading to PLAIN_TEXT would route an empty-text-tolerant
|
||||
* bucket onto the panel (a half-rendered card showing only the
|
||||
* filename). The historical contract for office types with missing
|
||||
* text is "fall through to the legacy download path"; preserve that
|
||||
* by returning null here. The downgrade only matters when there's
|
||||
* actual text content that needs safe-escaping. */
|
||||
if (OFFICE_HTML_BUCKETS.has(type) && attachment.textFormat !== 'html') {
|
||||
if (!attachment.text) {
|
||||
return null;
|
||||
}
|
||||
return TOOL_ARTIFACT_TYPES.PLAIN_TEXT;
|
||||
}
|
||||
if (
|
||||
!attachment.text &&
|
||||
type !== TOOL_ARTIFACT_TYPES.PLAIN_TEXT &&
|
||||
type !== TOOL_ARTIFACT_TYPES.MARKDOWN &&
|
||||
type !== TOOL_ARTIFACT_TYPES.CODE
|
||||
) {
|
||||
/* HTML, REACT, MERMAID, and the office preview buckets all require
|
||||
* real content — their renderers (sandpack iframes / mermaid.js /
|
||||
* the office HTML pipeline) error or render blank without it. The
|
||||
* artifact stays unregistered until the backend produces text;
|
||||
* `ToolArtifactCard`'s self-heal effect re-fires on drift so the
|
||||
* card transitions cleanly when text arrives. */
|
||||
return null;
|
||||
}
|
||||
return type;
|
||||
|
|
@ -700,6 +827,12 @@ export function fileToArtifact(
|
|||
type !== TOOL_ARTIFACT_TYPES.MARKDOWN &&
|
||||
type !== TOOL_ARTIFACT_TYPES.CODE
|
||||
) {
|
||||
/* HTML, REACT, MERMAID, and the office preview buckets all require
|
||||
* real content — their renderers (sandpack iframes / mermaid.js /
|
||||
* the office HTML pipeline) error or render blank without it. The
|
||||
* artifact stays unregistered until the backend produces text;
|
||||
* `ToolArtifactCard`'s self-heal effect re-fires on drift so the
|
||||
* card transitions cleanly when text arrives. */
|
||||
return null;
|
||||
}
|
||||
/* For CODE artifacts, resolve the language hint at construction time
|
||||
|
|
|
|||
46
package-lock.json
generated
46
package-lock.json
generated
|
|
@ -120,6 +120,7 @@
|
|||
"passport-local": "^1.0.0",
|
||||
"pdfjs-dist": "^5.4.624",
|
||||
"rate-limit-redis": "^4.2.0",
|
||||
"sanitize-html": "^2.13.0",
|
||||
"sharp": "^0.33.5",
|
||||
"traverse": "^0.6.7",
|
||||
"ua-parser-js": "^1.0.36",
|
||||
|
|
@ -131,6 +132,7 @@
|
|||
"zod": "^3.22.4"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/sanitize-html": "^2.13.0",
|
||||
"jest": "^30.2.0",
|
||||
"mongodb-memory-server": "^11.0.1",
|
||||
"nodemon": "^3.0.3",
|
||||
|
|
@ -21178,6 +21180,16 @@
|
|||
"integrity": "sha512-60BCwRFOZCQhDncwQdxxeOEEkbc5dIMccYLwbxsS4TUNeVECQ/pBJ0j09mrHOl/JJvpRPGwO9SvE4nR2Nb/a4Q==",
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/@types/sanitize-html": {
|
||||
"version": "2.16.1",
|
||||
"resolved": "https://registry.npmjs.org/@types/sanitize-html/-/sanitize-html-2.16.1.tgz",
|
||||
"integrity": "sha512-n9wjs8bCOTyN/ynwD8s/nTcTreIHB1vf31vhLMGqUPNHaweKC4/fAl4Dj+hUlCTKYgm4P3k83fmiFfzkZ6sgMA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"htmlparser2": "^10.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/scheduler": {
|
||||
"version": "0.16.8",
|
||||
"resolved": "https://registry.npmjs.org/@types/scheduler/-/scheduler-0.16.8.tgz",
|
||||
|
|
@ -25380,7 +25392,6 @@
|
|||
"version": "4.3.1",
|
||||
"resolved": "https://registry.npmjs.org/deepmerge/-/deepmerge-4.3.1.tgz",
|
||||
"integrity": "sha512-3sUqbMEc77XqpdNO7FRyRog+eW3ph+GYCbj+rK+uYyRMuwsVy0rMiVtPn+QJlKFvWP/1PYpapqYn0Me2knFn+A==",
|
||||
"dev": true,
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
|
|
@ -26177,7 +26188,6 @@
|
|||
"version": "4.0.0",
|
||||
"resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-4.0.0.tgz",
|
||||
"integrity": "sha512-TtpcNJ3XAzx3Gq8sWRzJaVajRs0uVxA2YAkdb1jm2YkPz4G6egUFAyA3n5vtEIZefPk5Wa4UXbKuS5fKkJWdgA==",
|
||||
"dev": true,
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
},
|
||||
|
|
@ -29585,6 +29595,15 @@
|
|||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/is-plain-object": {
|
||||
"version": "5.0.0",
|
||||
"resolved": "https://registry.npmjs.org/is-plain-object/-/is-plain-object-5.0.0.tgz",
|
||||
"integrity": "sha512-VRSzKkbMm5jMDoKLbltAkFQ5Qr7VDiTFGXxYFXXowVj387GeGNOCsOH6Msy00SGZ3Fp84b1Naa1psqgcCIEP5Q==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/is-potential-custom-element-name": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/is-potential-custom-element-name/-/is-potential-custom-element-name-1.0.1.tgz",
|
||||
|
|
@ -35341,6 +35360,12 @@
|
|||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/parse-srcset": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/parse-srcset/-/parse-srcset-1.0.2.tgz",
|
||||
"integrity": "sha512-/2qh0lav6CmI15FzA3i/2Bzk2zCgQhGMkvhOhKNcBVQ1ldgpbfiNTVslmooUmWJcADi1f1kIeynbDRVzNlfR6Q==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/parse5": {
|
||||
"version": "7.3.0",
|
||||
"resolved": "https://registry.npmjs.org/parse5/-/parse5-7.3.0.tgz",
|
||||
|
|
@ -39963,6 +39988,20 @@
|
|||
"resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz",
|
||||
"integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg=="
|
||||
},
|
||||
"node_modules/sanitize-html": {
|
||||
"version": "2.17.3",
|
||||
"resolved": "https://registry.npmjs.org/sanitize-html/-/sanitize-html-2.17.3.tgz",
|
||||
"integrity": "sha512-Kn4srCAo2+wZyvCNKCSyB2g8RQ8IkX/gQs2uqoSRNu5t9I2qvUyAVvRDiFUVAiX3N3PNuwStY0eNr+ooBHVWEg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"deepmerge": "^4.2.2",
|
||||
"escape-string-regexp": "^4.0.0",
|
||||
"htmlparser2": "^10.1.0",
|
||||
"is-plain-object": "^5.0.0",
|
||||
"parse-srcset": "^1.0.2",
|
||||
"postcss": "^8.3.11"
|
||||
}
|
||||
},
|
||||
"node_modules/sax": {
|
||||
"version": "1.5.0",
|
||||
"resolved": "https://registry.npmjs.org/sax/-/sax-1.5.0.tgz",
|
||||
|
|
@ -44243,6 +44282,7 @@
|
|||
"@types/node": "^20.3.0",
|
||||
"@types/node-fetch": "^2.6.13",
|
||||
"@types/react": "^18.2.18",
|
||||
"@types/sanitize-html": "^2.13.0",
|
||||
"@types/winston": "^2.4.4",
|
||||
"@types/yauzl": "^2.10.3",
|
||||
"aws-sdk-client-mock": "^4.1.0",
|
||||
|
|
@ -44256,6 +44296,7 @@
|
|||
"rimraf": "^6.1.3",
|
||||
"rollup": "^4.34.9",
|
||||
"rollup-plugin-peer-deps-external": "^2.2.4",
|
||||
"sanitize-html": "^2.13.0",
|
||||
"ts-node": "^10.9.2",
|
||||
"typescript": "^5.0.4",
|
||||
"xlsx": "https://cdn.sheetjs.com/xlsx-0.20.3/xlsx-0.20.3.tgz",
|
||||
|
|
@ -44297,6 +44338,7 @@
|
|||
"node-fetch": "2.7.0",
|
||||
"pdfjs-dist": "^5.4.624",
|
||||
"rate-limit-redis": "^4.2.0",
|
||||
"sanitize-html": "^2.13.0",
|
||||
"sharp": "^0.33.5",
|
||||
"undici": "^7.24.1",
|
||||
"yauzl": "^3.2.1",
|
||||
|
|
|
|||
|
|
@ -64,6 +64,7 @@
|
|||
"@types/node": "^20.3.0",
|
||||
"@types/node-fetch": "^2.6.13",
|
||||
"@types/react": "^18.2.18",
|
||||
"@types/sanitize-html": "^2.13.0",
|
||||
"@types/winston": "^2.4.4",
|
||||
"@types/yauzl": "^2.10.3",
|
||||
"aws-sdk-client-mock": "^4.1.0",
|
||||
|
|
@ -77,6 +78,7 @@
|
|||
"rimraf": "^6.1.3",
|
||||
"rollup": "^4.34.9",
|
||||
"rollup-plugin-peer-deps-external": "^2.2.4",
|
||||
"sanitize-html": "^2.13.0",
|
||||
"ts-node": "^10.9.2",
|
||||
"typescript": "^5.0.4",
|
||||
"xlsx": "https://cdn.sheetjs.com/xlsx-0.20.3/xlsx-0.20.3.tgz",
|
||||
|
|
@ -122,6 +124,7 @@
|
|||
"node-fetch": "2.7.0",
|
||||
"pdfjs-dist": "^5.4.624",
|
||||
"rate-limit-redis": "^4.2.0",
|
||||
"sanitize-html": "^2.13.0",
|
||||
"sharp": "^0.33.5",
|
||||
"undici": "^7.24.1",
|
||||
"yauzl": "^3.2.1",
|
||||
|
|
|
|||
|
|
@ -823,6 +823,25 @@ export interface AttachmentData {
|
|||
height?: number;
|
||||
/** Associated tool call ID */
|
||||
tool_call_id?: string;
|
||||
/**
|
||||
* Inline text or sanitized HTML preview (sized to MAX_TEXT_CACHE_BYTES).
|
||||
* Populated by `extractCodeArtifactText` for tool-output files: raw text
|
||||
* for plain-text artifacts, sanitized rich HTML for office formats
|
||||
* (DOCX/XLSX/CSV/PPTX). The frontend feeds HTML through the Sandpack
|
||||
* `static` template via `index.html`. Null if extraction was unavailable
|
||||
* (binary, oversized, or unsupported).
|
||||
*/
|
||||
text?: string | null;
|
||||
/**
|
||||
* Format of the `text` field — `'html'` if `text` is a complete
|
||||
* sanitized HTML document the client is permitted to inject into the
|
||||
* iframe via `index.html`, `'text'` if it's plain text. Clients MUST
|
||||
* gate office-bucket routing on `textFormat === 'html'`; legacy
|
||||
* attachments and RAG-extracted plain text don't have this set and
|
||||
* must default to safe (markdown-escaped) rendering. Codex P1 review
|
||||
* on PR #12934.
|
||||
*/
|
||||
textFormat?: 'html' | 'text' | null;
|
||||
/** Additional metadata */
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,21 +1,52 @@
|
|||
import * as os from 'os';
|
||||
import * as path from 'path';
|
||||
import { extractCodeArtifactText, MAX_TEXT_CACHE_BYTES, MAX_TEXT_EXTRACT_BYTES } from './extract';
|
||||
import {
|
||||
extractCodeArtifactText,
|
||||
getExtractedTextFormat,
|
||||
MAX_TEXT_CACHE_BYTES,
|
||||
MAX_TEXT_EXTRACT_BYTES,
|
||||
} from './extract';
|
||||
|
||||
const docxText = '__DOCX_PARSED__';
|
||||
/* parseDocument throws on any originalname containing this token, so
|
||||
* tests can force a failure with whatever extension/MIME they need
|
||||
* (the legacy docx-named constant is preserved for backward-compat
|
||||
* with tests that still reference it). */
|
||||
const docxFailureName = 'force-docx-failure.docx';
|
||||
const parseDocumentCalls: Array<{ path: string; originalname: string }> = [];
|
||||
|
||||
jest.mock('~/files/documents/crud', () => ({
|
||||
parseDocument: jest.fn(async ({ file }: { file: { path: string; originalname: string } }) => {
|
||||
parseDocumentCalls.push({ path: file.path, originalname: file.originalname });
|
||||
if (file.originalname === docxFailureName) {
|
||||
if (file.originalname.includes('force-docx-failure')) {
|
||||
throw new Error('parse failed');
|
||||
}
|
||||
return { text: docxText, filename: file.originalname, bytes: docxText.length };
|
||||
}),
|
||||
}));
|
||||
|
||||
/* The office HTML producer is mocked here so the existing fallback-path
|
||||
* assertions (parseDocument receives the canonical MIME) keep exercising
|
||||
* `extractDocument`. Tests that need real HTML output drive `bufferToOfficeHtml`
|
||||
* directly via its own spec file (`html.spec.ts`); a separate `office-html`
|
||||
* describe block below exercises the integration with this mock relaxed.
|
||||
*
|
||||
* `officeHtmlBucket` is the gate predicate the upstream uses to decide
|
||||
* whether to call the dispatcher at all. We pass through to the real
|
||||
* implementation so the gate routes the same files in tests as in prod. */
|
||||
const mockOfficeHtml = jest.fn(
|
||||
async (_buffer: Buffer, _name: string, _mime: string) => null as string | null,
|
||||
);
|
||||
jest.mock('~/files/documents/html', () => {
|
||||
const actual =
|
||||
jest.requireActual<typeof import('~/files/documents/html')>('~/files/documents/html');
|
||||
return {
|
||||
bufferToOfficeHtml: (buffer: Buffer, name: string, mime: string) =>
|
||||
mockOfficeHtml(buffer, name, mime),
|
||||
officeHtmlBucket: actual.officeHtmlBucket,
|
||||
};
|
||||
});
|
||||
|
||||
describe('extractCodeArtifactText', () => {
|
||||
describe('utf8-text', () => {
|
||||
it('decodes a UTF-8 buffer', async () => {
|
||||
|
|
@ -70,73 +101,78 @@ describe('extractCodeArtifactText', () => {
|
|||
});
|
||||
|
||||
describe('document', () => {
|
||||
/* These tests exercise the legacy `extractDocument` path, which now
|
||||
* only fires for `category === 'document'` files that the office HTML
|
||||
* dispatcher does NOT claim — i.e. PDF and ODT. All ZIP-backed office
|
||||
* formats (DOCX/XLSX/XLS/ODS) bypass `extractDocument` entirely
|
||||
* because they're HTML-or-null per the SEC fix in PR #12934
|
||||
* (Codex P1 review: text-fallback under `index.html` was XSS). */
|
||||
beforeEach(() => {
|
||||
parseDocumentCalls.length = 0;
|
||||
});
|
||||
|
||||
it('routes through parseDocument and returns its text', async () => {
|
||||
const buffer = Buffer.from('PKfake-docx');
|
||||
it('routes through parseDocument and returns its text (ODT)', async () => {
|
||||
const buffer = Buffer.from('PKfake-odt');
|
||||
const text = await extractCodeArtifactText(
|
||||
buffer,
|
||||
'report.docx',
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
'notes.odt',
|
||||
'application/vnd.oasis.opendocument.text',
|
||||
'document',
|
||||
);
|
||||
expect(text).toBe(docxText);
|
||||
});
|
||||
|
||||
it('returns null when parseDocument throws', async () => {
|
||||
const buffer = Buffer.from('PKfake-docx');
|
||||
const buffer = Buffer.from('PKfake-odt');
|
||||
const text = await extractCodeArtifactText(
|
||||
buffer,
|
||||
docxFailureName,
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
docxFailureName.replace('.docx', '.odt'),
|
||||
'application/vnd.oasis.opendocument.text',
|
||||
'document',
|
||||
);
|
||||
expect(text).toBeNull();
|
||||
});
|
||||
|
||||
it('rewrites a generic sniffed MIME to the canonical document MIME by extension', async () => {
|
||||
it('rewrites a generic sniffed MIME to the canonical document MIME by extension (ODT)', async () => {
|
||||
// Code-output buffers for office docs are commonly sniffed as
|
||||
// application/zip — without canonicalization, parseDocument would
|
||||
// reject these and inline previews would silently disappear.
|
||||
const buffer = Buffer.from('PKfake-docx');
|
||||
await extractCodeArtifactText(buffer, 'report.docx', 'application/zip', 'document');
|
||||
expect(parseDocumentCalls[0]?.originalname).toBe('report.docx');
|
||||
const buffer = Buffer.from('PKfake-odt');
|
||||
await extractCodeArtifactText(buffer, 'notes.odt', 'application/zip', 'document');
|
||||
expect(parseDocumentCalls[0]?.originalname).toBe('notes.odt');
|
||||
});
|
||||
|
||||
it.each([
|
||||
[
|
||||
'report.docx',
|
||||
'application/zip',
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
],
|
||||
[
|
||||
'data.xlsx',
|
||||
'application/octet-stream',
|
||||
'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
|
||||
],
|
||||
['legacy.xls', 'application/octet-stream', 'application/vnd.ms-excel'],
|
||||
['sheet.ods', 'application/zip', 'application/vnd.oasis.opendocument.spreadsheet'],
|
||||
['notes.odt', 'application/zip', 'application/vnd.oasis.opendocument.text'],
|
||||
])('passes canonical mimetype for %s when sniff returns %s', async (name, sniffed, _canon) => {
|
||||
const parseDocumentMock = (
|
||||
jest.requireMock('~/files/documents/crud') as {
|
||||
parseDocument: jest.Mock;
|
||||
}
|
||||
).parseDocument;
|
||||
parseDocumentMock.mockClear();
|
||||
await extractCodeArtifactText(Buffer.from('PK'), name, sniffed, 'document');
|
||||
const call = parseDocumentMock.mock.calls[0]?.[0];
|
||||
expect(call?.file?.mimetype).toBe(_canon);
|
||||
});
|
||||
['report.pdf', 'application/octet-stream', 'application/pdf'],
|
||||
])(
|
||||
'passes canonical mimetype for %s when sniff returns %s (legacy parseDocument path)',
|
||||
async (name, sniffed, _canon) => {
|
||||
const parseDocumentMock = (
|
||||
jest.requireMock('~/files/documents/crud') as {
|
||||
parseDocument: jest.Mock;
|
||||
}
|
||||
).parseDocument;
|
||||
parseDocumentMock.mockClear();
|
||||
await extractCodeArtifactText(Buffer.from('PK'), name, sniffed, 'document');
|
||||
const call = parseDocumentMock.mock.calls[0]?.[0];
|
||||
/* PDF doesn't have an entry in `documentMimeFromExtension` so the
|
||||
* sniffed MIME passes through unchanged. ODT does — gets
|
||||
* canonicalized to ODT_MIME. */
|
||||
expect(call?.file?.mimetype).toBe(_canon === 'application/pdf' ? sniffed : _canon);
|
||||
},
|
||||
);
|
||||
|
||||
it('writes the temp file inside os.tmpdir() regardless of artifact name', async () => {
|
||||
const buffer = Buffer.from('PKfake-docx');
|
||||
/* Path traversal defense — even a malicious filename like
|
||||
* `../../../etc/passwd.odt` must end up inside os.tmpdir() with a
|
||||
* sanitized basename. Uses ODT since the office HTML path now
|
||||
* short-circuits all ZIP-backed office formats. */
|
||||
const buffer = Buffer.from('PKfake-odt');
|
||||
await extractCodeArtifactText(
|
||||
buffer,
|
||||
'../../../etc/passwd.docx',
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
'../../../etc/passwd.odt',
|
||||
'application/vnd.oasis.opendocument.text',
|
||||
'document',
|
||||
);
|
||||
const call = parseDocumentCalls[0];
|
||||
|
|
@ -144,12 +180,30 @@ describe('extractCodeArtifactText', () => {
|
|||
const tmpRoot = path.resolve(os.tmpdir());
|
||||
expect(path.resolve(call.path).startsWith(tmpRoot)).toBe(true);
|
||||
expect(call.path).not.toContain('..');
|
||||
expect(call.originalname).toBe('passwd.docx');
|
||||
expect(call.originalname).toBe('passwd.odt');
|
||||
});
|
||||
|
||||
it('does NOT call parseDocument for office HTML types (DOCX/XLSX/XLS/ODS)', async () => {
|
||||
/* Lock in the SEC contract: the four office HTML buckets are
|
||||
* HTML-or-null and never fall back to `extractDocument`. */
|
||||
mockOfficeHtml.mockResolvedValue(null);
|
||||
const cases: Array<[string, string]> = [
|
||||
['report.docx', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'],
|
||||
['data.xlsx', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'],
|
||||
['legacy.xls', 'application/vnd.ms-excel'],
|
||||
['sheet.ods', 'application/vnd.oasis.opendocument.spreadsheet'],
|
||||
];
|
||||
for (const [name, mime] of cases) {
|
||||
const text = await extractCodeArtifactText(Buffer.from('PK'), name, mime, 'document');
|
||||
expect(text).toBeNull();
|
||||
}
|
||||
expect(parseDocumentCalls.length).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('skipped categories', () => {
|
||||
it('returns null for pptx', async () => {
|
||||
it('returns null for pptx when HTML rendering also fails', async () => {
|
||||
mockOfficeHtml.mockResolvedValueOnce(null);
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('PK'),
|
||||
'slides.pptx',
|
||||
|
|
@ -169,4 +223,377 @@ describe('extractCodeArtifactText', () => {
|
|||
expect(text).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('office-html', () => {
|
||||
beforeEach(() => {
|
||||
mockOfficeHtml.mockReset();
|
||||
parseDocumentCalls.length = 0;
|
||||
});
|
||||
|
||||
it('returns the HTML rendering for a docx when the producer succeeds', async () => {
|
||||
mockOfficeHtml.mockResolvedValueOnce('<!DOCTYPE html><html><body>docx html</body></html>');
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('PK'),
|
||||
'report.docx',
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
'document',
|
||||
);
|
||||
expect(text).toBe('<!DOCTYPE html><html><body>docx html</body></html>');
|
||||
expect(parseDocumentCalls.length).toBe(0);
|
||||
});
|
||||
|
||||
it('returns the HTML rendering for a pptx when the producer succeeds', async () => {
|
||||
mockOfficeHtml.mockResolvedValueOnce('<!DOCTYPE html><html><body>slides</body></html>');
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('PK'),
|
||||
'deck.pptx',
|
||||
'application/vnd.openxmlformats-officedocument.presentationml.presentation',
|
||||
'pptx',
|
||||
);
|
||||
expect(text).toContain('slides');
|
||||
});
|
||||
|
||||
it('returns the HTML rendering for csv (overriding utf8-text raw output)', async () => {
|
||||
mockOfficeHtml.mockResolvedValueOnce('<!DOCTYPE html><table><tr><td>a</td></tr></table>');
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('a,b\n1,2', 'utf-8'),
|
||||
'data.csv',
|
||||
'text/csv',
|
||||
'utf8-text',
|
||||
);
|
||||
expect(text).toContain('<table>');
|
||||
});
|
||||
|
||||
/* SECURITY: office types are HTML-or-null, with NO text fallback.
|
||||
* Codex P1 review on PR #12934 caught that the previous fallback
|
||||
* shipped raw text under an `index.html` slot on the client — a
|
||||
* literal `<script>` in document body would have been rendered as
|
||||
* executable markup. The tests below lock in the safe contract:
|
||||
* failed HTML rendering → null → file becomes download-only. */
|
||||
it('returns null (does not fall back to raw text) when CSV HTML rendering fails', async () => {
|
||||
mockOfficeHtml.mockResolvedValueOnce(null);
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('a,b\n1,2\n', 'utf-8'),
|
||||
'data.csv',
|
||||
'text/csv',
|
||||
'utf8-text',
|
||||
);
|
||||
expect(text).toBeNull();
|
||||
});
|
||||
|
||||
it('returns null (does not fall back to parseDocument) when DOCX HTML rendering returns null', async () => {
|
||||
mockOfficeHtml.mockResolvedValueOnce(null);
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('PKfake-docx'),
|
||||
'report.docx',
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
'document',
|
||||
);
|
||||
expect(text).toBeNull();
|
||||
/* parseDocument MUST NOT be called — its plain-text output would
|
||||
* be injected into the iframe as HTML. */
|
||||
expect(parseDocumentCalls.length).toBe(0);
|
||||
});
|
||||
|
||||
it('returns null (does not fall back to parseDocument) when DOCX HTML rendering throws', async () => {
|
||||
mockOfficeHtml.mockRejectedValueOnce(new Error('mammoth blew up'));
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('PKfake-docx'),
|
||||
'report.docx',
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
'document',
|
||||
);
|
||||
expect(text).toBeNull();
|
||||
expect(parseDocumentCalls.length).toBe(0);
|
||||
});
|
||||
|
||||
it('XSS regression: failed XLSX HTML render does not ship raw text containing <script>', async () => {
|
||||
/* If an attacker-controlled .xlsx makes mammoth/SheetJS time out
|
||||
* or throw, the previous fallback would call extractDocument and
|
||||
* ship its plain-text output verbatim in `attachment.text`. The
|
||||
* client routes by extension to SPREADSHEET and feeds that text
|
||||
* into `index.html`. A spreadsheet cell containing the literal
|
||||
* string `<script>alert(1)</script>` would then execute inside
|
||||
* the Sandpack iframe. The fix returns null instead, and the
|
||||
* client's empty-text gate keeps the artifact off the panel. */
|
||||
mockOfficeHtml.mockResolvedValueOnce(null);
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('PKfake-xlsx'),
|
||||
'attack.xlsx',
|
||||
'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
|
||||
'document',
|
||||
);
|
||||
expect(text).toBeNull();
|
||||
expect(parseDocumentCalls.length).toBe(0);
|
||||
});
|
||||
|
||||
it('substitutes a "preview too large" banner when HTML exceeds the cache cap', async () => {
|
||||
/* Regression for review finding #2 on PR #12934. The earlier
|
||||
* implementation byte-truncated the producer's HTML at the
|
||||
* UTF-8 boundary, which would land mid-tag and ship malformed
|
||||
* markup like `<table><tr><td>con\n…[truncated]` to the iframe.
|
||||
* The new behavior swaps the entire payload for a small valid
|
||||
* HTML banner — under the cap by construction. */
|
||||
const huge = 'X'.repeat(MAX_TEXT_CACHE_BYTES + 5_000);
|
||||
mockOfficeHtml.mockResolvedValueOnce(huge);
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('PK'),
|
||||
'big.docx',
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
'document',
|
||||
);
|
||||
expect(text).not.toBeNull();
|
||||
// Always under the cap.
|
||||
expect(Buffer.byteLength(text!, 'utf-8')).toBeLessThanOrEqual(MAX_TEXT_CACHE_BYTES);
|
||||
// Valid HTML doc, not byte-truncated markup.
|
||||
expect(text!).toMatch(/^<!DOCTYPE html>/);
|
||||
expect(text!).toContain('</html>');
|
||||
// No truncation marker (which would only appear on the legacy path).
|
||||
expect(text!).not.toContain('…[truncated]');
|
||||
// User-facing banner content.
|
||||
expect(text!).toContain('Preview exceeds the size limit');
|
||||
});
|
||||
|
||||
it('passes through HTML output unchanged when within the cache cap', async () => {
|
||||
const small = '<!DOCTYPE html><html><body><p>hello</p></body></html>';
|
||||
mockOfficeHtml.mockResolvedValueOnce(small);
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('PK'),
|
||||
'small.docx',
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
'document',
|
||||
);
|
||||
expect(text).toBe(small);
|
||||
});
|
||||
|
||||
it('falls back to parseDocument for pdf (HTML producer returns null for unsupported types)', async () => {
|
||||
// Default mock returns null — the producer's own dispatcher would do the
|
||||
// same for PDF since pdf has no HTML rendering. Whether we call it or
|
||||
// skip it is an implementation detail; what matters is that PDF still
|
||||
// routes to parseDocument and yields the docx-mock text.
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('%PDF-1.4'),
|
||||
'doc.pdf',
|
||||
'application/pdf',
|
||||
'document',
|
||||
);
|
||||
expect(text).toBe(docxText);
|
||||
expect(parseDocumentCalls[0]?.originalname).toBe('doc.pdf');
|
||||
});
|
||||
|
||||
it('does not call the office HTML producer for plain .txt utf8-text', async () => {
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('hello world\n', 'utf-8'),
|
||||
'note.txt',
|
||||
'text/plain',
|
||||
'utf8-text',
|
||||
);
|
||||
expect(mockOfficeHtml).not.toHaveBeenCalled();
|
||||
expect(text).toBe('hello world\n');
|
||||
});
|
||||
|
||||
it('routes extensionless CSV-by-MIME (utf8-text category) through the office HTML path', async () => {
|
||||
/* Regression for the Codex review on PR #12934. A tool emitting
|
||||
* `data` with `text/csv` classifies as `utf8-text` (csv has no
|
||||
* extension here, MIME is text/* which the classifier treats as
|
||||
* utf8-text). The previous gate skipped the office-render branch
|
||||
* because the extension wasn't in OFFICE_HTML_EXTENSIONS — so the
|
||||
* raw CSV text shipped to the client, which routes by MIME to the
|
||||
* SPREADSHEET bucket and expects a full HTML document. The fix
|
||||
* shares the dispatcher's MIME-aware predicate so the gate fires
|
||||
* here too. */
|
||||
mockOfficeHtml.mockResolvedValueOnce('<!DOCTYPE html><table><tr><td>1</td></tr></table>');
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('a,b\n1,2', 'utf-8'),
|
||||
'data',
|
||||
'text/csv',
|
||||
'utf8-text',
|
||||
);
|
||||
expect(mockOfficeHtml).toHaveBeenCalledWith(expect.any(Buffer), 'data', 'text/csv');
|
||||
expect(text).toContain('<table>');
|
||||
});
|
||||
|
||||
it.each([
|
||||
['workbook', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'],
|
||||
['workbook', 'application/vnd.ms-excel'],
|
||||
['workbook', 'application/vnd.oasis.opendocument.spreadsheet'],
|
||||
['report', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'],
|
||||
['deck', 'application/vnd.openxmlformats-officedocument.presentationml.presentation'],
|
||||
])('routes extensionless office files by MIME alone (%s, %s)', async (name, mime) => {
|
||||
mockOfficeHtml.mockResolvedValueOnce('<!DOCTYPE html><body>x</body></html>');
|
||||
const category = mime.includes('presentation')
|
||||
? 'pptx'
|
||||
: mime.startsWith('text/')
|
||||
? 'utf8-text'
|
||||
: 'document';
|
||||
const text = await extractCodeArtifactText(Buffer.from('PK'), name, mime, category);
|
||||
expect(mockOfficeHtml).toHaveBeenCalledWith(expect.any(Buffer), name, mime);
|
||||
expect(text).toContain('<body>');
|
||||
});
|
||||
|
||||
it('returns null (and falls back to download UI) when the producer rejects a zip bomb', async () => {
|
||||
/* Defense-in-depth check for SEC review on PR #12934. When
|
||||
* `bufferToOfficeHtml` throws `ZipBombError` (because a zip-bomb
|
||||
* DOCX/XLSX/PPTX got through the compressed-size gate), the outer
|
||||
* extractor must swallow it and return null — that signals to the
|
||||
* code-output controller to register the file as a regular
|
||||
* download instead of a panel artifact. Crucially, it must NOT
|
||||
* fall back to `extractDocument` text either: the client would
|
||||
* inject that text into `index.html` and a literal `<script>`
|
||||
* tag in the document body would execute (Codex P1 review). */
|
||||
const bombError = Object.assign(new Error('zip bomb suspected'), {
|
||||
name: 'ZipBombError',
|
||||
code: 'ZIP_BOMB',
|
||||
});
|
||||
mockOfficeHtml.mockRejectedValueOnce(bombError);
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('PK'),
|
||||
'evil.docx',
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
'document',
|
||||
);
|
||||
expect(text).toBeNull();
|
||||
expect(parseDocumentCalls.length).toBe(0);
|
||||
});
|
||||
|
||||
/* Regression for Codex P2 review on PR #12934. The classifier returns
|
||||
* 'other' for a small but real set of inputs that the new dispatcher
|
||||
* can still route — extensionless `application/csv`, extensionless
|
||||
* office MIMEs with parameters, etc. The early `category === 'other'`
|
||||
* return must NOT short-circuit before `hasOfficeHtmlPath` is checked,
|
||||
* or those inputs silently lose the rich preview. */
|
||||
it('routes extensionless application/csv through office HTML even when category=other', async () => {
|
||||
mockOfficeHtml.mockResolvedValueOnce('<!DOCTYPE html><table><tr><td>1</td></tr></table>');
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('a,b\n1,2', 'utf-8'),
|
||||
'data',
|
||||
'application/csv',
|
||||
'other',
|
||||
);
|
||||
expect(mockOfficeHtml).toHaveBeenCalledWith(expect.any(Buffer), 'data', 'application/csv');
|
||||
expect(text).toContain('<table>');
|
||||
});
|
||||
|
||||
it('routes extensionless office MIME with parameters through office HTML even when category=other', async () => {
|
||||
mockOfficeHtml.mockResolvedValueOnce('<!DOCTYPE html><body>x</body></html>');
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from('PK'),
|
||||
'workbook',
|
||||
'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet; charset=binary',
|
||||
'other',
|
||||
);
|
||||
expect(mockOfficeHtml).toHaveBeenCalled();
|
||||
expect(text).toContain('<body>');
|
||||
});
|
||||
|
||||
it('still returns null for true binary "other" files (no MIME match)', async () => {
|
||||
/* Defense check: the category=other early return is preserved when
|
||||
* `hasOfficeHtmlPath` returns false. A JPEG should not be handed to
|
||||
* the office producer. */
|
||||
const text = await extractCodeArtifactText(
|
||||
Buffer.from([0xff, 0xd8, 0xff]),
|
||||
'photo.jpg',
|
||||
'image/jpeg',
|
||||
'other',
|
||||
);
|
||||
expect(mockOfficeHtml).not.toHaveBeenCalled();
|
||||
expect(text).toBeNull();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
/* `getExtractedTextFormat` is the trust-flag classifier consumed by
|
||||
* `processCodeOutput` (api/server/services/Files/Code/process.js) to
|
||||
* persist `textFormat` on the file record. The client's security gate
|
||||
* in `detectArtifactTypeFromFile` reads that flag to decide whether
|
||||
* routing an office attachment to the HTML preview bucket is safe.
|
||||
* These tests pin the contract: HTML for office paths, 'text' for
|
||||
* everything else, null for missing input. Codex P1 review on PR #12934. */
|
||||
describe('getExtractedTextFormat', () => {
|
||||
it.each([
|
||||
['report.docx', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'],
|
||||
['data.csv', 'text/csv'],
|
||||
['workbook.xlsx', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'],
|
||||
['legacy.xls', 'application/vnd.ms-excel'],
|
||||
['sheet.ods', 'application/vnd.oasis.opendocument.spreadsheet'],
|
||||
['slides.pptx', 'application/vnd.openxmlformats-officedocument.presentationml.presentation'],
|
||||
])('returns "html" for office path %s (%s)', (name, mime) => {
|
||||
expect(getExtractedTextFormat(name, mime, '<table>...</table>')).toBe('html');
|
||||
});
|
||||
|
||||
it.each([
|
||||
['noext', 'application/x-ms-excel', '<table></table>'],
|
||||
['noext', 'application/x-msexcel', '<table></table>'],
|
||||
['noext', 'application/msexcel', '<table></table>'],
|
||||
['noext', 'application/x-excel', '<table></table>'],
|
||||
['noext', 'application/x-dos_ms_excel', '<table></table>'],
|
||||
['noext', 'application/xls', '<table></table>'],
|
||||
['noext', 'application/x-xls', '<table></table>'],
|
||||
])('returns "html" for legacy XLS MIME alias %s (%s)', (name, mime, text) => {
|
||||
/* Mirrors the client's `excelMimeTypes` regex acceptance — keeping
|
||||
* the trust flag aligned across boundaries means no extensionless
|
||||
* XLS upload with a legacy MIME ever lands on the HTML bucket
|
||||
* without `textFormat: 'html'`. */
|
||||
expect(getExtractedTextFormat(name, mime, text)).toBe('html');
|
||||
});
|
||||
|
||||
it('returns "text" for plain UTF-8 outputs (notes, source code, JSON)', () => {
|
||||
expect(getExtractedTextFormat('note.txt', 'text/plain', 'hello world')).toBe('text');
|
||||
expect(getExtractedTextFormat('script.py', 'text/x-python', 'print(1)')).toBe('text');
|
||||
expect(getExtractedTextFormat('data.json', 'application/json', '{"a":1}')).toBe('text');
|
||||
});
|
||||
|
||||
it('returns "text" for parseDocument paths that are NOT office HTML buckets', () => {
|
||||
/* PDF/ODT/HTML go through `parseDocument` and produce plain text;
|
||||
* mark them as such so the client never injects them as HTML. */
|
||||
expect(getExtractedTextFormat('doc.pdf', 'application/pdf', 'extracted text')).toBe('text');
|
||||
expect(
|
||||
getExtractedTextFormat('notes.odt', 'application/vnd.oasis.opendocument.text', 'odt text'),
|
||||
).toBe('text');
|
||||
expect(getExtractedTextFormat('page.html', 'text/html', '<p>raw</p>')).toBe('text');
|
||||
});
|
||||
|
||||
it('returns null when the extractor produced nothing', () => {
|
||||
/* `text == null` (extraction skipped, parser failed, binary unsupported)
|
||||
* → no `textFormat` to persist. The downstream caller short-circuits
|
||||
* the field so the DB doesn't carry a half-truth. */
|
||||
expect(
|
||||
getExtractedTextFormat(
|
||||
'report.docx',
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
null,
|
||||
),
|
||||
).toBeNull();
|
||||
expect(getExtractedTextFormat('photo.jpg', 'image/jpeg', null)).toBeNull();
|
||||
});
|
||||
|
||||
it('returns "html" even when text is empty (the path is what counts, not content length)', () => {
|
||||
/* An office producer that returns "" is still an office path — the
|
||||
* trust flag follows the dispatch decision, not the byte count. The
|
||||
* client's empty-text gate keeps the artifact off the panel anyway. */
|
||||
expect(
|
||||
getExtractedTextFormat(
|
||||
'report.docx',
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
'',
|
||||
),
|
||||
).toBe('html');
|
||||
});
|
||||
|
||||
it('classifies by MIME alone when the filename has no recognized extension', () => {
|
||||
/* extensionless office files (e.g. tool-emitted blobs with a generic
|
||||
* name) still get the right trust flag if the MIME is canonical. */
|
||||
expect(
|
||||
getExtractedTextFormat(
|
||||
'noext',
|
||||
'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
|
||||
'<table></table>',
|
||||
),
|
||||
).toBe('html');
|
||||
});
|
||||
|
||||
it('returns "text" when neither extension nor MIME marks the file as office', () => {
|
||||
expect(getExtractedTextFormat('noext', 'application/octet-stream', 'whatever')).toBe('text');
|
||||
expect(getExtractedTextFormat('', '', 'whatever')).toBe('text');
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -5,15 +5,58 @@ import { randomUUID } from 'crypto';
|
|||
import { logger } from '@librechat/data-schemas';
|
||||
import type { CodeArtifactCategory } from './classify';
|
||||
import { parseDocument } from '~/files/documents/crud';
|
||||
import { bufferToOfficeHtml, officeHtmlBucket } from '~/files/documents/html';
|
||||
import { isBinaryBuffer } from '~/skills/binary';
|
||||
import { withTimeout } from '~/utils/promise';
|
||||
|
||||
export const MAX_TEXT_CACHE_BYTES = 512 * 1024;
|
||||
export const MAX_TEXT_EXTRACT_BYTES = 1024 * 1024;
|
||||
const DOCUMENT_PARSE_TIMEOUT_MS = 8_000;
|
||||
const OFFICE_HTML_TIMEOUT_MS = 12_000;
|
||||
const TRUNCATION_MARKER = '\n\n…[truncated]';
|
||||
const TRUNCATION_MARKER_BYTES = Buffer.byteLength(TRUNCATION_MARKER, 'utf-8');
|
||||
|
||||
/**
|
||||
* Decide whether a buffer is a candidate for rich HTML preview. Wraps the
|
||||
* shared `officeHtmlBucket` predicate from `~/files/documents/html` so the
|
||||
* gate here stays in lock-step with what the dispatcher will actually
|
||||
* route — including extensionless office files identified solely by MIME
|
||||
* (e.g. a tool emitting `data` with `text/csv`, which would otherwise
|
||||
* classify as `utf8-text`, skip the office gate, and ship raw CSV text
|
||||
* to the client's SPREADSHEET bucket that expects full HTML).
|
||||
*/
|
||||
const hasOfficeHtmlPath = (name: string, mimeType: string): boolean =>
|
||||
officeHtmlBucket(name, mimeType) !== null;
|
||||
|
||||
/**
|
||||
* Classify the format of a string returned by `extractCodeArtifactText`
|
||||
* so callers can persist it alongside the text and the client can gate
|
||||
* "inject into iframe as HTML" on a trusted signal. Returns:
|
||||
* - `'html'` when the file went down the office-HTML producer path
|
||||
* (output is a complete sanitized HTML document)
|
||||
* - `'text'` for everything else (utf8 plain text, parseDocument
|
||||
* output for PDF/ODT, etc.)
|
||||
* - `null` when there's no text to format (caller should also skip
|
||||
* setting `textFormat` on the record)
|
||||
*
|
||||
* Inference is by extension/MIME via `officeHtmlBucket`, mirroring the
|
||||
* dispatch logic inside `extractCodeArtifactText` exactly. Keeping the
|
||||
* inference at the caller (rather than baking format into the
|
||||
* extractor's return type) avoids breaking the existing function
|
||||
* signature and its 30+ tests, while still giving downstream consumers
|
||||
* a definitive trust signal.
|
||||
*/
|
||||
export function getExtractedTextFormat(
|
||||
name: string,
|
||||
mimeType: string,
|
||||
text: string | null,
|
||||
): 'html' | 'text' | null {
|
||||
if (text == null) {
|
||||
return null;
|
||||
}
|
||||
return hasOfficeHtmlPath(name, mimeType) ? 'html' : 'text';
|
||||
}
|
||||
|
||||
/**
|
||||
* Truncate UTF-8 content to fit within MAX_TEXT_CACHE_BYTES. Walks back to a
|
||||
* code-point boundary so the cut never lands inside a multi-byte sequence
|
||||
|
|
@ -101,13 +144,77 @@ const extractDocument = async (
|
|||
};
|
||||
|
||||
/**
|
||||
* Extract a UTF-8 text representation of a code-execution artifact for inline
|
||||
* Minimal valid HTML document substituted when a producer's output
|
||||
* exceeds `MAX_TEXT_CACHE_BYTES`. Byte-truncating the producer's HTML
|
||||
* would land mid-tag (e.g. `<table><tr><td>con\n…[truncated]`) and the
|
||||
* Sandpack iframe would render the malformed markup unpredictably —
|
||||
* see review finding #2 on PR #12934. The banner stays under the cap by
|
||||
* construction.
|
||||
*/
|
||||
const OVERSIZED_HTML_BANNER = `<!DOCTYPE html>
|
||||
<html lang="en"><head><meta charset="UTF-8"><title>Preview</title>
|
||||
<style>body{font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,sans-serif;color:#6b7280;padding:16px;font-size:14px;line-height:1.5}@media(prefers-color-scheme:dark){body{color:#9ca3af;background:#1a1a2e}}</style>
|
||||
</head><body>Preview exceeds the size limit. Download the file to view the full contents.</body></html>`;
|
||||
|
||||
/**
|
||||
* Render an office-format buffer as a sanitized HTML preview document. Used
|
||||
* for docx, xlsx/xls/ods, csv, and pptx — produces interactive HTML the
|
||||
* frontend feeds into the Sandpack `static` template via `index.html`.
|
||||
*
|
||||
* **HTML-or-null contract** (security-critical, do NOT relax). Returns
|
||||
* `null` if the file isn't an office type, rendering timed out, or the
|
||||
* parser threw. Callers MUST NOT fall back to plain-text extraction on
|
||||
* null — the client routes office types into `index.html` and would
|
||||
* inject any text-shaped fallback as executable HTML, creating an XSS
|
||||
* vector. The pre-fix versions of this function permitted text fallback
|
||||
* and that path was a real vulnerability; see Codex P1 review on
|
||||
* PR #12934 (commit b06f08a) for the original bug and remediation.
|
||||
*
|
||||
* On oversized output (>`MAX_TEXT_CACHE_BYTES`), returns a small
|
||||
* "preview too large" banner document instead of byte-truncating the
|
||||
* producer's HTML — slicing mid-tag would ship malformed markup to the
|
||||
* iframe.
|
||||
*/
|
||||
const renderOfficeHtml = async (
|
||||
buffer: Buffer,
|
||||
name: string,
|
||||
mimeType: string,
|
||||
): Promise<string | null> => {
|
||||
try {
|
||||
const html = await withTimeout(
|
||||
bufferToOfficeHtml(buffer, name, mimeType),
|
||||
OFFICE_HTML_TIMEOUT_MS,
|
||||
`bufferToOfficeHtml exceeded ${OFFICE_HTML_TIMEOUT_MS}ms`,
|
||||
);
|
||||
if (html == null) {
|
||||
return null;
|
||||
}
|
||||
if (Buffer.byteLength(html, 'utf-8') > MAX_TEXT_CACHE_BYTES) {
|
||||
return OVERSIZED_HTML_BANNER;
|
||||
}
|
||||
return html;
|
||||
} catch (error) {
|
||||
logger.debug(
|
||||
`[renderOfficeHtml] Failed to render "${name}" (${mimeType}): ${(error as Error).message}`,
|
||||
);
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Extract a string representation of a code-execution artifact for inline
|
||||
* rendering. Returns `null` for binary, oversized, or unsupported files; the
|
||||
* caller should fall back to the standard download UI in that case.
|
||||
*
|
||||
* Office types (docx, xlsx/xls/ods, csv, pptx) are rendered as sanitized
|
||||
* HTML by the producers in `~/files/documents/html`. The frontend feeds the
|
||||
* HTML into the Sandpack `static` template via `index.html`. CSV is special-
|
||||
* cased here — its category is `utf8-text` (raw CSV is text), but we want
|
||||
* the styled-table preview when the file extension says CSV.
|
||||
*
|
||||
* - office (docx/xlsx/xls/ods/csv/pptx): sanitized HTML preview
|
||||
* - utf8-text: decodes the buffer (with a binary safety net)
|
||||
* - document: dispatches to the existing PDF/DOCX/XLSX/ODT parser
|
||||
* - pptx: not yet supported in this PR — returns null (follow-up work)
|
||||
* - document: dispatches to the existing PDF/ODT parser
|
||||
* - other: returns null (binary file, no inline preview)
|
||||
*/
|
||||
export async function extractCodeArtifactText(
|
||||
|
|
@ -116,17 +223,51 @@ export async function extractCodeArtifactText(
|
|||
mimeType: string,
|
||||
category: CodeArtifactCategory,
|
||||
): Promise<string | null> {
|
||||
if (category === 'other' || category === 'pptx') {
|
||||
return null;
|
||||
}
|
||||
if (buffer.length > MAX_TEXT_EXTRACT_BYTES) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
/* Office HTML preview is independent of `category` — `officeHtmlBucket`
|
||||
* routes by extension OR (parameter-normalized) MIME, and may route
|
||||
* inputs the legacy classifier labels as 'other'. Concrete cases:
|
||||
* - extensionless `application/csv` (CSV MIMEs aren't in the
|
||||
* classifier's text-MIME set and don't start with `text/`)
|
||||
* - extensionless office MIMEs with parameters like
|
||||
* `application/vnd...spreadsheetml.sheet; charset=binary`
|
||||
* Without checking `hasOfficeHtmlPath` BEFORE the `category === 'other'`
|
||||
* early return, those inputs would silently fall back to download-only
|
||||
* even though the new dispatcher and the client both expect HTML.
|
||||
*
|
||||
* For office types it is **HTML-or-null** with no text fallback.
|
||||
* The client routes these by extension/MIME to the office preview
|
||||
* buckets and feeds `attachment.text` straight into the Sandpack
|
||||
* iframe's `index.html`. Substituting plain text on producer failure
|
||||
* (timeout, malformed file, zip-bomb rejection) would render literal
|
||||
* `<script>` from a DOCX/XLSX/CSV body as executable markup — a
|
||||
* direct XSS vector. Returning null here lets the client's empty-
|
||||
* text gate keep the artifact off the panel and fall back to the
|
||||
* regular download UI, matching what PPTX already does. */
|
||||
if (hasOfficeHtmlPath(name, mimeType)) {
|
||||
const html = await renderOfficeHtml(buffer, name, mimeType);
|
||||
return html;
|
||||
}
|
||||
if (category === 'other') {
|
||||
return null;
|
||||
}
|
||||
if (category === 'utf8-text') {
|
||||
return extractUtf8(buffer);
|
||||
}
|
||||
return await extractDocument(buffer, name, mimeType);
|
||||
if (category === 'document') {
|
||||
/* Reaches here only for non-office documents (PDF, ODT) — neither
|
||||
* is routed to an HTML preview bucket on the client (PDF has no
|
||||
* client routing, ODT routes to PLAIN_TEXT which renders through
|
||||
* the markdown viewer with proper escaping). Plain text is safe. */
|
||||
return await extractDocument(buffer, name, mimeType);
|
||||
}
|
||||
/* category === 'pptx' that didn't go through the office HTML path
|
||||
* (shouldn't happen — pptx ext is in OFFICE_HTML_EXTENSIONS — but
|
||||
* defended in depth). */
|
||||
return null;
|
||||
} catch (error) {
|
||||
logger.debug(
|
||||
`[extractCodeArtifactText] Failed to extract "${name}" (${mimeType}): ${(error as Error).message}`,
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ import yauzl from 'yauzl';
|
|||
import { megabyte, excelMimeTypes, FileSources } from 'librechat-data-provider';
|
||||
import type { TextItem } from 'pdfjs-dist/types/src/display/api';
|
||||
import type { MistralOCRUploadResult } from '~/types';
|
||||
import { assertSafeZipSize } from './zipSafety';
|
||||
|
||||
type FileParseFn = (file: Express.Multer.File) => Promise<string>;
|
||||
|
||||
|
|
@ -93,8 +94,14 @@ async function pdfToText(file: Express.Multer.File): Promise<string> {
|
|||
|
||||
/** Parses Word document, returns text inside. */
|
||||
async function wordDocToText(file: Express.Multer.File): Promise<string> {
|
||||
const buffer = await fs.promises.readFile(file.path);
|
||||
/* Reject zip-bomb DOCX before mammoth's internal extractor runs.
|
||||
* mammoth has no decompressed-size cap of its own; without this, a
|
||||
* sub-1MB compressed bomb (~200x ratio) would block the event loop
|
||||
* and spike RSS to ~1GB. See SEC review on PR #12934. */
|
||||
await assertSafeZipSize(buffer, { name: file.originalname ?? 'docx' });
|
||||
const { extractRawText } = await import('mammoth');
|
||||
const rawText = await extractRawText({ buffer: await fs.promises.readFile(file.path) });
|
||||
const rawText = await extractRawText({ buffer });
|
||||
return rawText.value;
|
||||
}
|
||||
|
||||
|
|
@ -104,6 +111,12 @@ async function excelSheetToText(file: Express.Multer.File): Promise<string> {
|
|||
// readFile() fails with "Cannot access file". read() takes a pre-loaded Buffer instead.
|
||||
const { read, utils } = await import('xlsx');
|
||||
const data = await fs.promises.readFile(file.path);
|
||||
/* Reject zip-bomb XLSX/ODS before SheetJS's internal extractor runs.
|
||||
* `.xls` (BIFF/CFB) is not a ZIP — magic-byte check skips the
|
||||
* validator for it (yauzl would reject it as malformed anyway). */
|
||||
if (data.length >= 4 && data[0] === 0x50 && data[1] === 0x4b) {
|
||||
await assertSafeZipSize(data, { name: file.originalname ?? 'spreadsheet' });
|
||||
}
|
||||
const workbook = read(data, { type: 'buffer' });
|
||||
|
||||
let text = '';
|
||||
|
|
|
|||
975
packages/api/src/files/documents/html.spec.ts
Normal file
975
packages/api/src/files/documents/html.spec.ts
Normal file
|
|
@ -0,0 +1,975 @@
|
|||
import path from 'path';
|
||||
import * as fs from 'fs';
|
||||
import JSZip from 'jszip';
|
||||
import { megabyte } from 'librechat-data-provider';
|
||||
import {
|
||||
_internal,
|
||||
bufferToOfficeHtml,
|
||||
csvToHtml,
|
||||
excelSheetToHtml,
|
||||
officeHtmlBucket,
|
||||
pptxToHtml,
|
||||
pptxToSlideListHtml,
|
||||
sanitizeOfficeHtml,
|
||||
wordDocToHtml,
|
||||
} from './html';
|
||||
import { ZipBombError } from './zipSafety';
|
||||
|
||||
const fixturesDir = __dirname;
|
||||
const readFixture = (name: string): Buffer => fs.readFileSync(path.join(fixturesDir, name));
|
||||
|
||||
describe('Office HTML producers', () => {
|
||||
describe('wordDocToHtml', () => {
|
||||
/* The dispatcher chooses CDN vs mammoth by buffer size. The fixture
|
||||
* sample.docx is small (~6 KB) so it goes down the CDN path. Tests
|
||||
* that need the mammoth output specifically call
|
||||
* `_internal.wordDocToHtmlViaMammoth` directly. */
|
||||
|
||||
test('routes a small docx (≤ cap) through the CDN-rendered path', async () => {
|
||||
const html = await wordDocToHtml(readFixture('sample.docx'));
|
||||
expect(html).toMatch(/^<!DOCTYPE html>/);
|
||||
// CDN wrapper signatures.
|
||||
expect(html).toContain('id="lc-doc-data"');
|
||||
expect(html).toContain('docx.renderAsync');
|
||||
expect(html).toContain('cdn.jsdelivr.net/npm/docx-preview@');
|
||||
/* The CDN doc now ALSO embeds the mammoth-rendered fallback in a
|
||||
* hidden `#lc-fallback` block — Codex P2 review on PR #12934. The
|
||||
* iframe bootstrap reveals it whenever `docx-preview` can't load
|
||||
* (corporate firewall, offline) so air-gapped operators see a
|
||||
* readable preview instead of "Preview unavailable". The
|
||||
* fallback's `<article class="lc-docx">` wrapper is the
|
||||
* server-rendered mammoth output, sanitized through the same
|
||||
* pipeline as the standalone mammoth path. */
|
||||
expect(html).toContain('id="lc-fallback"');
|
||||
expect(html).toContain('<article class="lc-docx">');
|
||||
});
|
||||
|
||||
test('routes a docx above the size cap through the mammoth fallback', async () => {
|
||||
/* Synthesize an oversized buffer by reading the fixture and
|
||||
* verifying the dispatcher picks the mammoth path. We can't
|
||||
* cheaply forge a multi-hundred-KB DOCX that mammoth can still
|
||||
* parse, so this asserts the size predicate by calling
|
||||
* `wordDocToHtmlViaMammoth` directly — the dispatcher's `if`
|
||||
* branch is itself trivial to inspect. */
|
||||
const html = await _internal.wordDocToHtmlViaMammoth(readFixture('sample.docx'));
|
||||
expect(html).toContain('<article class="lc-docx">');
|
||||
expect(html).toContain('This is a sample DOCX file.');
|
||||
});
|
||||
|
||||
test('mammoth fallback strips <script> tags and event handlers', async () => {
|
||||
const html = await _internal.wordDocToHtmlViaMammoth(readFixture('sample.docx'));
|
||||
expect(html).not.toMatch(/<script\b/i);
|
||||
expect(html).not.toMatch(/onerror=/i);
|
||||
});
|
||||
|
||||
test('mammoth fallback emits the docx-specific extra CSS', async () => {
|
||||
const html = await _internal.wordDocToHtmlViaMammoth(readFixture('sample.docx'));
|
||||
expect(html).toContain('.lc-docx table tr:first-child td');
|
||||
expect(html).toContain('.lc-docx p:has(> strong:only-child)');
|
||||
expect(html).toContain('.lc-docx h2');
|
||||
});
|
||||
|
||||
describe('CDN-rendered path', () => {
|
||||
/* These tests lock in the security-relevant structure of the
|
||||
* embedded-binary HTML wrapper. If any of these assertions break
|
||||
* after a refactor, the iframe is one config tweak away from
|
||||
* being a vehicle for outbound exfiltration or supply-chain
|
||||
* compromise. */
|
||||
|
||||
/* `wordDocToHtmlViaCdn` now takes a pre-rendered mammoth body
|
||||
* string as a second argument (Codex P2 review on PR #12934 —
|
||||
* the body is embedded inside `#lc-fallback` for air-gapped
|
||||
* deployments). Tests use a placeholder body to assert pure
|
||||
* wrapper-structure behavior; the dispatcher-level test above
|
||||
* exercises the real mammoth body from the sample fixture. */
|
||||
const FAKE_FALLBACK_BODY = '<p>fallback-body</p>';
|
||||
|
||||
test('embeds the binary as base64 that round-trips to the original bytes', async () => {
|
||||
const original = readFixture('sample.docx');
|
||||
const html = await _internal.wordDocToHtmlViaCdn(original, FAKE_FALLBACK_BODY);
|
||||
const match = html.match(
|
||||
/<script id="lc-doc-data" type="application\/octet-stream;base64">([^<]*)<\/script>/,
|
||||
);
|
||||
expect(match).not.toBeNull();
|
||||
const decoded = Buffer.from(match![1], 'base64');
|
||||
expect(decoded.equals(original)).toBe(true);
|
||||
});
|
||||
|
||||
test('pins both CDN scripts to specific versions with SRI integrity', async () => {
|
||||
const html = await _internal.wordDocToHtmlViaCdn(
|
||||
readFixture('sample.docx'),
|
||||
FAKE_FALLBACK_BODY,
|
||||
);
|
||||
// Both deps loaded from jsdelivr at pinned versions.
|
||||
expect(html).toContain('https://cdn.jsdelivr.net/npm/jszip@3.10.1/');
|
||||
expect(html).toContain('https://cdn.jsdelivr.net/npm/docx-preview@0.3.7/');
|
||||
// Both have SRI integrity attributes.
|
||||
const integrityMatches = html.match(/integrity="sha384-[A-Za-z0-9+/=]+"/g);
|
||||
expect(integrityMatches).not.toBeNull();
|
||||
expect(integrityMatches!.length).toBe(2);
|
||||
// Both have crossorigin="anonymous" (required for SRI on cross-origin).
|
||||
const crossoriginMatches = html.match(/crossorigin="anonymous"/g);
|
||||
expect(crossoriginMatches).not.toBeNull();
|
||||
expect(crossoriginMatches!.length).toBe(2);
|
||||
});
|
||||
|
||||
test('CSP locks the iframe down: no outbound connect, no eval, scripts only from jsdelivr', async () => {
|
||||
const html = await _internal.wordDocToHtmlViaCdn(
|
||||
readFixture('sample.docx'),
|
||||
FAKE_FALLBACK_BODY,
|
||||
);
|
||||
const cspMatch = html.match(
|
||||
/<meta http-equiv="Content-Security-Policy" content="([^"]+)">/,
|
||||
);
|
||||
expect(cspMatch).not.toBeNull();
|
||||
const csp = cspMatch![1];
|
||||
/* `connect-src` is restricted to self + jsdelivr (where the
|
||||
* renderer script came from). Allowing these is necessary for
|
||||
* DevTools sourcemap fetches; broader exfiltration paths
|
||||
* (arbitrary HTTPS, websockets, etc.) stay blocked. Manual
|
||||
* e2e on PR #12934 — strict `'none'` filled DevTools console
|
||||
* with `.min.js.map` violations every time the iframe was
|
||||
* inspected. */
|
||||
expect(csp).toMatch(/connect-src 'self' https:\/\/cdn\.jsdelivr\.net/);
|
||||
expect(csp).not.toMatch(/connect-src[^;]*\*/); // no wildcard
|
||||
// No `<base>` tampering, no form submission either.
|
||||
expect(csp).toMatch(/base-uri 'none'/);
|
||||
expect(csp).toMatch(/form-action 'none'/);
|
||||
// Scripts only from jsdelivr (plus inline for our renderer
|
||||
// bootstrap). No 'unsafe-eval' anywhere.
|
||||
expect(csp).toMatch(/script-src https:\/\/cdn\.jsdelivr\.net 'unsafe-inline'/);
|
||||
expect(csp).not.toMatch(/unsafe-eval/);
|
||||
});
|
||||
|
||||
test('embeds the mammoth-rendered fallback body in #lc-fallback (air-gapped deployments)', async () => {
|
||||
const html = await _internal.wordDocToHtmlViaCdn(
|
||||
readFixture('sample.docx'),
|
||||
FAKE_FALLBACK_BODY,
|
||||
);
|
||||
/* Visible loading state. */
|
||||
expect(html).toContain('Loading preview…');
|
||||
/* The fallback body now contains the server-rendered mammoth
|
||||
* output (the placeholder body in this test). When the iframe
|
||||
* detects `docx-preview` failed to load, `showFallback`
|
||||
* un-hides this block — Codex P2 review on PR #12934. The old
|
||||
* static "Preview unavailable" text is gone in favor of a
|
||||
* notice + the actual document content. */
|
||||
expect(html).toContain('id="lc-fallback"');
|
||||
expect(html).toContain(FAKE_FALLBACK_BODY);
|
||||
expect(html).toContain('High-fidelity renderer unavailable');
|
||||
/* The bootstrap script checks `typeof docx === 'undefined'`
|
||||
* so a CDN outage degrades to the fallback rather than an
|
||||
* empty iframe. */
|
||||
expect(html).toContain("typeof docx === 'undefined'");
|
||||
/* And it hides the empty render slot when fallback shows so
|
||||
* the mammoth content owns the viewport. */
|
||||
expect(html).toContain("document.getElementById('lc-render')");
|
||||
expect(html).toContain('render.hidden = true');
|
||||
});
|
||||
|
||||
test('size-fallback threshold is the documented 350 KB', async () => {
|
||||
/* Lock the public threshold so a future refactor doesn't drift
|
||||
* away from the value referenced in the JSDoc and the
|
||||
* `MAX_TEXT_CACHE_BYTES` reasoning above it. */
|
||||
expect(_internal.MAX_DOCX_CDN_BINARY_BYTES).toBe(350 * 1024);
|
||||
});
|
||||
|
||||
test('output cap mirrors `MAX_TEXT_CACHE_BYTES` from extract.ts', async () => {
|
||||
/* Pin the cycle-avoidance constant. If the upstream
|
||||
* `MAX_TEXT_CACHE_BYTES` ever changes (e.g. lifting the cap
|
||||
* for office types specifically), update both at the same
|
||||
* time or the dispatcher's size-budget path will misfire. */
|
||||
expect(_internal.OFFICE_HTML_OUTPUT_CAP).toBe(512 * 1024);
|
||||
});
|
||||
|
||||
test('output stays within the cache cap for the standard fixture', async () => {
|
||||
/* The fixture isn't large enough to hit the size-budget
|
||||
* fallback, but the resulting HTML *must* fit under the cap so
|
||||
* `attachment.text` doesn't get truncated mid-document.
|
||||
* Pinning this on the standard fixture catches regressions
|
||||
* where wrapper boilerplate or DOCX_EXTRA_CSS grows past the
|
||||
* 512 KB ceiling. Codex P2 review on PR #12934. */
|
||||
const html = await wordDocToHtml(readFixture('sample.docx'));
|
||||
expect(Buffer.byteLength(html, 'utf-8')).toBeLessThanOrEqual(
|
||||
_internal.OFFICE_HTML_OUTPUT_CAP,
|
||||
);
|
||||
});
|
||||
|
||||
test('locks color-scheme to light so OS dark-mode does not fade body text', async () => {
|
||||
/* docx-preview emits page-style rendering (white pages with
|
||||
* the doc's native colors). When the iframe declares both
|
||||
* light and dark color-schemes AND the OS is in dark mode,
|
||||
* browsers inherit dark-mode colors for unset text properties
|
||||
* — docx-preview's body text (no explicit color) inherited
|
||||
* our `--fg: #e5e7eb` on the white page bg, rendering as
|
||||
* barely-visible light grey. Manual e2e regression on PR
|
||||
* #12934. The CDN doc must declare ONLY light scheme; the
|
||||
* mammoth-only fallback (`wrapAsDocument`) is unaffected
|
||||
* because it owns its own bg + text colors. */
|
||||
const html = await _internal.wordDocToHtmlViaCdn(
|
||||
readFixture('sample.docx'),
|
||||
'<p>fallback</p>',
|
||||
);
|
||||
expect(html).toMatch(/color-scheme:\s*light\b/);
|
||||
expect(html).not.toMatch(/color-scheme:\s*light\s+dark/);
|
||||
expect(html).not.toMatch(/prefers-color-scheme:\s*dark/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('OFFICE_PREVIEW_DISABLE_CDN escape hatch', () => {
|
||||
/* Air-gapped / corporate-filtered networks where jsdelivr is
|
||||
* unreachable need a way to force the mammoth path so DOCX
|
||||
* previews don't degrade to "Preview unavailable" on every open.
|
||||
* Codex P2 review on PR #12934. */
|
||||
const ORIGINAL = process.env.OFFICE_PREVIEW_DISABLE_CDN;
|
||||
afterEach(() => {
|
||||
if (ORIGINAL === undefined) {
|
||||
delete process.env.OFFICE_PREVIEW_DISABLE_CDN;
|
||||
} else {
|
||||
process.env.OFFICE_PREVIEW_DISABLE_CDN = ORIGINAL;
|
||||
}
|
||||
});
|
||||
|
||||
test('default behavior (env unset): small docx → CDN path', async () => {
|
||||
delete process.env.OFFICE_PREVIEW_DISABLE_CDN;
|
||||
const html = await wordDocToHtml(readFixture('sample.docx'));
|
||||
expect(html).toContain('id="lc-doc-data"');
|
||||
});
|
||||
|
||||
it.each([['true'], ['1'], ['yes'], ['TRUE'], ['Yes'], [' true ']])(
|
||||
'env=%s forces the mammoth fallback even for small files',
|
||||
async (value) => {
|
||||
process.env.OFFICE_PREVIEW_DISABLE_CDN = value;
|
||||
const html = await wordDocToHtml(readFixture('sample.docx'));
|
||||
// Mammoth-path signature, NOT CDN path.
|
||||
expect(html).toContain('<article class="lc-docx">');
|
||||
expect(html).not.toContain('id="lc-doc-data"');
|
||||
expect(html).not.toContain('cdn.jsdelivr.net');
|
||||
},
|
||||
);
|
||||
|
||||
it.each([['false'], ['0'], ['no'], [''], ['anything-else']])(
|
||||
'env=%j does not disable the CDN path',
|
||||
async (value) => {
|
||||
process.env.OFFICE_PREVIEW_DISABLE_CDN = value;
|
||||
const html = await wordDocToHtml(readFixture('sample.docx'));
|
||||
expect(html).toContain('id="lc-doc-data"');
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('excelSheetToHtml', () => {
|
||||
test('renders all sheets of a multi-sheet workbook into the HTML document', async () => {
|
||||
const html = await excelSheetToHtml(readFixture('sample.xlsx'));
|
||||
expect(html).toMatch(/^<!DOCTYPE html>/);
|
||||
// Each sheet name should appear as a tab label.
|
||||
expect(html).toContain('Sheet One');
|
||||
expect(html).toContain('Second Sheet');
|
||||
// Cell values from both sheets should render.
|
||||
expect(html).toContain('first');
|
||||
expect(html).toContain('Second');
|
||||
// Tab strip uses pure-CSS radio inputs — verify the chrome wired up.
|
||||
expect(html).toContain('lc-sheet-tab-radio');
|
||||
expect(html).toContain('lc-sheet-panel-0');
|
||||
expect(html).toContain('lc-sheet-panel-1');
|
||||
});
|
||||
|
||||
test('omits the tab strip when only one sheet is present', async () => {
|
||||
const html = await excelSheetToHtml(readFixture('sample.xls'));
|
||||
// Cell content should render even though there's no tab strip.
|
||||
expect(html).toContain('first');
|
||||
expect(html).toContain('<table');
|
||||
// Single sheet — no <nav class="lc-sheet-tabs">.
|
||||
expect(html).not.toContain('class="lc-sheet-tabs"');
|
||||
});
|
||||
|
||||
test('renders ods workbooks the same way', async () => {
|
||||
const html = await excelSheetToHtml(readFixture('sample.ods'));
|
||||
expect(html).toContain('Sheet One');
|
||||
expect(html).toContain('Second Sheet');
|
||||
});
|
||||
});
|
||||
|
||||
describe('csvToHtml', () => {
|
||||
test('renders a basic CSV as a single-table HTML document with no tab strip', async () => {
|
||||
const csv = Buffer.from('name,age,city\nAlice,30,NYC\nBob,25,SF\n', 'utf-8');
|
||||
const html = await csvToHtml(csv);
|
||||
expect(html).toMatch(/^<!DOCTYPE html>/);
|
||||
expect(html).toContain('Alice');
|
||||
expect(html).toContain('NYC');
|
||||
expect(html).toContain('<table');
|
||||
expect(html).not.toContain('class="lc-sheet-tabs"');
|
||||
});
|
||||
|
||||
test('handles CSV with embedded commas via quoted fields', async () => {
|
||||
const csv = Buffer.from('label,value\n"hello, world",42\n', 'utf-8');
|
||||
const html = await csvToHtml(csv);
|
||||
expect(html).toContain('hello, world');
|
||||
expect(html).toContain('42');
|
||||
});
|
||||
|
||||
test('handles an empty CSV without crashing', async () => {
|
||||
const html = await csvToHtml(Buffer.from('', 'utf-8'));
|
||||
expect(html).toMatch(/^<!DOCTYPE html>/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('pptxToSlideListHtml', () => {
|
||||
/** Build a minimal valid PPTX with N synthesized slides for testing. */
|
||||
const buildPptx = async (
|
||||
slides: Array<{ title: string; body?: string[] }>,
|
||||
): Promise<Buffer> => {
|
||||
const zip = new JSZip();
|
||||
const slideXml = (title: string, body: string[] = []) => {
|
||||
const titleP = `<a:p><a:r><a:t>${title}</a:t></a:r></a:p>`;
|
||||
const bodyPs = body.map((b) => `<a:p><a:r><a:t>${b}</a:t></a:r></a:p>`).join('');
|
||||
return `<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
|
||||
<p:sld xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main"
|
||||
xmlns:p="http://schemas.openxmlformats.org/presentationml/2006/main">
|
||||
<p:cSld>
|
||||
<p:spTree>
|
||||
<p:sp>
|
||||
<p:txBody>
|
||||
${titleP}
|
||||
${bodyPs}
|
||||
</p:txBody>
|
||||
</p:sp>
|
||||
</p:spTree>
|
||||
</p:cSld>
|
||||
</p:sld>`;
|
||||
};
|
||||
slides.forEach((s, i) => {
|
||||
zip.file(`ppt/slides/slide${i + 1}.xml`, slideXml(s.title, s.body));
|
||||
});
|
||||
// Add a dummy non-slide entry so we exercise the filter.
|
||||
zip.file('docProps/core.xml', '<core/>');
|
||||
return zip.generateAsync({ type: 'nodebuffer' });
|
||||
};
|
||||
|
||||
test('extracts slide titles and body bullets in slide-number order', async () => {
|
||||
const pptx = await buildPptx([
|
||||
{ title: 'Welcome', body: ['First point', 'Second point'] },
|
||||
{ title: 'Agenda', body: ['Item A', 'Item B', 'Item C'] },
|
||||
{ title: 'Thanks!' },
|
||||
]);
|
||||
const html = await pptxToSlideListHtml(pptx);
|
||||
expect(html).toMatch(/^<!DOCTYPE html>/);
|
||||
expect(html).toContain('Slide 1');
|
||||
expect(html).toContain('Welcome');
|
||||
expect(html).toContain('First point');
|
||||
expect(html).toContain('Slide 2');
|
||||
expect(html).toContain('Agenda');
|
||||
expect(html).toContain('Item C');
|
||||
expect(html).toContain('Slide 3');
|
||||
expect(html).toContain('Thanks!');
|
||||
// Title appears before body in the doc.
|
||||
expect(html.indexOf('Welcome')).toBeLessThan(html.indexOf('First point'));
|
||||
});
|
||||
|
||||
test('handles a slide with no extractable text gracefully', async () => {
|
||||
const pptx = await buildPptx([{ title: '' }]);
|
||||
const html = await pptxToSlideListHtml(pptx);
|
||||
expect(html).toContain('Slide 1');
|
||||
expect(html).toContain('(empty slide)');
|
||||
});
|
||||
|
||||
test('returns a friendly empty-state document when no slides are present', async () => {
|
||||
const zip = new JSZip();
|
||||
zip.file('docProps/core.xml', '<core/>');
|
||||
const pptx = await zip.generateAsync({ type: 'nodebuffer' });
|
||||
const html = await pptxToSlideListHtml(pptx);
|
||||
expect(html).toContain('contains no readable slides');
|
||||
});
|
||||
|
||||
test('decodes XML entities in slide text', async () => {
|
||||
const pptx = await buildPptx([{ title: 'A & B', body: ['x < y'] }]);
|
||||
const html = await pptxToSlideListHtml(pptx);
|
||||
expect(html).toContain('A & B'); // re-escaped on output
|
||||
expect(html).toContain('x < y');
|
||||
});
|
||||
});
|
||||
|
||||
describe('pptxToHtml dispatcher', () => {
|
||||
/** Mirror the buildPptx helper from the slide-list block. */
|
||||
const buildPptx = async (
|
||||
slides: Array<{ title: string; body?: string[] }>,
|
||||
): Promise<Buffer> => {
|
||||
const zip = new JSZip();
|
||||
const slideXml = (title: string, body: string[] = []) => {
|
||||
const titleP = `<a:p><a:r><a:t>${title}</a:t></a:r></a:p>`;
|
||||
const bodyPs = body.map((b) => `<a:p><a:r><a:t>${b}</a:t></a:r></a:p>`).join('');
|
||||
return `<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
|
||||
<p:sld xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main"
|
||||
xmlns:p="http://schemas.openxmlformats.org/presentationml/2006/main">
|
||||
<p:cSld><p:spTree><p:sp><p:txBody>${titleP}${bodyPs}</p:txBody></p:sp></p:spTree></p:cSld>
|
||||
</p:sld>`;
|
||||
};
|
||||
slides.forEach((s, i) => {
|
||||
zip.file(`ppt/slides/slide${i + 1}.xml`, slideXml(s.title, s.body));
|
||||
});
|
||||
zip.file('docProps/core.xml', '<core/>');
|
||||
return zip.generateAsync({ type: 'nodebuffer' });
|
||||
};
|
||||
|
||||
test('routes a small pptx (≤ cap) through the CDN-rendered path', async () => {
|
||||
const pptx = await buildPptx([{ title: 'Hello', body: ['First slide'] }]);
|
||||
const html = await pptxToHtml(pptx);
|
||||
expect(html).toMatch(/^<!DOCTYPE html>/);
|
||||
expect(html).toContain('id="lc-doc-data"');
|
||||
expect(html).toContain('pptxPreview.init');
|
||||
expect(html).toContain('cdn.jsdelivr.net/npm/pptx-preview@');
|
||||
/* The slide-list `<ol class="lc-pptx-list">` is now embedded
|
||||
* INSIDE the CDN doc as a runtime fallback for the silent-
|
||||
* empty-render case (pptx-preview's compatibility issues with
|
||||
* pptxgenjs decks). It lives in the hidden `#lc-fallback`
|
||||
* block; the bootstrap reveals it when `hasRenderedContent()`
|
||||
* detects empty wrappers. Manual e2e on PR #12934. */
|
||||
expect(html).toContain('class="lc-pptx-list"');
|
||||
expect(html).toContain('id="lc-fallback"');
|
||||
});
|
||||
|
||||
describe('CDN-rendered path', () => {
|
||||
test('embeds the binary as base64 that round-trips to the original bytes', async () => {
|
||||
const original = await buildPptx([{ title: 'Test' }]);
|
||||
const html = await _internal.pptxToHtmlViaCdn(
|
||||
original,
|
||||
'<ol class="lc-pptx-list"><li>fb</li></ol>',
|
||||
);
|
||||
const match = html.match(
|
||||
/<script id="lc-doc-data" type="application\/octet-stream;base64">([^<]*)<\/script>/,
|
||||
);
|
||||
expect(match).not.toBeNull();
|
||||
const decoded = Buffer.from(match![1], 'base64');
|
||||
expect(decoded.equals(original)).toBe(true);
|
||||
});
|
||||
|
||||
test('pins pptx-preview to a specific version with SRI integrity', async () => {
|
||||
const pptx = await buildPptx([{ title: 'X' }]);
|
||||
const html = await _internal.pptxToHtmlViaCdn(
|
||||
pptx,
|
||||
'<ol class="lc-pptx-list"><li>fb</li></ol>',
|
||||
);
|
||||
expect(html).toContain('https://cdn.jsdelivr.net/npm/pptx-preview@1.0.7/');
|
||||
expect(html).toMatch(/integrity="sha384-[A-Za-z0-9+/=]+"/);
|
||||
expect(html).toContain('crossorigin="anonymous"');
|
||||
});
|
||||
|
||||
test('CSP locks the iframe down: connect-src restricted, no eval, no base/form tampering', async () => {
|
||||
const pptx = await buildPptx([{ title: 'X' }]);
|
||||
const html = await _internal.pptxToHtmlViaCdn(
|
||||
pptx,
|
||||
'<ol class="lc-pptx-list"><li>fb</li></ol>',
|
||||
);
|
||||
const cspMatch = html.match(
|
||||
/<meta http-equiv="Content-Security-Policy" content="([^"]+)">/,
|
||||
);
|
||||
expect(cspMatch).not.toBeNull();
|
||||
const csp = cspMatch![1];
|
||||
/* `connect-src` allows self + jsdelivr (the script's origin)
|
||||
* so DevTools sourcemap fetches succeed and any same-origin
|
||||
* fetches the renderer makes at runtime work. Wildcard
|
||||
* exfiltration paths stay blocked. Manual e2e on PR #12934 —
|
||||
* strict `'none'` filled DevTools console with `.min.js.map`
|
||||
* violations every time the iframe was inspected. */
|
||||
expect(csp).toMatch(/connect-src 'self' https:\/\/cdn\.jsdelivr\.net/);
|
||||
expect(csp).not.toMatch(/connect-src[^;]*\*/); // no wildcard
|
||||
expect(csp).toMatch(/base-uri 'none'/);
|
||||
expect(csp).toMatch(/form-action 'none'/);
|
||||
expect(csp).toMatch(/script-src https:\/\/cdn\.jsdelivr\.net 'unsafe-inline'/);
|
||||
expect(csp).not.toMatch(/unsafe-eval/);
|
||||
/* PPTX must allow blob:-only Web Workers — pptx-preview's
|
||||
* bundled echarts dep spins up workers via blob: URLs for
|
||||
* chart rendering. Without this, the renderer's async
|
||||
* pipeline throws unhandled rejections and the iframe shows
|
||||
* a black screen. */
|
||||
expect(csp).toMatch(/worker-src blob:/);
|
||||
});
|
||||
|
||||
test('exposes a fallback message that surfaces if the renderer fails to load or times out', async () => {
|
||||
const pptx = await buildPptx([{ title: 'X' }]);
|
||||
const html = await _internal.pptxToHtmlViaCdn(
|
||||
pptx,
|
||||
'<ol class="lc-pptx-list"><li>fb</li></ol>',
|
||||
);
|
||||
// Visible loading state + fallback that swaps in on error.
|
||||
expect(html).toContain('Loading preview…');
|
||||
/* Fallback now embeds the slide-list view + a notice
|
||||
* explaining the high-fidelity renderer failed. The old
|
||||
* static "Preview unavailable" text is replaced with the
|
||||
* actual readable content. Manual e2e on PR #12934 (pptxgenjs
|
||||
* compatibility issue with pptx-preview). */
|
||||
expect(html).toContain('High-fidelity renderer unavailable');
|
||||
expect(html).toContain('class="lc-pptx-list"');
|
||||
// The renderer-not-loaded check.
|
||||
expect(html).toContain("typeof pptxPreview === 'undefined'");
|
||||
// The unhandledrejection + error listeners — pptx-preview's
|
||||
// bundled deps raise async rejections that don't surface
|
||||
// through the outer Promise.
|
||||
expect(html).toContain("addEventListener('unhandledrejection'");
|
||||
expect(html).toContain("addEventListener('error'");
|
||||
// The 8-second timeout safety net for silent renderer failures.
|
||||
expect(html).toContain('renderer-timeout');
|
||||
expect(html).toContain('renderer-empty-slide-list');
|
||||
expect(html).toContain("querySelectorAll('.pptx-preview-slide-wrapper')");
|
||||
/* Diagnostic surfacing: the failure reason must be exposed
|
||||
* three ways so a debugging user can find it without reading
|
||||
* source — manual e2e on PR #12934 ("'Preview unavailable'
|
||||
* with no way to know why"):
|
||||
* 1. Visible inline via a `<details>` element so it's one
|
||||
* click away in the iframe itself
|
||||
* 2. `title` attribute on the fallback for hover tooltip
|
||||
* 3. `console.error` so DevTools shows it in red */
|
||||
expect(html).toContain('id="lc-fallback-reason"');
|
||||
expect(html).toContain('Diagnostic details');
|
||||
expect(html).toContain('fallback.title = reasonText');
|
||||
expect(html).toContain("console.error('[pptx-preview] fallback fired:'");
|
||||
});
|
||||
|
||||
test('size-fallback threshold is the documented 350 KB', () => {
|
||||
expect(_internal.MAX_PPTX_CDN_BINARY_BYTES).toBe(350 * 1024);
|
||||
});
|
||||
|
||||
test('embeds the slide-list fallback in the CDN doc with empty-render detection', async () => {
|
||||
/* pptx-preview silently produces empty 960×540 wrappers for
|
||||
* pptxgenjs-generated decks (manual e2e on PR #12934 — it
|
||||
* parses the file enough to create the placeholder, then
|
||||
* fails to populate it). The bootstrap now detects the
|
||||
* empty-render case via `hasRenderedContent()` and reveals
|
||||
* the embedded slide-list fallback so the user always gets
|
||||
* readable content. */
|
||||
const pptx = await buildPptx([{ title: 'A' }, { title: 'B' }]);
|
||||
const html = await _internal.pptxToHtmlViaCdn(
|
||||
pptx,
|
||||
'<ol class="lc-pptx-list"><li>SENTINEL_FALLBACK</li></ol>',
|
||||
);
|
||||
/* Slide-list fallback body is embedded inside `#lc-fallback`. */
|
||||
expect(html).toContain('id="lc-fallback"');
|
||||
expect(html).toContain('SENTINEL_FALLBACK');
|
||||
/* Empty-render detection helper exists in the bootstrap. */
|
||||
expect(html).toContain('hasRenderedContent');
|
||||
expect(html).toContain('renderer-empty-slide-list');
|
||||
expect(html).toContain("querySelectorAll('.pptx-preview-slide-wrapper')");
|
||||
/* The slide-list fallback CSS is inlined so the fallback
|
||||
* renders with the same look as the standalone slide-list
|
||||
* path (CSP locks `style-src` to inline only). */
|
||||
expect(html).toContain('.lc-pptx-list');
|
||||
expect(html).toContain('.lc-pptx-slide');
|
||||
});
|
||||
|
||||
test('bootstrap wraps + scales each slide so it fits the iframe width', async () => {
|
||||
/* pptx-preview emits slides at the init dimensions (960×540
|
||||
* by default). Without post-processing, narrow artifact panels
|
||||
* scroll horizontally and slides spill outside the viewport.
|
||||
* The bootstrap wraps each rendered slide in `.lc-slide-wrap`
|
||||
* and applies `transform: scale(panel_width / 960)` so the
|
||||
* panel always fits — manual e2e feedback on PR #12934.
|
||||
*
|
||||
* The wrap is applied ONCE, after `previewer.preview` resolves
|
||||
* (and the post-render container is visible). Wrapping during
|
||||
* streaming via MutationObserver caused pptx-preview to throw
|
||||
* — its internal pipeline holds references to the appended
|
||||
* slides and broke when we moved them under a parent wrap. */
|
||||
const pptx = await buildPptx([{ title: 'A' }, { title: 'B' }]);
|
||||
const html = await _internal.pptxToHtmlViaCdn(
|
||||
pptx,
|
||||
'<ol class="lc-pptx-list"><li>fb</li></ol>',
|
||||
);
|
||||
/* The wrapper class used by the CSS rules. */
|
||||
expect(html).toContain('lc-slide-wrap');
|
||||
/* The wrap function + the per-slide scale function. The
|
||||
* scale uses each slide's actual rendered native width (not
|
||||
* a constant) so panels wider than 960px still fill — no
|
||||
* upscale cap means we never leave whitespace on the sides
|
||||
* of the panel. */
|
||||
expect(html).toContain('wrapSlides');
|
||||
expect(html).toContain('scaleFor');
|
||||
expect(html).toContain('availableWidth');
|
||||
/* The width-only scale formula. Negative assertion below
|
||||
* forbids re-introducing the height-aware variant — that
|
||||
* caused pptx-preview to render slides as solid-black
|
||||
* rectangles (manual e2e regression on PR #12934). */
|
||||
expect(html).toContain('availableWidth() / (nativeW || SLIDE_W)');
|
||||
/* Negative assertions for two failed iterations:
|
||||
* - `Math.min(1, ...)` upscale cap left whitespace on
|
||||
* panels wider than 960px.
|
||||
* - `Math.min(sw, sh)` height-aware scale + viewport-
|
||||
* derived height calculations interacted with pptx-
|
||||
* preview's internal layout and produced black slides.
|
||||
* - `body { display: flex }` and `min-height: 100vh` on
|
||||
* html/body also broke pptx-preview's render path.
|
||||
* Vertical empty space below short decks is acceptable; the
|
||||
* iframe bg shows through and matches the panel theme. */
|
||||
expect(html).not.toMatch(/Math\.min\(\s*1\s*,/);
|
||||
expect(html).not.toContain('Math.min(sw, sh)');
|
||||
expect(html).not.toContain('availableHeight');
|
||||
expect(html).not.toContain('min-height: 100vh');
|
||||
expect(html).not.toMatch(/body\s*\{[^}]*display:\s*flex/);
|
||||
/* Container is hidden during render and revealed by the
|
||||
* `finalize` step so the unscaled flash never reaches the
|
||||
* user. */
|
||||
expect(html).toContain("container.style.visibility = 'hidden'");
|
||||
expect(html).toContain("container.style.visibility = 'visible'");
|
||||
/* ResizeObserver re-fits on panel resize. (No
|
||||
* MutationObserver — streaming wraps broke pptx-preview.) */
|
||||
expect(html).toContain('ResizeObserver');
|
||||
expect(html).not.toContain('MutationObserver');
|
||||
/* Native dimensions cached on the slide dataset so re-fitting
|
||||
* on resize never measures an already-transformed box. */
|
||||
expect(html).toContain('lcNativeW');
|
||||
});
|
||||
});
|
||||
|
||||
describe('OFFICE_PREVIEW_DISABLE_CDN escape hatch', () => {
|
||||
const ORIGINAL = process.env.OFFICE_PREVIEW_DISABLE_CDN;
|
||||
afterEach(() => {
|
||||
if (ORIGINAL === undefined) {
|
||||
delete process.env.OFFICE_PREVIEW_DISABLE_CDN;
|
||||
} else {
|
||||
process.env.OFFICE_PREVIEW_DISABLE_CDN = ORIGINAL;
|
||||
}
|
||||
});
|
||||
|
||||
test('default behavior (env unset): small pptx → CDN path', async () => {
|
||||
delete process.env.OFFICE_PREVIEW_DISABLE_CDN;
|
||||
const pptx = await buildPptx([{ title: 'Hello' }]);
|
||||
const html = await pptxToHtml(pptx);
|
||||
expect(html).toContain('id="lc-doc-data"');
|
||||
});
|
||||
|
||||
it.each([['true'], ['1'], ['yes'], ['TRUE']])(
|
||||
'env=%s forces the slide-list fallback even for small files',
|
||||
async (value) => {
|
||||
process.env.OFFICE_PREVIEW_DISABLE_CDN = value;
|
||||
const pptx = await buildPptx([{ title: 'Hello', body: ['Line 1'] }]);
|
||||
const html = await pptxToHtml(pptx);
|
||||
// Slide-list signature.
|
||||
expect(html).toContain('class="lc-pptx-list"');
|
||||
expect(html).toContain('Hello');
|
||||
expect(html).not.toContain('id="lc-doc-data"');
|
||||
expect(html).not.toContain('cdn.jsdelivr.net');
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('bufferToOfficeHtml dispatcher', () => {
|
||||
test('routes by extension when MIME is generic', async () => {
|
||||
const html = await bufferToOfficeHtml(
|
||||
readFixture('sample.docx'),
|
||||
'sample.docx',
|
||||
'application/octet-stream',
|
||||
);
|
||||
expect(html).not.toBeNull();
|
||||
// sample.docx is small → CDN path. Lock the dispatcher routing
|
||||
// by checking for a CDN-path signature rather than the literal
|
||||
// document text (which only appears in the mammoth fallback).
|
||||
expect(html!).toContain('id="lc-doc-data"');
|
||||
expect(html!).toContain('docx-preview@');
|
||||
});
|
||||
|
||||
test('routes by MIME when extension is missing', async () => {
|
||||
const html = await bufferToOfficeHtml(
|
||||
readFixture('sample.xlsx'),
|
||||
'workbook',
|
||||
'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
|
||||
);
|
||||
expect(html).not.toBeNull();
|
||||
expect(html!).toContain('Sheet One');
|
||||
});
|
||||
|
||||
test('routes csv by extension', async () => {
|
||||
const html = await bufferToOfficeHtml(
|
||||
Buffer.from('a,b\n1,2', 'utf-8'),
|
||||
'data.csv',
|
||||
'application/octet-stream',
|
||||
);
|
||||
expect(html).not.toBeNull();
|
||||
expect(html!).toContain('<table');
|
||||
});
|
||||
|
||||
test('routes csv by MIME when extension is missing', async () => {
|
||||
const html = await bufferToOfficeHtml(Buffer.from('a,b\n1,2', 'utf-8'), 'data', 'text/csv');
|
||||
expect(html).not.toBeNull();
|
||||
expect(html!).toContain('<table');
|
||||
});
|
||||
|
||||
test('returns null for unrecognized types', async () => {
|
||||
const html = await bufferToOfficeHtml(Buffer.from('hello'), 'notes.txt', 'text/plain');
|
||||
expect(html).toBeNull();
|
||||
});
|
||||
|
||||
test('extension wins over MIME (sniff misclassifies docx as application/zip)', async () => {
|
||||
const html = await bufferToOfficeHtml(
|
||||
readFixture('sample.docx'),
|
||||
'sample.docx',
|
||||
'application/zip',
|
||||
);
|
||||
expect(html).not.toBeNull();
|
||||
expect(html!).toContain('lc-docx');
|
||||
});
|
||||
});
|
||||
|
||||
describe('officeHtmlBucket predicate', () => {
|
||||
/* The shared predicate is the single source of truth for "should the
|
||||
* office HTML pipeline handle this file?". The upstream gate in
|
||||
* `extract.ts` calls it directly; the dispatcher above delegates to
|
||||
* it. Tests here lock in MIME-only routing for extensionless inputs
|
||||
* (the case Codex flagged on PR #12934). */
|
||||
it.each([
|
||||
['report.docx', 'application/zip', 'docx'],
|
||||
['report.docx', '', 'docx'],
|
||||
['noext', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', 'docx'],
|
||||
['data.csv', 'application/octet-stream', 'csv'],
|
||||
['data', 'text/csv', 'csv'],
|
||||
['data', 'application/csv', 'csv'],
|
||||
['workbook.xlsx', '', 'spreadsheet'],
|
||||
['legacy.xls', '', 'spreadsheet'],
|
||||
['sheet.ods', '', 'spreadsheet'],
|
||||
['noext', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'spreadsheet'],
|
||||
['noext', 'application/vnd.ms-excel', 'spreadsheet'],
|
||||
['noext', 'application/vnd.oasis.opendocument.spreadsheet', 'spreadsheet'],
|
||||
['deck.pptx', '', 'pptx'],
|
||||
[
|
||||
'noext',
|
||||
'application/vnd.openxmlformats-officedocument.presentationml.presentation',
|
||||
'pptx',
|
||||
],
|
||||
])('classifies (%s, %s) as %s', (name, mime, expected) => {
|
||||
expect(officeHtmlBucket(name, mime)).toBe(expected);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['note.txt', 'text/plain'],
|
||||
['code.py', 'text/x-python'],
|
||||
['data.json', 'application/json'],
|
||||
['photo.jpg', 'image/jpeg'],
|
||||
['doc.pdf', 'application/pdf'],
|
||||
['archive.zip', 'application/zip'],
|
||||
['notes.odt', 'application/vnd.oasis.opendocument.text'],
|
||||
['noext', 'text/plain'],
|
||||
['noext', ''],
|
||||
])('returns null for non-office (%s, %s)', (name, mime) => {
|
||||
expect(officeHtmlBucket(name, mime)).toBeNull();
|
||||
});
|
||||
|
||||
/* Regression for Codex P2 review on PR #12934. A binary office file
|
||||
* with a mismatched MIME (e.g. a tool sandbox sets `text/csv` on
|
||||
* everything it ships) must NOT be re-routed to a different bucket
|
||||
* just because the MIME matches a different bucket's pattern. The
|
||||
* documented "extension wins" precedence is enforced by checking
|
||||
* extensions exhaustively before any MIME pattern fires. */
|
||||
it.each([
|
||||
['deck.pptx', 'text/csv', 'pptx'],
|
||||
['workbook.xlsx', 'text/csv', 'spreadsheet'],
|
||||
[
|
||||
'legacy.xls',
|
||||
'application/vnd.openxmlformats-officedocument.presentationml.presentation',
|
||||
'spreadsheet',
|
||||
],
|
||||
['sheet.ods', 'text/csv', 'spreadsheet'],
|
||||
['report.docx', 'text/csv', 'docx'],
|
||||
['report.docx', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'docx'],
|
||||
[
|
||||
'data.csv',
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
'csv',
|
||||
],
|
||||
])('extension wins over conflicting MIME: (%s, %s) → %s', (name, mime, expected) => {
|
||||
expect(officeHtmlBucket(name, mime)).toBe(expected);
|
||||
});
|
||||
|
||||
/* Regression for Codex P2 (server/client routing parity). The
|
||||
* client's `detectArtifactTypeFromFile` routes by extension first
|
||||
* for ANY known extension (not just office) — `.txt` →
|
||||
* PLAIN_TEXT regardless of MIME. If the server fell back to MIME
|
||||
* for these files and produced office HTML, the client would
|
||||
* still route to PLAIN_TEXT, escape the HTML through the markdown
|
||||
* viewer, and the user would see raw `<html>...` markup. The
|
||||
* server now refuses to fall back to MIME when extension is
|
||||
* non-empty — symmetric "extension wins" precedence with the
|
||||
* client.
|
||||
*
|
||||
* Trade-off: a true DOCX renamed to `myfile.bin` + DOCX MIME no
|
||||
* longer routes through office HTML on the server. Acceptable —
|
||||
* extension/MIME mismatch is user error, and the security gate
|
||||
* on the client would have produced a PLAIN_TEXT fall-through
|
||||
* anyway (no preview either way). */
|
||||
it.each([
|
||||
// Known non-office extension + office MIME — must NOT route to office
|
||||
['notes.txt', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'],
|
||||
['notes.md', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'],
|
||||
['data.json', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'],
|
||||
['code.py', 'application/vnd.openxmlformats-officedocument.presentationml.presentation'],
|
||||
['page.html', 'text/csv'],
|
||||
['styles.css', 'text/csv'],
|
||||
// Unknown extension + office MIME — also returns null (server is
|
||||
// conservative; client's security gate will downgrade these to
|
||||
// PLAIN_TEXT anyway since `textFormat` would not be 'html')
|
||||
['blob.bin', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'],
|
||||
['data.dat', 'text/csv'],
|
||||
])('non-empty non-office extension wins over office MIME: (%s, %s) → null', (name, mime) => {
|
||||
expect(officeHtmlBucket(name, mime)).toBeNull();
|
||||
});
|
||||
|
||||
/* Regression for Codex P2 review on PR #12934. Real Content-Type
|
||||
* headers carry parameters like `; charset=utf-8` and `; boundary`;
|
||||
* the predicate must strip them before matching, otherwise the
|
||||
* backend silently falls through to raw text while the client's
|
||||
* `baseMime` strips the same parameters and routes the file to the
|
||||
* spreadsheet bucket — yielding a broken preview. */
|
||||
it.each([
|
||||
['data', 'text/csv; charset=utf-8', 'csv'],
|
||||
['data', 'text/csv;charset=utf-8', 'csv'],
|
||||
['data', 'TEXT/CSV; CHARSET=UTF-8', 'csv'],
|
||||
['data', 'application/csv; charset=ascii', 'csv'],
|
||||
['data', 'text/comma-separated-values', 'csv'],
|
||||
['data', 'text/comma-separated-values; charset=utf-8', 'csv'],
|
||||
[
|
||||
'workbook',
|
||||
'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet; charset=binary',
|
||||
'spreadsheet',
|
||||
],
|
||||
[
|
||||
'report',
|
||||
'application/vnd.openxmlformats-officedocument.wordprocessingml.document; charset=binary',
|
||||
'docx',
|
||||
],
|
||||
[
|
||||
'deck',
|
||||
'application/vnd.openxmlformats-officedocument.presentationml.presentation; foo=bar',
|
||||
'pptx',
|
||||
],
|
||||
])('strips MIME parameters before matching: (%s, %s) → %s', (name, mime, expected) => {
|
||||
expect(officeHtmlBucket(name, mime)).toBe(expected);
|
||||
});
|
||||
});
|
||||
|
||||
describe('zip-bomb defense (SEC review on PR #12934)', () => {
|
||||
/* Build a sub-1MB compressed ZIP that inflates to >25MB (default
|
||||
* per-entry cap). Mirrors the SEC validation PoC: highly compressed
|
||||
* runs of zero bytes squeeze through any compressed-size gate but
|
||||
* blow up the parser. */
|
||||
const buildBombArchive = async (
|
||||
entries: Array<{ name: string; decompressedBytes: number }>,
|
||||
): Promise<Buffer> => {
|
||||
const zip = new JSZip();
|
||||
for (const { name, decompressedBytes } of entries) {
|
||||
zip.file(name, Buffer.alloc(decompressedBytes, 0));
|
||||
}
|
||||
return zip.generateAsync({
|
||||
type: 'nodebuffer',
|
||||
compression: 'DEFLATE',
|
||||
compressionOptions: { level: 9 },
|
||||
});
|
||||
};
|
||||
|
||||
test('wordDocToHtml rejects a zip-bomb DOCX before mammoth touches it', async () => {
|
||||
const bomb = await buildBombArchive([
|
||||
{ name: 'word/document.xml', decompressedBytes: 50 * megabyte },
|
||||
]);
|
||||
expect(bomb.length).toBeLessThan(1 * megabyte);
|
||||
await expect(wordDocToHtml(bomb)).rejects.toThrow(ZipBombError);
|
||||
});
|
||||
|
||||
test('excelSheetToHtml rejects a zip-bomb XLSX before SheetJS touches it', async () => {
|
||||
const bomb = await buildBombArchive([
|
||||
{ name: 'xl/worksheets/sheet1.xml', decompressedBytes: 50 * megabyte },
|
||||
]);
|
||||
expect(bomb.length).toBeLessThan(1 * megabyte);
|
||||
await expect(excelSheetToHtml(bomb)).rejects.toThrow(ZipBombError);
|
||||
});
|
||||
|
||||
test('pptxToSlideListHtml rejects a zip-bomb PPTX before slide extraction', async () => {
|
||||
const bomb = await buildBombArchive([
|
||||
{ name: 'ppt/slides/slide1.xml', decompressedBytes: 50 * megabyte },
|
||||
]);
|
||||
expect(bomb.length).toBeLessThan(1 * megabyte);
|
||||
await expect(pptxToSlideListHtml(bomb)).rejects.toThrow(ZipBombError);
|
||||
});
|
||||
|
||||
test('bufferToOfficeHtml propagates ZipBombError so callers can fail safe', async () => {
|
||||
const bomb = await buildBombArchive([
|
||||
{ name: 'word/document.xml', decompressedBytes: 50 * megabyte },
|
||||
]);
|
||||
await expect(bufferToOfficeHtml(bomb, 'evil.docx', '')).rejects.toThrow(ZipBombError);
|
||||
});
|
||||
|
||||
test('legitimate small office files are not impacted by the safety check', async () => {
|
||||
/* Real DOCX fixture from the test fixtures directory should still
|
||||
* render — paranoid validation that the safety check doesn't false-
|
||||
* positive on tiny legitimate inputs. Small fixture takes the CDN
|
||||
* path, so we assert by wrapper structure instead of doc text. */
|
||||
const fixture = readFixture('sample.docx');
|
||||
const html = await wordDocToHtml(fixture);
|
||||
expect(html).toMatch(/^<!DOCTYPE html>/);
|
||||
expect(html).toContain('id="lc-doc-data"');
|
||||
});
|
||||
});
|
||||
|
||||
describe('sanitizeOfficeHtml security', () => {
|
||||
test('strips <script> tags entirely', async () => {
|
||||
const out = await sanitizeOfficeHtml('<p>before<script>alert(1)</script>after</p>');
|
||||
expect(out).not.toMatch(/<script\b/i);
|
||||
expect(out).not.toContain('alert(1)');
|
||||
expect(out).toContain('before');
|
||||
expect(out).toContain('after');
|
||||
});
|
||||
|
||||
test('drops event-handler attributes from surviving tags', async () => {
|
||||
const out = await sanitizeOfficeHtml('<img src="https://x.test/a.png" onerror="alert(1)">');
|
||||
expect(out).not.toMatch(/onerror=/i);
|
||||
expect(out).toContain('https://x.test/a.png');
|
||||
});
|
||||
|
||||
test('strips javascript: URLs from anchors', async () => {
|
||||
const out = await sanitizeOfficeHtml('<a href="javascript:alert(1)">click</a>');
|
||||
expect(out).not.toMatch(/javascript:/i);
|
||||
// The text survives even when the href is dropped.
|
||||
expect(out).toContain('click');
|
||||
});
|
||||
|
||||
test('rejects data: URLs in <a href> (only <img src> may use data:)', async () => {
|
||||
/* The Sandpack iframe sandbox does NOT gate `target="_blank"`
|
||||
* navigations. A surviving `<a href="data:text/html,...">` would
|
||||
* open attacker-controlled HTML in a new tab on click. The
|
||||
* sanitizer scopes `data:` to <img> only — global schemes are
|
||||
* http/https/mailto. Regression guard for the Codex review on
|
||||
* PR #12934. */
|
||||
const out = await sanitizeOfficeHtml(
|
||||
'<a href="data:text/html,<script>alert(1)</script>">click</a>',
|
||||
);
|
||||
expect(out).not.toContain('data:');
|
||||
expect(out).not.toContain('script');
|
||||
expect(out).toContain('click');
|
||||
});
|
||||
|
||||
test('preserves data: URLs in <img src> (mammoth inlines DOCX images as base64)', async () => {
|
||||
const tinyPng =
|
||||
'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=';
|
||||
const out = await sanitizeOfficeHtml(`<img src="${tinyPng}" alt="dot">`);
|
||||
expect(out).toContain(tinyPng);
|
||||
expect(out).toContain('alt="dot"');
|
||||
});
|
||||
|
||||
test('preserves http(s) and mailto links on anchors', async () => {
|
||||
const out = await sanitizeOfficeHtml(
|
||||
'<a href="https://example.com">site</a> <a href="mailto:a@b.test">mail</a>',
|
||||
);
|
||||
expect(out).toContain('https://example.com');
|
||||
expect(out).toContain('mailto:a@b.test');
|
||||
});
|
||||
|
||||
test('forces target=_blank rel=noopener on surviving anchors', async () => {
|
||||
const out = await sanitizeOfficeHtml('<a href="https://example.com">link</a>');
|
||||
expect(out).toContain('target="_blank"');
|
||||
expect(out).toContain('rel="noopener noreferrer"');
|
||||
});
|
||||
|
||||
test('strips <iframe> entirely', async () => {
|
||||
const out = await sanitizeOfficeHtml('<p>x</p><iframe src="https://evil.test"></iframe>');
|
||||
expect(out).not.toMatch(/<iframe\b/i);
|
||||
expect(out).not.toContain('evil.test');
|
||||
});
|
||||
});
|
||||
});
|
||||
1633
packages/api/src/files/documents/html.ts
Normal file
1633
packages/api/src/files/documents/html.ts
Normal file
File diff suppressed because it is too large
Load diff
385
packages/api/src/files/documents/libreoffice.spec.ts
Normal file
385
packages/api/src/files/documents/libreoffice.spec.ts
Normal file
|
|
@ -0,0 +1,385 @@
|
|||
import { spawnSync } from 'child_process';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import {
|
||||
_resetLibreOfficeProbeCache,
|
||||
buildPdfEmbedDocument,
|
||||
convertOfficeToPdf,
|
||||
isLibreOfficeEnabled,
|
||||
isLibreOfficeEnabledFor,
|
||||
LibreOfficeConversionError,
|
||||
LibreOfficeUnavailableError,
|
||||
LIBREOFFICE_TIMEOUT_MS,
|
||||
MAX_LIBREOFFICE_PDF_BYTES,
|
||||
probeLibreOfficeBinary,
|
||||
tryLibreOfficePreview,
|
||||
} from './libreoffice';
|
||||
|
||||
/* Detect whether the host has a LibreOffice binary so the integration
|
||||
* tests below can conditionally engage. Most CI runners don't have
|
||||
* LibreOffice installed (it's a 500 MB dependency); we still exercise
|
||||
* the env-gating, the wrapper builder, and the failure-fallthrough
|
||||
* contract in those environments. */
|
||||
function hasLibreOfficeOnPath(): boolean {
|
||||
for (const candidate of ['soffice', 'libreoffice']) {
|
||||
const result = spawnSync(candidate, ['--version'], { stdio: 'ignore' });
|
||||
if (result.status === 0) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
const LIBREOFFICE_INSTALLED = hasLibreOfficeOnPath();
|
||||
const itIfLibreOffice = LIBREOFFICE_INSTALLED ? it : it.skip;
|
||||
|
||||
/* Fixtures sit next to the source files (matches `html.spec.ts` resolution). */
|
||||
const FIXTURES_DIR = __dirname;
|
||||
function readFixture(name: string): Buffer {
|
||||
return fs.readFileSync(path.join(FIXTURES_DIR, name));
|
||||
}
|
||||
|
||||
describe('libreoffice (env gating + wrapper)', () => {
|
||||
const ORIGINAL_FLAG = process.env.OFFICE_PREVIEW_LIBREOFFICE;
|
||||
|
||||
afterEach(() => {
|
||||
_resetLibreOfficeProbeCache();
|
||||
if (ORIGINAL_FLAG === undefined) {
|
||||
delete process.env.OFFICE_PREVIEW_LIBREOFFICE;
|
||||
} else {
|
||||
process.env.OFFICE_PREVIEW_LIBREOFFICE = ORIGINAL_FLAG;
|
||||
}
|
||||
});
|
||||
|
||||
describe('isLibreOfficeEnabled (any-format check)', () => {
|
||||
it.each([['true'], ['1'], ['yes'], ['TRUE'], ['Yes'], [' true ']])(
|
||||
'returns true for %j (truthy = all formats enabled)',
|
||||
(value) => {
|
||||
process.env.OFFICE_PREVIEW_LIBREOFFICE = value;
|
||||
expect(isLibreOfficeEnabled()).toBe(true);
|
||||
},
|
||||
);
|
||||
|
||||
it.each([['false'], ['0'], ['no'], ['']])('returns false for %j', (value) => {
|
||||
process.env.OFFICE_PREVIEW_LIBREOFFICE = value;
|
||||
expect(isLibreOfficeEnabled()).toBe(false);
|
||||
});
|
||||
|
||||
it('returns false when the env var is unset', () => {
|
||||
delete process.env.OFFICE_PREVIEW_LIBREOFFICE;
|
||||
expect(isLibreOfficeEnabled()).toBe(false);
|
||||
});
|
||||
|
||||
it('returns true for a comma-separated format list (any format counts)', () => {
|
||||
process.env.OFFICE_PREVIEW_LIBREOFFICE = 'pptx';
|
||||
expect(isLibreOfficeEnabled()).toBe(true);
|
||||
process.env.OFFICE_PREVIEW_LIBREOFFICE = 'pptx,docx';
|
||||
expect(isLibreOfficeEnabled()).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('isLibreOfficeEnabledFor (per-format opt-in)', () => {
|
||||
/* Per-format gating lets operators trade off cold-start latency
|
||||
* against fidelity per format. DOCX renders ~instantly via
|
||||
* docx-preview, but PPTX needs LibreOffice for any reasonable
|
||||
* fidelity (pptx-preview chokes on pptxgenjs decks). Operators
|
||||
* should be able to enable LibreOffice for PPTX only. */
|
||||
it.each([
|
||||
['true', 'pptx', true],
|
||||
['true', 'docx', true],
|
||||
['true', 'xlsx', true],
|
||||
['1', 'pptx', true],
|
||||
['yes', 'docx', true],
|
||||
['false', 'pptx', false],
|
||||
['', 'pptx', false],
|
||||
['pptx', 'pptx', true],
|
||||
['pptx', 'docx', false],
|
||||
['docx', 'pptx', false],
|
||||
['docx', 'docx', true],
|
||||
['pptx,docx', 'pptx', true],
|
||||
['pptx,docx', 'docx', true],
|
||||
['pptx,docx', 'xlsx', false],
|
||||
[' pptx , docx ', 'pptx', true], // whitespace-tolerant
|
||||
['PPTX', 'pptx', true], // case-insensitive
|
||||
['pptx, ,docx', 'docx', true], // empty list entries dropped
|
||||
['pptx, ,docx', 'pptx', true],
|
||||
['pptx, ,docx', 'xlsx', false],
|
||||
])('env=%j format=%j → %s', (envValue, format, expected) => {
|
||||
process.env.OFFICE_PREVIEW_LIBREOFFICE = envValue;
|
||||
expect(isLibreOfficeEnabledFor(format)).toBe(expected);
|
||||
});
|
||||
|
||||
it('returns false for any format when the env var is unset', () => {
|
||||
delete process.env.OFFICE_PREVIEW_LIBREOFFICE;
|
||||
expect(isLibreOfficeEnabledFor('pptx')).toBe(false);
|
||||
expect(isLibreOfficeEnabledFor('docx')).toBe(false);
|
||||
expect(isLibreOfficeEnabledFor('xlsx')).toBe(false);
|
||||
});
|
||||
|
||||
it('matches format names case-insensitively', () => {
|
||||
process.env.OFFICE_PREVIEW_LIBREOFFICE = 'pptx';
|
||||
expect(isLibreOfficeEnabledFor('PPTX')).toBe(true);
|
||||
expect(isLibreOfficeEnabledFor('Pptx')).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildPdfEmbedDocument', () => {
|
||||
/* The wrapper structure is the security-relevant surface. These
|
||||
* tests lock down the CSP, the iframe shape, and the fallback
|
||||
* contract — independent of whether LibreOffice is actually
|
||||
* installed. Same posture as the docx-preview / pptx-preview CDN
|
||||
* wrappers. */
|
||||
const FAKE_PDF_B64 = 'JVBERi0xLjQK'; // "%PDF-1.4\n" base64
|
||||
|
||||
it('emits a complete sandboxed HTML document that renders via pdf.js to canvas', () => {
|
||||
const html = buildPdfEmbedDocument(FAKE_PDF_B64);
|
||||
expect(html).toMatch(/^<!DOCTYPE html>/);
|
||||
expect(html).toContain('<title>Preview</title>');
|
||||
/* PDF bytes embedded as a base64 data block — pdf.js decodes
|
||||
* them at runtime and renders to canvas. We do NOT use any
|
||||
* `<iframe src="data:application/pdf;...">` or `src="blob:...">`
|
||||
* pattern because Chrome blocks BOTH data: AND blob: PDF
|
||||
* navigations in sandboxed iframes (the built-in PDF viewer
|
||||
* requires a top-level browsing context). The Sandpack host
|
||||
* iframe is sandboxed, so neither approach renders. PDF.js
|
||||
* draws to canvas which works in any context. Manual e2e on
|
||||
* PR #12934. */
|
||||
expect(html).toContain('id="lc-pdf-data"');
|
||||
expect(html).toContain(FAKE_PDF_B64);
|
||||
expect(html).toContain('id="lc-render"');
|
||||
/* Negative assertions: we MUST NOT have any nested iframe
|
||||
* navigation to a PDF URL. A future "let's just embed the PDF
|
||||
* natively" rewrite can't silently re-introduce the Chrome
|
||||
* block. */
|
||||
expect(html).not.toMatch(/src="data:application\/pdf/);
|
||||
expect(html).not.toMatch(/src="blob:[^"]*application\/pdf/);
|
||||
expect(html).not.toMatch(/<iframe[^>]+id="lc-pdf"/);
|
||||
/* PDF.js loaded from CDN. */
|
||||
expect(html).toContain('cdn.jsdelivr.net/npm/pdfjs-dist@');
|
||||
expect(html).toContain('pdfjsLib.getDocument');
|
||||
expect(html).toContain('GlobalWorkerOptions.workerSrc');
|
||||
expect(html).toContain('page.render');
|
||||
});
|
||||
|
||||
it('CSP allows pdf.js script + worker from jsdelivr; locks down everything else', () => {
|
||||
const html = buildPdfEmbedDocument(FAKE_PDF_B64);
|
||||
const cspMatch = html.match(/<meta http-equiv="Content-Security-Policy" content="([^"]+)">/);
|
||||
expect(cspMatch).not.toBeNull();
|
||||
const csp = cspMatch![1];
|
||||
expect(csp).toMatch(/default-src 'none'/);
|
||||
/* pdf.js needs its main script + worker. Both come from the
|
||||
* same jsdelivr host. */
|
||||
expect(csp).toMatch(/script-src https:\/\/cdn\.jsdelivr\.net 'unsafe-inline'/);
|
||||
expect(csp).toMatch(/worker-src[^;]*https:\/\/cdn\.jsdelivr\.net/);
|
||||
expect(csp).toMatch(/worker-src[^;]*\bblob:/);
|
||||
/* Negative assertions for the previous-iteration approaches:
|
||||
* no `frame-src` (no nested iframes anymore), no PDF data: or
|
||||
* blob: navigation paths. */
|
||||
expect(csp).not.toMatch(/frame-src/);
|
||||
/* No outbound HTTP from the rendered iframe — pdf.js doesn't
|
||||
* fetch anything (PDF bytes are inline, fonts subset-embedded
|
||||
* by LibreOffice). */
|
||||
expect(csp).toMatch(/connect-src 'none'/);
|
||||
expect(csp).not.toMatch(/unsafe-eval/);
|
||||
expect(csp).toMatch(/base-uri 'none'/);
|
||||
expect(csp).toMatch(/form-action 'none'/);
|
||||
});
|
||||
|
||||
it('exposes a fallback message + diagnostic disclosure when pdf.js fails', () => {
|
||||
const html = buildPdfEmbedDocument(FAKE_PDF_B64);
|
||||
expect(html).toContain('id="lc-fallback"');
|
||||
expect(html).toContain('PDF preview unavailable');
|
||||
expect(html).toContain('id="lc-fallback-reason"');
|
||||
expect(html).toContain('Diagnostic details');
|
||||
/* Multiple failure paths feed showFallback: pdf.js not loaded,
|
||||
* unhandled rejection from the parser, sync error from the
|
||||
* bootstrap, render timeout. */
|
||||
expect(html).toContain("typeof pdfjsLib === 'undefined'");
|
||||
expect(html).toContain("addEventListener('unhandledrejection'");
|
||||
expect(html).toContain('pdf-render-timeout');
|
||||
expect(html).toContain('15000');
|
||||
/* Reasons logged to console.error for power-user debugging. */
|
||||
expect(html).toContain("console.error('[libreoffice-pdf] fallback fired:'");
|
||||
});
|
||||
|
||||
it('embeds large base64 payloads inside the data block without escaping issues', () => {
|
||||
/* The base64 alphabet (A-Za-z0-9+/=) contains no characters that
|
||||
* could break out of `<script type="application/octet-stream;
|
||||
* base64">...</script>`. Sanity-check the data round-trip. */
|
||||
const big = 'A'.repeat(100_000);
|
||||
const html = buildPdfEmbedDocument(big);
|
||||
const dataBlock = html.match(
|
||||
/<script id="lc-pdf-data" type="application\/octet-stream;base64">([^<]+)<\/script>/,
|
||||
);
|
||||
expect(dataBlock).not.toBeNull();
|
||||
expect(dataBlock![1]).toBe(big);
|
||||
});
|
||||
});
|
||||
|
||||
describe('tryLibreOfficePreview (gating + fallthrough contract)', () => {
|
||||
it('returns null when the env flag is unset (default behavior)', async () => {
|
||||
delete process.env.OFFICE_PREVIEW_LIBREOFFICE;
|
||||
const buf = readFixture('sample.docx');
|
||||
const out = await tryLibreOfficePreview(buf, 'docx', 512 * 1024);
|
||||
expect(out).toBeNull();
|
||||
});
|
||||
|
||||
it('returns null when env flag is "false"', async () => {
|
||||
process.env.OFFICE_PREVIEW_LIBREOFFICE = 'false';
|
||||
const buf = readFixture('sample.docx');
|
||||
const out = await tryLibreOfficePreview(buf, 'docx', 512 * 1024);
|
||||
expect(out).toBeNull();
|
||||
});
|
||||
|
||||
it('never throws — falls through to null on any conversion failure', async () => {
|
||||
/* Even if the binary IS available, a malformed buffer should
|
||||
* cause `convertOfficeToPdf` to throw and `tryLibreOfficePreview`
|
||||
* to swallow it. The dispatcher pipeline takes over from there. */
|
||||
process.env.OFFICE_PREVIEW_LIBREOFFICE = 'true';
|
||||
const garbage = Buffer.from('this-is-definitely-not-a-docx');
|
||||
let threw = false;
|
||||
try {
|
||||
const out = await tryLibreOfficePreview(garbage, 'docx', 512 * 1024);
|
||||
expect(out).toBeNull();
|
||||
} catch {
|
||||
threw = true;
|
||||
}
|
||||
expect(threw).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('probeLibreOfficeBinary', () => {
|
||||
it("caches the probe result so we don't re-spawn `--version` on every call", async () => {
|
||||
_resetLibreOfficeProbeCache();
|
||||
const first = await probeLibreOfficeBinary();
|
||||
const second = await probeLibreOfficeBinary();
|
||||
/* Reference equality — second call returns the cached object. */
|
||||
expect(second).toBe(first);
|
||||
});
|
||||
|
||||
it('returns availability=false with a reason when the binary is missing', async () => {
|
||||
_resetLibreOfficeProbeCache();
|
||||
/* Force the probe to miss by clobbering PATH so neither
|
||||
* `soffice` nor `libreoffice` resolves. */
|
||||
const originalPath = process.env.PATH;
|
||||
process.env.PATH = '/nonexistent';
|
||||
try {
|
||||
const probe = await probeLibreOfficeBinary();
|
||||
expect(probe.available).toBe(false);
|
||||
expect(probe.binary).toBeNull();
|
||||
expect(probe.reason).toMatch(/found on .?PATH/i);
|
||||
} finally {
|
||||
process.env.PATH = originalPath;
|
||||
_resetLibreOfficeProbeCache();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('error tagging', () => {
|
||||
it('LibreOfficeUnavailableError preserves the .name tag for callers', () => {
|
||||
const err = new LibreOfficeUnavailableError('binary missing');
|
||||
expect(err.name).toBe('LibreOfficeUnavailableError');
|
||||
expect(err).toBeInstanceOf(Error);
|
||||
});
|
||||
|
||||
it('LibreOfficeConversionError carries optional stderr context', () => {
|
||||
const err = new LibreOfficeConversionError('exit 1', 'soffice: oops');
|
||||
expect(err.name).toBe('LibreOfficeConversionError');
|
||||
expect(err.stderr).toBe('soffice: oops');
|
||||
});
|
||||
|
||||
it('exports tunable subprocess limits so tests can read documented values', () => {
|
||||
expect(LIBREOFFICE_TIMEOUT_MS).toBe(30_000);
|
||||
expect(MAX_LIBREOFFICE_PDF_BYTES).toBe(50 * 1024 * 1024);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
/* eslint-disable jest/no-standalone-expect -- the `itIfLibreOffice` alias
|
||||
* collapses to `it.skip` when LibreOffice isn't installed; the lint rule
|
||||
* doesn't realize the `expect`s below are still scoped to a real test
|
||||
* block via the alias. Disabling locally is cleaner than restructuring
|
||||
* to satisfy a static check that's wrong about this pattern. */
|
||||
describe('libreoffice integration (skipped unless LibreOffice is on $PATH)', () => {
|
||||
/* These tests engage only when `soffice` (or `libreoffice`) is
|
||||
* actually installed. Local dev with the binary present runs them;
|
||||
* stock CI runners skip. The skip is per-test rather than describe-
|
||||
* level so a printout makes the gating visible. */
|
||||
const ORIGINAL_FLAG = process.env.OFFICE_PREVIEW_LIBREOFFICE;
|
||||
|
||||
beforeEach(() => {
|
||||
_resetLibreOfficeProbeCache();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (ORIGINAL_FLAG === undefined) {
|
||||
delete process.env.OFFICE_PREVIEW_LIBREOFFICE;
|
||||
} else {
|
||||
process.env.OFFICE_PREVIEW_LIBREOFFICE = ORIGINAL_FLAG;
|
||||
}
|
||||
});
|
||||
|
||||
itIfLibreOffice('probeLibreOfficeBinary detects the installed binary', async () => {
|
||||
const probe = await probeLibreOfficeBinary();
|
||||
expect(probe.available).toBe(true);
|
||||
expect(probe.binary).toMatch(/^(soffice|libreoffice)$/);
|
||||
expect(probe.versionLine).toMatch(/LibreOffice/i);
|
||||
});
|
||||
|
||||
itIfLibreOffice(
|
||||
'convertOfficeToPdf converts a DOCX to PDF bytes',
|
||||
async () => {
|
||||
const buf = readFixture('sample.docx');
|
||||
const pdf = await convertOfficeToPdf(buf, 'docx');
|
||||
/* PDF magic bytes: 25 50 44 46 == "%PDF". */
|
||||
expect(pdf.subarray(0, 4).toString('ascii')).toBe('%PDF');
|
||||
expect(pdf.length).toBeGreaterThan(500);
|
||||
},
|
||||
35_000,
|
||||
);
|
||||
|
||||
itIfLibreOffice(
|
||||
'convertOfficeToPdf converts a PPTX to PDF bytes',
|
||||
async () => {
|
||||
const buf = readFixture('sample.pptx');
|
||||
const pdf = await convertOfficeToPdf(buf, 'pptx');
|
||||
expect(pdf.subarray(0, 4).toString('ascii')).toBe('%PDF');
|
||||
},
|
||||
35_000,
|
||||
);
|
||||
|
||||
itIfLibreOffice(
|
||||
'tryLibreOfficePreview produces a PDF embed doc when env=true and binary is available',
|
||||
async () => {
|
||||
process.env.OFFICE_PREVIEW_LIBREOFFICE = 'true';
|
||||
_resetLibreOfficeProbeCache();
|
||||
const buf = readFixture('sample.docx');
|
||||
const out = await tryLibreOfficePreview(buf, 'docx', 512 * 1024);
|
||||
expect(out).not.toBeNull();
|
||||
expect(out!).toMatch(/^<!DOCTYPE html>/);
|
||||
/* PDF bytes embedded as a base64 data block; pdf.js renders to
|
||||
* canvas (Chrome blocks both data: and blob: PDF navigations
|
||||
* in sandboxed iframes — the canvas path is the only thing
|
||||
* that works in our context). */
|
||||
expect(out!).toContain('id="lc-pdf-data"');
|
||||
expect(out!).toContain('pdfjsLib.getDocument');
|
||||
expect(Buffer.byteLength(out!, 'utf-8')).toBeLessThanOrEqual(512 * 1024);
|
||||
},
|
||||
35_000,
|
||||
);
|
||||
|
||||
itIfLibreOffice(
|
||||
'returns null when the embedded PDF would exceed the output cap',
|
||||
async () => {
|
||||
/* Force the cap below the smallest possible PDF so the size
|
||||
* check trips. The conversion still runs (verifying the
|
||||
* subprocess works) but `tryLibreOfficePreview` declines to
|
||||
* emit oversized output and the dispatcher falls through. */
|
||||
process.env.OFFICE_PREVIEW_LIBREOFFICE = 'true';
|
||||
_resetLibreOfficeProbeCache();
|
||||
const buf = readFixture('sample.docx');
|
||||
const out = await tryLibreOfficePreview(buf, 'docx', 100);
|
||||
expect(out).toBeNull();
|
||||
},
|
||||
35_000,
|
||||
);
|
||||
});
|
||||
556
packages/api/src/files/documents/libreoffice.ts
Normal file
556
packages/api/src/files/documents/libreoffice.ts
Normal file
|
|
@ -0,0 +1,556 @@
|
|||
import { spawn } from 'child_process';
|
||||
import { mkdtemp, readFile, rm, writeFile } from 'fs/promises';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
|
||||
/**
|
||||
* LibreOffice-backed office preview pipeline.
|
||||
*
|
||||
* Convert a DOCX / PPTX (eventually XLSX, ODT) buffer to PDF via
|
||||
* `soffice --headless --convert-to pdf`, base64-encode it into a sandboxed
|
||||
* HTML document, and let pdf.js render each page to `<canvas>` inside the
|
||||
* Sandpack iframe. Result: high-fidelity rendering for any format
|
||||
* LibreOffice opens (which is "everything"), works inside Sandpack's
|
||||
* sandboxed iframe (where the browser's native PDF viewer is blocked).
|
||||
* The trade-off is the LibreOffice binary on the server (~250-350 MB disk,
|
||||
* ~2-3 s cold-start per first conversion in a process).
|
||||
*
|
||||
* Off by default. Operators opt in via the `OFFICE_PREVIEW_LIBREOFFICE`
|
||||
* env var AND ensuring `soffice` (or `libreoffice`) is on `$PATH`. The
|
||||
* env value is interpreted three ways:
|
||||
* - Truthy (`true`, `1`, `yes`): all formats use LibreOffice
|
||||
* - Falsy (`false`, `0`, `no`, empty, unset): no formats — fall through
|
||||
* - Comma-separated list (`pptx`, `pptx,docx`): only those formats
|
||||
*
|
||||
* The list form lets operators trade off cold-start latency against
|
||||
* fidelity per format. Practically: `pptx` is the most common opt-in —
|
||||
* pptx-preview chokes on pptxgenjs decks and the slide-list fallback
|
||||
* loses all formatting; LibreOffice handles them well. DOCX renders
|
||||
* ~instantly via docx-preview so paying the ~2-3 s LibreOffice cold-
|
||||
* start there is rarely worth it.
|
||||
*
|
||||
* When the gate is closed for a given format we fall through to the
|
||||
* existing CDN/mammoth/slide-list pipeline so a misconfiguration
|
||||
* doesn't break previews.
|
||||
*
|
||||
* Hardening:
|
||||
* - Subprocess runs in an isolated temp directory (no shared profile, no
|
||||
* access to the operator's home) and a stripped env (`PATH`, `HOME`,
|
||||
* `TMPDIR` only). LibreOffice's `-env:UserInstallation` flag forces a
|
||||
* fresh user profile per call so concurrent conversions can't collide
|
||||
* on shared ~/.config/libreoffice locks.
|
||||
* - 30-second timeout — soffice has been known to hang on malformed input;
|
||||
* `SIGKILL` after the timer expires.
|
||||
* - 50 MB PDF output cap — a runaway document or a deliberate pathology
|
||||
* can't blow the server's disk.
|
||||
* - Macros stay disabled (LibreOffice's default `--macro-security high`
|
||||
* plus our `--norestore --invisible --nodefault` flags).
|
||||
*
|
||||
* What this module is NOT:
|
||||
* - A LibreOffice service / pool. Each call spawns a fresh subprocess.
|
||||
* If load grows we'd want to keep a long-lived `soffice` process and
|
||||
* drive it via UNO, but that's a v2 concern.
|
||||
* - A native PDF viewer integration. We tried that first — Chrome blocks
|
||||
* `<iframe src="data:application/pdf">` AND `<iframe src="blob:...">`
|
||||
* navigation in sandboxed iframes (the built-in viewer requires a
|
||||
* top-level browsing context). pdf.js is the only thing that works
|
||||
* in our iframe topology.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Parse `OFFICE_PREVIEW_LIBREOFFICE` into the set of formats opted in.
|
||||
* Three forms:
|
||||
* - Truthy (`true`/`1`/`yes`): all formats
|
||||
* - Falsy (`false`/`0`/`no`/empty/unset): no formats
|
||||
* - Comma-separated list (`pptx`/`pptx,docx`): just those formats
|
||||
*
|
||||
* Returning `'all'` (vs. a Set containing every format) keeps the gate
|
||||
* future-proof — adding a new format to the LibreOffice route doesnt
|
||||
* require operators to re-enumerate their env value.
|
||||
*/
|
||||
type LibreOfficeFormatEnablement = 'all' | ReadonlySet<string> | null;
|
||||
|
||||
function parseLibreOfficeEnablement(value: string | undefined): LibreOfficeFormatEnablement {
|
||||
if (value == null) {
|
||||
return null;
|
||||
}
|
||||
const trimmed = value.trim();
|
||||
if (trimmed === '') {
|
||||
return null;
|
||||
}
|
||||
if (/^(1|true|yes)$/i.test(trimmed)) {
|
||||
return 'all';
|
||||
}
|
||||
if (/^(0|false|no)$/i.test(trimmed)) {
|
||||
return null;
|
||||
}
|
||||
/* Comma-separated format list. Lowercased + trimmed; empty entries
|
||||
* dropped so trailing commas / `pptx, ,docx` don't enable spurious
|
||||
* formats. */
|
||||
const formats = trimmed
|
||||
.split(',')
|
||||
.map((s) => s.trim().toLowerCase())
|
||||
.filter(Boolean);
|
||||
return formats.length > 0 ? new Set(formats) : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the LibreOffice path is enabled for this specific format.
|
||||
* Read at call time (not module load) so tests can flip the env
|
||||
* without rebuilding.
|
||||
*
|
||||
* @param format extension token (`docx`, `pptx`, `xlsx`, `odt`, ...)
|
||||
*/
|
||||
export function isLibreOfficeEnabledFor(format: string): boolean {
|
||||
const enabled = parseLibreOfficeEnablement(process.env.OFFICE_PREVIEW_LIBREOFFICE);
|
||||
if (enabled === null) {
|
||||
return false;
|
||||
}
|
||||
if (enabled === 'all') {
|
||||
return true;
|
||||
}
|
||||
return enabled.has(format.toLowerCase());
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether ANY format is enabled — kept for diagnostic / probe code that
|
||||
* wants to short-circuit binary checks when the feature is fully off.
|
||||
* Most production callers should use `isLibreOfficeEnabledFor(format)`.
|
||||
*/
|
||||
export function isLibreOfficeEnabled(): boolean {
|
||||
return parseLibreOfficeEnablement(process.env.OFFICE_PREVIEW_LIBREOFFICE) !== null;
|
||||
}
|
||||
|
||||
interface BinaryProbe {
|
||||
available: boolean;
|
||||
binary: string | null;
|
||||
versionLine: string | null;
|
||||
reason?: string;
|
||||
}
|
||||
|
||||
let cachedProbe: BinaryProbe | null = null;
|
||||
|
||||
/**
|
||||
* Probe `soffice` then `libreoffice` once per process and cache the result.
|
||||
* If the operator installs LibreOffice after server start, they need to
|
||||
* restart to pick it up — simpler than invalidating the cache on filesystem
|
||||
* events for a feature that's already opt-in.
|
||||
*/
|
||||
export async function probeLibreOfficeBinary(): Promise<BinaryProbe> {
|
||||
if (cachedProbe) {
|
||||
return cachedProbe;
|
||||
}
|
||||
for (const candidate of ['soffice', 'libreoffice']) {
|
||||
try {
|
||||
const versionLine = await runVersion(candidate);
|
||||
cachedProbe = { available: true, binary: candidate, versionLine };
|
||||
return cachedProbe;
|
||||
} catch {
|
||||
/* try next candidate */
|
||||
}
|
||||
}
|
||||
cachedProbe = {
|
||||
available: false,
|
||||
binary: null,
|
||||
versionLine: null,
|
||||
reason: 'neither `soffice` nor `libreoffice` found on $PATH',
|
||||
};
|
||||
return cachedProbe;
|
||||
}
|
||||
|
||||
function runVersion(binary: string): Promise<string> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const proc = spawn(binary, ['--version'], {
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
env: { PATH: process.env.PATH ?? '', HOME: tmpdir() },
|
||||
});
|
||||
let stdout = '';
|
||||
proc.stdout?.on('data', (chunk) => {
|
||||
stdout += String(chunk);
|
||||
});
|
||||
const timer = setTimeout(() => {
|
||||
proc.kill('SIGKILL');
|
||||
reject(new Error('--version timed out'));
|
||||
}, 5_000);
|
||||
proc.once('error', (err) => {
|
||||
clearTimeout(timer);
|
||||
reject(err);
|
||||
});
|
||||
proc.once('exit', (code) => {
|
||||
clearTimeout(timer);
|
||||
if (code === 0) {
|
||||
resolve(stdout.split('\n')[0]?.trim() ?? '');
|
||||
} else {
|
||||
reject(new Error(`exit ${code}`));
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/** Reset the cached probe — for tests; never called from production code. */
|
||||
export function _resetLibreOfficeProbeCache(): void {
|
||||
cachedProbe = null;
|
||||
}
|
||||
|
||||
/** Maximum subprocess wall-time. Tuned for cold-start on a small DOCX. */
|
||||
export const LIBREOFFICE_TIMEOUT_MS = 30_000;
|
||||
|
||||
/** Maximum PDF output size; refuse anything larger so a runaway doc can't fill the disk. */
|
||||
export const MAX_LIBREOFFICE_PDF_BYTES = 50 * 1024 * 1024;
|
||||
|
||||
/** Tag-distinct error so callers can distinguish "binary missing" from "conversion failed". */
|
||||
export class LibreOfficeUnavailableError extends Error {
|
||||
override readonly name = 'LibreOfficeUnavailableError';
|
||||
}
|
||||
|
||||
/** Tag-distinct error so callers can swallow conversion-side failures and fall through. */
|
||||
export class LibreOfficeConversionError extends Error {
|
||||
override readonly name = 'LibreOfficeConversionError';
|
||||
constructor(
|
||||
message: string,
|
||||
readonly stderr?: string,
|
||||
) {
|
||||
super(message);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Spawn a LibreOffice subprocess to convert `buffer` (in `extensionHint`
|
||||
* format — `docx`, `pptx`, `xlsx`, `odt`, `odp`, `ods`) to PDF. Returns
|
||||
* the PDF bytes.
|
||||
*
|
||||
* `extensionHint` is part of the temp filename so soffice infers the input
|
||||
* format correctly. We never trust the extension to gate format — the
|
||||
* caller (`html.ts` dispatcher) has already routed by MIME / magic bytes.
|
||||
*
|
||||
* Throws:
|
||||
* - `LibreOfficeUnavailableError` when the binary isn't on $PATH
|
||||
* - `LibreOfficeConversionError` for subprocess failures, timeout, or
|
||||
* oversized output
|
||||
*/
|
||||
export async function convertOfficeToPdf(buffer: Buffer, extensionHint: string): Promise<Buffer> {
|
||||
const probe = await probeLibreOfficeBinary();
|
||||
if (!probe.available || !probe.binary) {
|
||||
throw new LibreOfficeUnavailableError(probe.reason ?? 'LibreOffice binary unavailable');
|
||||
}
|
||||
const safeExt = extensionHint.replace(/[^a-z0-9]/gi, '').toLowerCase() || 'bin';
|
||||
const tempDir = await mkdtemp(join(tmpdir(), 'lc-libreoffice-'));
|
||||
try {
|
||||
const inputPath = join(tempDir, `input.${safeExt}`);
|
||||
await writeFile(inputPath, buffer);
|
||||
await runConversion(probe.binary, inputPath, tempDir);
|
||||
const outputPath = join(tempDir, 'input.pdf');
|
||||
const pdf = await readFile(outputPath);
|
||||
if (pdf.length > MAX_LIBREOFFICE_PDF_BYTES) {
|
||||
throw new LibreOfficeConversionError(
|
||||
`PDF output ${pdf.length} bytes exceeds cap ${MAX_LIBREOFFICE_PDF_BYTES}`,
|
||||
);
|
||||
}
|
||||
return pdf;
|
||||
} finally {
|
||||
/* Best-effort cleanup. If unlink fails (rare — Linux tmpfs, no
|
||||
* unmounts), the OS will reclaim the dir on next reboot. */
|
||||
await rm(tempDir, { recursive: true, force: true }).catch(() => undefined);
|
||||
}
|
||||
}
|
||||
|
||||
function runConversion(binary: string, inputPath: string, tempDir: string): Promise<void> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const userProfile = `file://${join(tempDir, 'userprof')}`;
|
||||
const proc = spawn(
|
||||
binary,
|
||||
[
|
||||
'--headless',
|
||||
'--norestore',
|
||||
'--invisible',
|
||||
'--nodefault',
|
||||
'--nofirststartwizard',
|
||||
'--nolockcheck',
|
||||
`-env:UserInstallation=${userProfile}`,
|
||||
'--convert-to',
|
||||
'pdf',
|
||||
'--outdir',
|
||||
tempDir,
|
||||
inputPath,
|
||||
],
|
||||
{
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
/* Stripped env. PATH is required for soffice to find its own libs;
|
||||
* HOME/TMPDIR pinned to the isolated temp dir so the subprocess
|
||||
* can't read the operator's profile or scribble outside our tree. */
|
||||
env: {
|
||||
PATH: process.env.PATH ?? '',
|
||||
HOME: tempDir,
|
||||
TMPDIR: tempDir,
|
||||
/* Suppress LibreOffice's "Recovery" dialog state on macOS. */
|
||||
DBUS_SESSION_BUS_ADDRESS: 'disabled:',
|
||||
},
|
||||
},
|
||||
);
|
||||
let stderr = '';
|
||||
proc.stderr?.on('data', (chunk) => {
|
||||
stderr += String(chunk);
|
||||
});
|
||||
const timer = setTimeout(() => {
|
||||
proc.kill('SIGKILL');
|
||||
reject(
|
||||
new LibreOfficeConversionError(
|
||||
`LibreOffice timeout after ${LIBREOFFICE_TIMEOUT_MS}ms`,
|
||||
stderr,
|
||||
),
|
||||
);
|
||||
}, LIBREOFFICE_TIMEOUT_MS);
|
||||
proc.once('error', (err) => {
|
||||
clearTimeout(timer);
|
||||
reject(new LibreOfficeConversionError(err.message, stderr));
|
||||
});
|
||||
proc.once('exit', (code) => {
|
||||
clearTimeout(timer);
|
||||
if (code === 0) {
|
||||
resolve();
|
||||
} else {
|
||||
reject(new LibreOfficeConversionError(`soffice exit ${code}`, stderr));
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the iframe HTML that embeds the PDF for in-panel rendering. The
|
||||
* PDF is base64-encoded as a `data:` URI and pointed to by an inner
|
||||
* canvas via PDF.js (Mozilla's pdfjs-dist) loaded from CDN.
|
||||
*
|
||||
* Why PDF.js + canvas (vs. native browser PDF viewer):
|
||||
* We tried `<iframe src="data:application/pdf;...">` first — Chrome
|
||||
* blocks data: PDF navigation in sandboxed iframes since Chrome 76.
|
||||
* We tried `<iframe src="blob:...">` next — Chrome ALSO blocks
|
||||
* blob: PDFs in sandboxed iframes (the built-in PDF viewer requires
|
||||
* a top-level browsing context). The Sandpack host iframe IS
|
||||
* sandboxed, so neither works. Manual e2e on PR #12934 — both
|
||||
* produced the "This page has been blocked by Chrome" interstitial.
|
||||
*
|
||||
* PDF.js renders to `<canvas>` which works in ANY context (sandboxed,
|
||||
* embedded, restricted CSP) because it's pure JS — no plugin, no
|
||||
* privileged viewer. ~1 MB CDN load is acceptable for a feature
|
||||
* that's already env-gated and opt-in.
|
||||
*
|
||||
* Worker handling:
|
||||
* PDF.js wants a Web Worker for parsing (offloads CPU from main
|
||||
* thread). The worker URL is loaded from the same jsdelivr origin;
|
||||
* CSP `worker-src https://cdn.jsdelivr.net blob:` allows it. blob:
|
||||
* covers the case where pdf.js wraps the worker source in a Blob
|
||||
* to dodge cross-origin worker restrictions.
|
||||
*/
|
||||
const PDF_JS_CDN = {
|
||||
/* Pinned to v3.11.174 (last v3 release) — it's a single-file UMD
|
||||
* bundle that loads via a plain `<script>` tag. v4+ uses ES modules
|
||||
* (`pdf.min.mjs`) which complicates the load + SRI flow. v3 still
|
||||
* receives security backports per Mozilla's policy.
|
||||
*
|
||||
* SRI hashes intentionally OMITTED here (unlike docx-preview /
|
||||
* pptx-preview) because the LibreOffice preview path is opt-in and
|
||||
* the operator has already chosen to trust the LibreOffice render
|
||||
* pipeline. Adding SRI is a follow-up worth doing once this path
|
||||
* is proven in production. */
|
||||
pdf: 'https://cdn.jsdelivr.net/npm/pdfjs-dist@3.11.174/build/pdf.min.js',
|
||||
worker: 'https://cdn.jsdelivr.net/npm/pdfjs-dist@3.11.174/build/pdf.worker.min.js',
|
||||
} as const;
|
||||
|
||||
export function buildPdfEmbedDocument(pdfBase64: string): string {
|
||||
/* CSP scoping:
|
||||
* - `default-src 'none'`: lock everything down.
|
||||
* - `script-src https://cdn.jsdelivr.net 'unsafe-inline'`: load
|
||||
* pdf.js from CDN + run our bootstrap inline.
|
||||
* - `worker-src https://cdn.jsdelivr.net blob:`: pdf.js spawns a
|
||||
* parser worker; jsdelivr is the origin we loaded the worker
|
||||
* script from, blob: covers pdf.js's same-origin worker wrap.
|
||||
* - `connect-src 'none'`: pdf.js doesn't fetch anything — the
|
||||
* PDF bytes are inline, fonts are subset-embedded by LibreOffice.
|
||||
* - `style-src 'unsafe-inline'`: page chrome.
|
||||
* - `img-src 'self' data: blob:`: pdf.js may rasterize embedded
|
||||
* bitmaps via canvas (data:/blob: covers internal handoffs). */
|
||||
const csp = [
|
||||
"default-src 'none'",
|
||||
"script-src https://cdn.jsdelivr.net 'unsafe-inline'",
|
||||
'worker-src https://cdn.jsdelivr.net blob:',
|
||||
"style-src 'unsafe-inline'",
|
||||
"img-src 'self' data: blob:",
|
||||
"connect-src 'none'",
|
||||
"base-uri 'none'",
|
||||
"form-action 'none'",
|
||||
].join('; ');
|
||||
return `<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||||
<meta http-equiv="Content-Security-Policy" content="${csp}">
|
||||
<title>Preview</title>
|
||||
<style>
|
||||
:root { color-scheme: light dark; --bg: #ffffff; --fg: #1f2937; --muted: #6b7280; }
|
||||
@media (prefers-color-scheme: dark) { :root { --bg: #1a1a2e; --fg: #e5e7eb; --muted: #9ca3af; } }
|
||||
html, body { margin: 0; padding: 0; min-height: 100vh; background: var(--bg); color: var(--fg); font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif; }
|
||||
#lc-render { padding: 16px; box-sizing: border-box; display: flex; flex-direction: column; align-items: center; gap: 16px; }
|
||||
#lc-render canvas { max-width: 100%; height: auto; box-shadow: 0 2px 8px rgba(0, 0, 0, 0.18); border-radius: 4px; background: #ffffff; }
|
||||
#lc-loading { padding: 24px; color: var(--muted); font-size: 14px; text-align: center; }
|
||||
#lc-fallback { display: none; padding: 24px; font-size: 14px; line-height: 1.5; color: var(--muted); text-align: center; }
|
||||
#lc-fallback.visible { display: block; }
|
||||
#lc-fallback-reason { font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 12px; color: var(--muted); margin-top: 6px; word-break: break-word; }
|
||||
</style>
|
||||
<script src="${PDF_JS_CDN.pdf}"></script>
|
||||
</head>
|
||||
<body>
|
||||
<div id="lc-render"><div id="lc-loading">Loading preview…</div></div>
|
||||
<div id="lc-fallback">
|
||||
<p>PDF preview unavailable. Please download the file to view it.</p>
|
||||
<details style="margin-top: 8px;">
|
||||
<summary style="cursor: pointer; font-size: 12px;">Diagnostic details</summary>
|
||||
<div id="lc-fallback-reason"></div>
|
||||
</details>
|
||||
</div>
|
||||
<script id="lc-pdf-data" type="application/octet-stream;base64">${pdfBase64}</script>
|
||||
<script>
|
||||
(function () {
|
||||
var container = document.getElementById('lc-render');
|
||||
var loading = document.getElementById('lc-loading');
|
||||
var fallback = document.getElementById('lc-fallback');
|
||||
var reasonEl = document.getElementById('lc-fallback-reason');
|
||||
var settled = false;
|
||||
|
||||
function showFallback(reason) {
|
||||
if (settled) { return; }
|
||||
settled = true;
|
||||
if (loading) { loading.remove(); }
|
||||
if (container) { container.style.display = 'none'; }
|
||||
if (fallback) { fallback.classList.add('visible'); }
|
||||
var text = reason ? String(reason).slice(0, 500) : 'no reason reported';
|
||||
if (reasonEl) { reasonEl.textContent = text; }
|
||||
if (typeof console !== 'undefined' && console.error) {
|
||||
console.error('[libreoffice-pdf] fallback fired:', text);
|
||||
}
|
||||
}
|
||||
function markSuccess() { settled = true; }
|
||||
|
||||
/* pdf.js wraps async errors as unhandled rejections; catch them at
|
||||
* the window level so we never silently fail. */
|
||||
window.addEventListener('unhandledrejection', function (e) {
|
||||
if (settled) { return; }
|
||||
showFallback((e.reason && e.reason.message) || 'unhandled-rejection');
|
||||
});
|
||||
window.addEventListener('error', function (e) {
|
||||
if (settled) { return; }
|
||||
showFallback((e.error && e.error.message) || e.message || 'script-error');
|
||||
});
|
||||
|
||||
if (typeof pdfjsLib === 'undefined' || typeof pdfjsLib.getDocument !== 'function') {
|
||||
showFallback('renderer-not-loaded (pdf.js failed to load from jsdelivr)');
|
||||
return;
|
||||
}
|
||||
|
||||
/* Point the worker at the same CDN version we loaded the main
|
||||
* script from. pdf.js fetches this URL and spawns a Worker; CSP
|
||||
* worker-src covers it. */
|
||||
pdfjsLib.GlobalWorkerOptions.workerSrc = '${PDF_JS_CDN.worker}';
|
||||
|
||||
try {
|
||||
var b64 = document.getElementById('lc-pdf-data').textContent.trim();
|
||||
var bytes = Uint8Array.from(atob(b64), function (c) { return c.charCodeAt(0); });
|
||||
|
||||
pdfjsLib.getDocument({ data: bytes }).promise.then(function (pdf) {
|
||||
if (loading) { loading.remove(); }
|
||||
|
||||
/* Render each page sequentially. We pick a render scale based
|
||||
* on the panel width and the first pages native dimensions so
|
||||
* the canvas matches the panel — no upscaling required by CSS
|
||||
* (which would blur it). DPR-aware so retina screens get crisp
|
||||
* pixels. */
|
||||
var dpr = window.devicePixelRatio || 1;
|
||||
var panelWidth = (container.clientWidth || window.innerWidth) - 32;
|
||||
|
||||
function renderPage(pageNum) {
|
||||
return pdf.getPage(pageNum).then(function (page) {
|
||||
var unscaledViewport = page.getViewport({ scale: 1 });
|
||||
var cssScale = Math.max(0.1, panelWidth / unscaledViewport.width);
|
||||
var viewport = page.getViewport({ scale: cssScale * dpr });
|
||||
var canvas = document.createElement('canvas');
|
||||
canvas.width = viewport.width;
|
||||
canvas.height = viewport.height;
|
||||
/* CSS dimensions in logical pixels; canvas backing store at
|
||||
* DPR multiplier for crisp rendering. */
|
||||
canvas.style.width = (viewport.width / dpr) + 'px';
|
||||
canvas.style.height = (viewport.height / dpr) + 'px';
|
||||
container.appendChild(canvas);
|
||||
var ctx = canvas.getContext('2d');
|
||||
return page.render({ canvasContext: ctx, viewport: viewport }).promise;
|
||||
});
|
||||
}
|
||||
|
||||
var chain = Promise.resolve();
|
||||
for (var i = 1; i <= pdf.numPages; i++) {
|
||||
(function (pageNum) {
|
||||
chain = chain.then(function () { return renderPage(pageNum); });
|
||||
})(i);
|
||||
}
|
||||
return chain.then(markSuccess);
|
||||
}).catch(function (err) {
|
||||
showFallback((err && err.message) || 'pdf-render-failed');
|
||||
});
|
||||
|
||||
/* Safety net: if 15 seconds in pdf.js hasnt rendered anything
|
||||
* visible, accept whatever we have or fall through. PDF.js is
|
||||
* usually fast (<1s for typical chat decks), but big PDFs with
|
||||
* many slides + DPR=2 can take longer. */
|
||||
setTimeout(function () {
|
||||
if (settled) { return; }
|
||||
if (container && container.querySelectorAll('canvas').length > 0) {
|
||||
markSuccess();
|
||||
return;
|
||||
}
|
||||
showFallback('pdf-render-timeout');
|
||||
}, 15000);
|
||||
} catch (err) {
|
||||
showFallback((err && err.message) || 'bootstrap-error');
|
||||
}
|
||||
})();
|
||||
</script>
|
||||
</body>
|
||||
</html>`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Try the LibreOffice path. Returns the wrapped HTML on success, or `null`
|
||||
* if LibreOffice isn't enabled / available / converted successfully — the
|
||||
* caller falls through to the existing pipeline. Never throws.
|
||||
*
|
||||
* `extensionHint`: `docx` / `pptx` / `xlsx` / `odt` / `odp` / `ods` etc.
|
||||
*
|
||||
* Output-size guarantee: returns `null` if the embedded HTML would exceed
|
||||
* the 512 KB `attachment.text` cache cap. Base64 inflates by ~33% so the
|
||||
* effective PDF cap is ~380 KB; LibreOffice's PDF/A-1 default produces
|
||||
* compact output for typical chat-emitted documents.
|
||||
*/
|
||||
export async function tryLibreOfficePreview(
|
||||
buffer: Buffer,
|
||||
extensionHint: string,
|
||||
outputCap: number,
|
||||
): Promise<string | null> {
|
||||
if (!isLibreOfficeEnabledFor(extensionHint)) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
const pdf = await convertOfficeToPdf(buffer, extensionHint);
|
||||
const base64 = pdf.toString('base64');
|
||||
const html = buildPdfEmbedDocument(base64);
|
||||
if (Buffer.byteLength(html, 'utf-8') > outputCap) {
|
||||
return null;
|
||||
}
|
||||
return html;
|
||||
} catch (err) {
|
||||
/* Swallow both unavailable + conversion errors. The dispatcher
|
||||
* pipeline is the authoritative renderer; LibreOffice is opportunistic
|
||||
* augmentation. The caller's logger will surface the underlying
|
||||
* pipeline if they care to instrument it. */
|
||||
void err;
|
||||
return null;
|
||||
}
|
||||
}
|
||||
124
packages/api/src/files/documents/zipSafety.spec.ts
Normal file
124
packages/api/src/files/documents/zipSafety.spec.ts
Normal file
|
|
@ -0,0 +1,124 @@
|
|||
import JSZip from 'jszip';
|
||||
import { megabyte } from 'librechat-data-provider';
|
||||
import { assertSafeZipSize, ZipBombError } from './zipSafety';
|
||||
|
||||
/**
|
||||
* Build a ZIP archive whose entries inflate to exactly `decompressedBytes`
|
||||
* each. The data is highly compressible (single repeated character) so
|
||||
* compressed size stays small — roughly 0.5% of inflated size for runs
|
||||
* of zero bytes. Used to simulate the zip-bomb attack pattern from the
|
||||
* SEC validation PoC on PR #12934.
|
||||
*/
|
||||
const buildBombArchive = async (
|
||||
entries: Array<{ name: string; decompressedBytes: number }>,
|
||||
): Promise<Buffer> => {
|
||||
const zip = new JSZip();
|
||||
for (const { name, decompressedBytes } of entries) {
|
||||
zip.file(name, Buffer.alloc(decompressedBytes, 0));
|
||||
}
|
||||
return zip.generateAsync({
|
||||
type: 'nodebuffer',
|
||||
compression: 'DEFLATE',
|
||||
compressionOptions: { level: 9 },
|
||||
});
|
||||
};
|
||||
|
||||
/** Build a small, well-formed ZIP for the happy-path tests. */
|
||||
const buildBenignArchive = async (): Promise<Buffer> => {
|
||||
const zip = new JSZip();
|
||||
zip.file('hello.txt', 'hello world');
|
||||
zip.file('subdir/note.txt', 'second entry');
|
||||
return zip.generateAsync({ type: 'nodebuffer' });
|
||||
};
|
||||
|
||||
describe('assertSafeZipSize', () => {
|
||||
test('passes a benign small archive', async () => {
|
||||
const buffer = await buildBenignArchive();
|
||||
await expect(assertSafeZipSize(buffer)).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
test('passes an archive whose entries are all under both caps', async () => {
|
||||
const buffer = await buildBombArchive([
|
||||
{ name: 'a.bin', decompressedBytes: 1 * megabyte },
|
||||
{ name: 'b.bin', decompressedBytes: 1 * megabyte },
|
||||
]);
|
||||
await expect(
|
||||
assertSafeZipSize(buffer, { maxTotalBytes: 10 * megabyte, maxEntryBytes: 5 * megabyte }),
|
||||
).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
test('throws ZipBombError when a single entry exceeds the per-entry cap', async () => {
|
||||
/* Single 5 MB inflated entry compresses to a few KB. Per-entry cap of
|
||||
* 1 MB should fire mid-inflate. */
|
||||
const buffer = await buildBombArchive([{ name: 'big.bin', decompressedBytes: 5 * megabyte }]);
|
||||
await expect(assertSafeZipSize(buffer, { maxEntryBytes: 1 * megabyte })).rejects.toThrow(
|
||||
ZipBombError,
|
||||
);
|
||||
});
|
||||
|
||||
test('throws ZipBombError when total decompressed size exceeds the total cap', async () => {
|
||||
/* Many small-but-not-tiny entries that individually pass the per-entry
|
||||
* cap but collectively bust the total cap. Catches the multi-entry
|
||||
* variant of the attack. */
|
||||
const buffer = await buildBombArchive(
|
||||
Array.from({ length: 5 }, (_, i) => ({
|
||||
name: `chunk${i}.bin`,
|
||||
decompressedBytes: 1 * megabyte,
|
||||
})),
|
||||
);
|
||||
await expect(
|
||||
assertSafeZipSize(buffer, { maxTotalBytes: 3 * megabyte, maxEntryBytes: 2 * megabyte }),
|
||||
).rejects.toThrow(ZipBombError);
|
||||
});
|
||||
|
||||
test('cap-violation error is a ZipBombError, not a generic Error', async () => {
|
||||
const buffer = await buildBombArchive([{ name: 'big.bin', decompressedBytes: 5 * megabyte }]);
|
||||
/* Distinguishing the bomb case from a generic parse failure lets
|
||||
* the UI surface a meaningful "preview unavailable, file too large
|
||||
* to inflate" message instead of a vague 500. */
|
||||
await expect(assertSafeZipSize(buffer, { maxEntryBytes: 1 * megabyte })).rejects.toMatchObject({
|
||||
name: 'ZipBombError',
|
||||
code: 'ZIP_BOMB',
|
||||
});
|
||||
});
|
||||
|
||||
test('rejects a malformed zip', async () => {
|
||||
/* Not a real zip — yauzl will throw a parse error (NOT a
|
||||
* ZipBombError; that distinction matters to callers). */
|
||||
const buffer = Buffer.from('not a real zip');
|
||||
await expect(assertSafeZipSize(buffer)).rejects.toThrow();
|
||||
await expect(assertSafeZipSize(buffer)).rejects.not.toBeInstanceOf(ZipBombError);
|
||||
});
|
||||
|
||||
test('handles archives containing directory entries without crashing', async () => {
|
||||
const zip = new JSZip();
|
||||
zip.folder('emptydir');
|
||||
zip.file('emptydir/file.txt', 'data');
|
||||
const buffer = await zip.generateAsync({ type: 'nodebuffer' });
|
||||
await expect(assertSafeZipSize(buffer)).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
test('uses provided name in the error message for caller-side surfacing', async () => {
|
||||
const buffer = await buildBombArchive([{ name: 'big.bin', decompressedBytes: 5 * megabyte }]);
|
||||
await expect(
|
||||
assertSafeZipSize(buffer, { maxEntryBytes: 1 * megabyte, name: 'evil.docx' }),
|
||||
).rejects.toThrow(/evil\.docx/);
|
||||
});
|
||||
|
||||
test('re-PoC: catches the SEC-validation attack pattern (sub-1MB compressed → 100MB+ inflated)', async () => {
|
||||
/* Mirrors the SEC validation PoC shape: a sub-1MB compressed
|
||||
* archive whose entries inflate to many tens of MB. Tests that the
|
||||
* default caps fire on this canonical attack without the caller
|
||||
* needing to override anything. The PoC inflated to ~200MB across
|
||||
* several entries; we use 50MB for test-suite speed (still well
|
||||
* over both default caps). */
|
||||
const buffer = await buildBombArchive([
|
||||
{ name: 'word/document.xml', decompressedBytes: 50 * megabyte },
|
||||
]);
|
||||
/* Defense-in-depth check: the compressed payload IS small (proves
|
||||
* the input would slip past a compressed-size gate). */
|
||||
expect(buffer.length).toBeLessThan(1 * megabyte);
|
||||
/* And the validator catches it on default caps. */
|
||||
await expect(assertSafeZipSize(buffer)).rejects.toThrow(ZipBombError);
|
||||
});
|
||||
});
|
||||
165
packages/api/src/files/documents/zipSafety.ts
Normal file
165
packages/api/src/files/documents/zipSafety.ts
Normal file
|
|
@ -0,0 +1,165 @@
|
|||
import yauzl from 'yauzl';
|
||||
import { megabyte } from 'librechat-data-provider';
|
||||
|
||||
/**
|
||||
* Default per-archive total decompressed-size cap. Office documents in
|
||||
* normal use rarely exceed a few MB inflated; 100 MB leaves generous
|
||||
* headroom for image-heavy templates while still catching the
|
||||
* pathological zip-bomb case (e.g. a 1 MB compressed XLSX that inflates
|
||||
* to 200+ MB of XML — see SEC review on PR #12934).
|
||||
*/
|
||||
const DEFAULT_MAX_TOTAL_BYTES = 100 * megabyte;
|
||||
|
||||
/**
|
||||
* Default per-entry decompressed-size cap. A single inflated entry
|
||||
* larger than this is essentially always either a bomb or content the
|
||||
* downstream parser would balk at anyway.
|
||||
*/
|
||||
const DEFAULT_MAX_ENTRY_BYTES = 25 * megabyte;
|
||||
|
||||
/**
|
||||
* Tag-distinct error so callers (e.g. the office HTML producers and the
|
||||
* RAG document parser) can distinguish a refused zip-bomb from generic
|
||||
* parse failures and emit a sensible "file too large to preview" UI
|
||||
* instead of silently degrading.
|
||||
*/
|
||||
export class ZipBombError extends Error {
|
||||
readonly code = 'ZIP_BOMB';
|
||||
constructor(message: string) {
|
||||
super(message);
|
||||
this.name = 'ZipBombError';
|
||||
}
|
||||
}
|
||||
|
||||
export interface ZipSafetyOptions {
|
||||
/** Per-archive total decompressed-byte cap. */
|
||||
maxTotalBytes?: number;
|
||||
/** Per-entry decompressed-byte cap. */
|
||||
maxEntryBytes?: number;
|
||||
/** Filename for error messages — does not need to match disk. */
|
||||
name?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate that a ZIP-backed buffer (DOCX, XLSX, XLS-as-OOXML, ODS,
|
||||
* ODT, PPTX, …) does not blow up under decompression beyond the given
|
||||
* caps. Streams every entry through yauzl and counts real decompressed
|
||||
* bytes mid-inflate — the central directory's `uncompressedSize` cannot
|
||||
* be trusted (it can be falsified to lie about the payload, the
|
||||
* specific bypass technique used in the SEC validation PoC for
|
||||
* PR #12934).
|
||||
*
|
||||
* Drops the decompressed bytes immediately (only counts them), so the
|
||||
* validator's own memory footprint is bounded by yauzl's stream
|
||||
* buffer regardless of payload size. CPU is bounded by `maxTotalBytes`
|
||||
* — once the cap fires, the underlying read stream is destroyed and
|
||||
* decompression stops.
|
||||
*
|
||||
* Throws `ZipBombError` on cap violation; throws plain `Error` on a
|
||||
* malformed ZIP. Resolves with `void` when the archive is fully walked
|
||||
* within caps.
|
||||
*/
|
||||
export function assertSafeZipSize(buffer: Buffer, options: ZipSafetyOptions = {}): Promise<void> {
|
||||
const maxTotalBytes = options.maxTotalBytes ?? DEFAULT_MAX_TOTAL_BYTES;
|
||||
const maxEntryBytes = options.maxEntryBytes ?? DEFAULT_MAX_ENTRY_BYTES;
|
||||
const label = options.name ?? 'archive';
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
yauzl.fromBuffer(buffer, { lazyEntries: true }, (err, zipfile) => {
|
||||
if (err) {
|
||||
return reject(err);
|
||||
}
|
||||
if (!zipfile) {
|
||||
return reject(new Error('Failed to open zip buffer'));
|
||||
}
|
||||
|
||||
let settled = false;
|
||||
let totalDecompressed = 0;
|
||||
const finish = (error: Error | null) => {
|
||||
if (settled) {
|
||||
return;
|
||||
}
|
||||
settled = true;
|
||||
try {
|
||||
zipfile.close();
|
||||
} catch {
|
||||
/* zipfile.close() is best-effort — yauzl will throw if a
|
||||
* stream is mid-flight. We've already settled the outer
|
||||
* promise, so swallow this. */
|
||||
}
|
||||
if (error) {
|
||||
reject(error);
|
||||
} else {
|
||||
resolve();
|
||||
}
|
||||
};
|
||||
|
||||
zipfile.readEntry();
|
||||
|
||||
zipfile.on('entry', (entry: yauzl.Entry) => {
|
||||
/* Directory entries (trailing slash) are zero-byte and don't
|
||||
* need to be opened. Saves a stream allocation per directory
|
||||
* in archives like .docx that have nested subfolders. */
|
||||
if (/\/$/.test(entry.fileName)) {
|
||||
zipfile.readEntry();
|
||||
return;
|
||||
}
|
||||
|
||||
zipfile.openReadStream(entry, (streamErr, readStream) => {
|
||||
if (streamErr || !readStream) {
|
||||
return finish(streamErr ?? new Error('Failed to open zip entry stream'));
|
||||
}
|
||||
|
||||
let entryBytes = 0;
|
||||
let entryCapped = false;
|
||||
|
||||
readStream.on('data', (chunk: Buffer) => {
|
||||
if (entryCapped) {
|
||||
return;
|
||||
}
|
||||
entryBytes += chunk.byteLength;
|
||||
totalDecompressed += chunk.byteLength;
|
||||
if (entryBytes > maxEntryBytes) {
|
||||
entryCapped = true;
|
||||
readStream.destroy();
|
||||
finish(
|
||||
new ZipBombError(
|
||||
`${label}: entry "${entry.fileName}" exceeds the ${
|
||||
maxEntryBytes / megabyte
|
||||
}MB per-entry decompressed cap`,
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (totalDecompressed > maxTotalBytes) {
|
||||
entryCapped = true;
|
||||
readStream.destroy();
|
||||
finish(
|
||||
new ZipBombError(
|
||||
`${label}: total decompressed size exceeds the ${
|
||||
maxTotalBytes / megabyte
|
||||
}MB cap (zip bomb suspected)`,
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
});
|
||||
|
||||
readStream.on('end', () => {
|
||||
if (!settled) {
|
||||
zipfile.readEntry();
|
||||
}
|
||||
});
|
||||
readStream.on('error', (readErr: Error) => {
|
||||
if (!entryCapped) {
|
||||
finish(readErr);
|
||||
}
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
zipfile.on('end', () => finish(null));
|
||||
zipfile.on('error', (zipErr: Error) => finish(zipErr));
|
||||
});
|
||||
});
|
||||
}
|
||||
|
|
@ -117,6 +117,15 @@ export type TFile = {
|
|||
expiresAt?: string | Date;
|
||||
preview?: string;
|
||||
text?: string;
|
||||
/**
|
||||
* Format of the `text` field. `'html'` means the backend produced
|
||||
* a sanitized full-document HTML preview the client may inject as
|
||||
* `index.html` inside the office artifact iframe. `'text'` (or
|
||||
* `undefined` for legacy records) is plain text and MUST NOT be
|
||||
* injected as HTML — render through the markdown/escaping path.
|
||||
* See Codex P1 review on PR #12934.
|
||||
*/
|
||||
textFormat?: 'html' | 'text' | null;
|
||||
metadata?: { fileIdentifier?: string };
|
||||
createdAt?: string | Date;
|
||||
updatedAt?: string | Date;
|
||||
|
|
|
|||
|
|
@ -54,6 +54,16 @@ const file: Schema<IMongoFile> = new Schema(
|
|||
text: {
|
||||
type: String,
|
||||
},
|
||||
textFormat: {
|
||||
/* 'html' when the backend produced a sanitized HTML preview
|
||||
* (office-type CDN/mammoth output), 'text' for plain-text
|
||||
* extracts (RAG / pdf-parse / mammoth.extractRawText). Clients
|
||||
* gate office-bucket routing on textFormat === 'html' to
|
||||
* prevent injecting RAG-extracted plain text into the iframe
|
||||
* as HTML. See Codex P1 review on PR #12934. */
|
||||
type: String,
|
||||
enum: ['html', 'text'],
|
||||
},
|
||||
context: {
|
||||
type: String,
|
||||
},
|
||||
|
|
|
|||
|
|
@ -8,6 +8,17 @@ export interface IMongoFile extends Omit<Document, 'model'> {
|
|||
temp_file_id?: string;
|
||||
bytes: number;
|
||||
text?: string;
|
||||
/**
|
||||
* Format of the `text` field — `'html'` when the backend produced
|
||||
* a sanitized full-document HTML preview (e.g. office types via
|
||||
* `bufferToOfficeHtml`), `'text'` for plain-text extracts (e.g.
|
||||
* RAG mammoth/pdf-parse output), `undefined` for legacy records
|
||||
* that pre-date the field. Clients MUST treat `undefined` as
|
||||
* `'text'` and refuse to inject the value into HTML contexts —
|
||||
* otherwise plain document text containing `<script>` tags would
|
||||
* become executable markup. See Codex P1 review on PR #12934.
|
||||
*/
|
||||
textFormat?: 'html' | 'text';
|
||||
filename: string;
|
||||
filepath: string;
|
||||
object: 'file';
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue