From f69e8e26f8a5a59a0ed3921abaf0be25155f160b Mon Sep 17 00:00:00 2001 From: Danny Avila Date: Wed, 29 Apr 2026 08:07:19 +0900 Subject: [PATCH] =?UTF-8?q?=F0=9F=AA=9F=20feat:=20Render=20Source-Code=20A?= =?UTF-8?q?rtifacts=20in=20the=20Side=20Panel=20(#12854)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * 🪟 feat: Render Source-Code Artifacts in the Side Panel (CODE bucket) PR #12832 wired markdown / mermaid / html / .jsx-tsx tool outputs through the side-panel artifact pipeline but explicitly punted on code files: > Everything else (csv, py, json, xls/docx/pptx, …) keeps PR #12829's > inline behaviour — dedicated viewers will land in follow-ups. This adds the code-file viewer. A `simple_graph.py` (and every other common source file) now opens in the side panel alongside markdown, mermaid, html, and react artifacts instead of falling back to the inline `
` rendering.

**Design.** New `CODE: 'application/vnd.code'` bucket reuses the static-
markdown sandpack template — `useArtifactProps` pre-wraps the source as
a fenced code block (` ```python\n...\n``` `) before handing it to
`getMarkdownFiles`. The fence carries a `language-` class through
`marked`, so a future highlighter swap-in (e.g. drop `highlight.js`
into the markdown template) picks up syntax colors automatically. The
`react-ts` (sandpack) template's React boot cost is avoided since
source files don't need it.

**Single source of truth for languages.** New `CODE_EXTENSION_TO_LANGUAGE`
map drives BOTH:
  - `EXTENSION_TO_TOOL_ARTIFACT_TYPE` routing (presence in this map =
    code file). Adding a new language is one entry.
  - The fenced-block language hint (exported as `languageForFilename`).

Identifiers follow the GitHub / `highlight.js` convention so the future
highlighter pickup is automatic.

**Scope.** Programming languages + stylesheets + shell + sql/graphql +
build files (Dockerfile/Makefile/HCL). Pure data formats
(CSV/TSV/JSON/JSONL/NDJSON/XML/YAML/TOML) and config dotfiles
(`.env`/`.ini`/`.conf`/`.cfg`) are intentionally NOT routed in this
pass — they're better served by dedicated viewers (CSV table view,
etc.) or remain inline. Adding them later is a one-entry change in
the map.

**JSX/TSX kept on the React (sandpack) bucket.** They're React component
sources; the existing live-preview should win over the static CODE
bucket. Plain `.js`/`.ts` source goes through CODE.

**MIME-type fallback.** The codeapi backend serves `text/x-python`,
`text/x-typescript`, etc. as `Content-Type` for source files, so a
file whose extension was stripped/renamed upstream still routes to
CODE via the MIME map.

**Empty-text gate.** CODE joins MARKDOWN/PLAIN_TEXT in the empty-text
exception (an empty `.py` is still a Python file). HTML/REACT/MERMAID
still require text — their viewers (sandpack/mermaid.js) error on
empty input.

**Files changed:**
- `client/src/utils/artifacts.ts` — `CODE` bucket constant,
  `CODE_EXTENSION_TO_LANGUAGE` map, exported `isCodeExtension` and
  `languageForFilename` helpers, extension/MIME routing additions,
  template + dependencies entries, empty-text gate exception, helper
  hoisting (extensionOf / baseMime moved up so the language map can
  reference them).
- `client/src/hooks/Artifacts/useArtifactProps.ts` — exported
  `wrapAsFencedCodeBlock`, CODE branch that wraps the source then
  routes through `getMarkdownFiles`.

**Tests (+22):**
- 8 parameterized routing cases (.py, .js, .go, .rs, .css, .sh, .sql,
  .kt) verify the CODE bucket fires.
- Extension wins when MIME is generic octet-stream (Python has no
  magic bytes; common case).
- Regression: jsx/tsx STAY on REACT bucket (no live-preview regression).
- Regression: data formats (CSV/JSON/YAML/TOML) and config dotfiles
  (.env/.ini) do NOT route to CODE.
- Empty-text exception for CODE (empty Python file is still a Python file).
- `useArtifactProps`: CODE → content.md / static template, fenced-block
  shape, language hint, unknown-extension fallback to raw extension,
  no-extension empty hint, index.html via markdown template.
- `wrapAsFencedCodeBlock`: language hint, empty hint, single-trailing-
  newline trim, multi-newline preservation, empty-source emit.

87/87 in artifact-impacted tests; 155/155 across the broader artifact
suite. No regressions in pre-existing markdown/mermaid/HTML/REACT/text
behavior.

* 🛡️ fix: Bare-filename routing + adaptive fence delimiter (codex P2 ×2)

Two follow-ups from Codex review on the CODE bucket:

1. **Bare-filename routing for extensionless build files (Codex P2).**
   `Dockerfile`, `Makefile`, `Gemfile`, `Rakefile`, `Vagrantfile`,
   `Brewfile` have no `.` in their basename — `extensionOf` returns
   `''` and the extension map can't match, so they fell through to
   inline rendering despite being in `CODE_EXTENSION_TO_LANGUAGE`.

   New `bareNameOf` helper returns the lowercased basename for
   extensionless filenames (returns `''` for files with a `.` so the
   extension and bare-name paths don't double-match). Both
   `detectArtifactTypeFromFile` and `languageForFilename` consult it as
   a second lookup against the same `CODE_EXTENSION_TO_LANGUAGE` map,
   so adding a new build file is one entry. Path-aware: takes the
   basename so `proj/Dockerfile` (path-preserving sanitizer output)
   still routes correctly.

   Added the four extra Ruby build-script names while I was here.

2. **Adaptive fence delimiter (Codex P2).** A hardcoded ` ``` ` fence
   breaks when the source contains a line starting with ` ``` ` —
   for example, a JS file containing a markdown-shaped template
   literal:
       const md = `
       ```
       hello
       ```
       `;
   CommonMark closes a fence on a line whose backtick run matches-or-
   exceeds the opener, so `marked` would close the outer fence at
   the inner `\`\`\`` and the rest of the file would render as
   markdown — corrupting the artifact and potentially altering
   formatting / links outside ``.

   New `longestLeadingBacktickRun(source)` scans for the longest
   start-of-line backtick run in the payload. Fence length =
   `max(3, longest + 1)` — strictly more than any internal run, so
   `marked` can never close the outer fence early. Only escalates
   when needed; the common case still uses a triple-backtick fence.

   Inline backticks (mid-line) don't count — they're not fence
   delimiters. Only column-zero runs trigger escalation, so e.g.
   a Python file with ` `inline ``` here` ` keeps the 3-fence.

+11 regression tests:
  - 8 parameterized cases: `Dockerfile`/`Makefile`/`Gemfile`/etc. route
    to CODE via bare-name fallback (case-insensitive on basename).
  - Path-aware: `proj/Dockerfile` recognized.
  - No double-match: `dockerfile.dev` (with extension) returns null.
  - Unknown extensionless files (`README`, `LICENSE`) stay null.
  - 4-backtick fence when source has ` ``` ` at start-of-line.
  - 5-backtick fence when source has ` ```` ` at start-of-line.
  - 3-backtick fence (default) for ordinary code.
  - Inline backticks don't escalate.
  - Source starting with backtick run at offset 0.

Plus 6 new `languageForFilename` tests covering bare-name fallback
and path-awareness.

108/108 in artifact-impacted tests (was 87, +21 tests). No regressions.

* 🛡️ fix: Indented fence detection + basename-scoped extensionOf (codex P2/P3)

Two follow-ups from the latest Codex review on the CODE bucket:

1. **Indented backtick runs (Codex P2).** `longestLeadingBacktickRun`
   was scanning `^(`+)` — column 0 only. CommonMark allows fence
   closers to be indented up to 3 spaces, so a JS source containing
   an indented `\`\`\`` (e.g. inside a template literal embedded in a
   class method) would still terminate our outer fence and the
   remainder would render as markdown.

   Updated regex to `^ {0,3}(`+)`. Tabs are not allowed in fence
   indentation (CommonMark expands them to 4 spaces, which is over
   the 3-space limit), so spaces alone suffice. Backticks indented
   4+ spaces are CommonMark "indented code blocks" — they can't
   terminate a fence, so we correctly don't escalate for them.

2. **`extensionOf` path-laden output (Codex P3).** `extensionOf` took
   `lastIndexOf('.')` across the FULL path string, so
   `pkg.v1/Dockerfile` yielded the nonsensical "extension"
   `v1/dockerfile`. `languageForFilename` returned that as the language
   hint (broken `language-v1/dockerfile` class on the fenced block),
   AND the routing's bare-name fallback couldn't fire because the
   extension lookup returned non-empty.

   New `basenameOf` helper strips path separators; `extensionOf` and
   `bareNameOf` both go through it. After the fix:
     - `pkg.v1/Dockerfile` → `extensionOf` returns `''` → `bareNameOf`
       returns `dockerfile` → routes to CODE with correct language.
     - `pkg.v1/main.go` → `extensionOf` returns `go` → routes correctly.
     - `pkg.v1/script.py` → `extensionOf` returns `py` → routes correctly.

+10 regression tests:
  - 5 parameterized cases covering 1-3 space indent at fence lengths
    3, 4, 5 (escalation kicks in correctly).
  - 4-space indent does NOT escalate (CommonMark indented-code-block
    territory; can't close a fence).
  - `pkg.v1/Dockerfile` and `a.b.c/Makefile` route to CODE +
    `languageForFilename` returns `dockerfile`/`makefile`.
  - Dotted-directory files (`pkg.v1/main.go`, `a.b.c/script.py`) still
    route correctly via the basename-scoped extension parse.

118/118 in artifact-impacted tests (was 108, +10 tests). No regressions.

* 🛡️ fix: Comprehensive review polish + MIME-derived language hint (codex P3)

Resolves all 8 valid findings from the comprehensive review and the
follow-up Codex P3 on the same PR. None are user-visible bugs; the set
spans correctness guards, dead-code removal, organization, and test
coverage.

**Comprehensive review #1 — Remove dead `isCodeExtension` export.**
Function was exported with zero callers anywhere in the codebase.

**Comprehensive review #2 — Guard the for-loop against silent overwrites.**
The `for (ext of CODE_EXTENSION_TO_LANGUAGE)` loop blindly assigned
each language extension to the CODE bucket. If a future contributor
added `jsx` or `tsx` to the language map (a natural mistake — they
ARE source code), the loop would silently overwrite the REACT bucket
entries and break the sandpack live-preview with no compile-time or
runtime error. Added `if (ext in EXTENSION_TO_TOOL_ARTIFACT_TYPE) continue`
so explicit map entries always win.

**Comprehensive review #3 — Add `fileToArtifact` end-to-end test for CODE.**
Routing was tested via `detectArtifactTypeFromFile`; full Artifact
construction (id / type / title / content / messageId / language) for
CODE was not. Added 5 new `fileToArtifact` cases.

**Comprehensive review #4 — Move pure utilities out of the hook file.**
`wrapAsFencedCodeBlock` and `longestLeadingBacktickRun` are pure
string transformations with no React dependencies. Moved both to
`utils/artifacts.ts`. Test files updated to import from the new
location.

**Comprehensive review #5 — Correct the MIME-map "mirrors" comment.**
Comment claimed the MIME map mirrored `CODE_EXTENSION_TO_LANGUAGE`, but
covered ~21 of ~60 entries. Reworded to "best-effort COMMON-CASE list,
not an exhaustive mirror" with the rationale (extension routing is
primary; MIME is a stripped-filename fallback).

**Comprehensive review #6 — Drop `lang ? lang : ''` ternary.**
`lang` is typed `string`; the only falsy value is `''`. Removed.
(Replaced via the MIME-fallback rewrite of `wrapAsFencedCodeBlock`,
where `lang` is now used directly without the ternary.)

**Comprehensive review #7 — Avoid double `basenameOf` computation.**
`extensionOf(filename)` and `bareNameOf(filename)` both internally
called `basenameOf` — when the extension lookup missed,
`detectArtifactTypeFromFile` paid for two parses of the same path.
Split into private `extensionFromBasename` / `bareNameFromBasename`
helpers; the caller computes `basenameOf` once and threads it through.

**Comprehensive review #8 — Trim verbose Dockerfile/Makefile comment.**
Inline comment block in the language map duplicated `bareNameOf`'s
JSDoc. Replaced with a one-line pointer.

**Codex P3 — MIME fallback for the CODE language hint.**
`detectArtifactTypeFromFile` routes `{ filename: 'noext', type:
'text/x-python' }` to CODE via the MIME bucket map, but then
`useArtifactProps` derived the language hint from `artifact.title`
ONLY — and `noext` has no extension, so `languageForFilename` returned
empty and the fenced block emitted with no `language-` class. The
future highlighter swap-in would lose syntax-color metadata for these
files.

  - New `MIME_TO_LANGUAGE` map covering the language MIMEs codeapi
    actually emits.
  - `languageForFilename(filename, mime?)` now takes an optional MIME
    second arg and falls back to it after the extension and bare-name
    paths.
  - `fileToArtifact` resolves the language at construction time
    (using both filename AND `attachment.type`) and stores it on
    `artifact.language`. The hook reads `artifact.language` directly
    rather than re-deriving from `title` alone, so the MIME signal
    survives end-to-end.
  - Title-derived fallback in the hook covers older callers that
    don't populate `language`.

Tests:
  +10 cases for the comprehensive review findings (CODE end-to-end
  via `fileToArtifact`, language storage, non-CODE language
  un-set).
  +6 cases for the MIME fallback (`languageForFilename(name, mime)`
  ordering, MIME parameter stripping, extension/bare-name vs MIME
  precedence, empty signal).
  +2 hook tests for `artifact.language` pre-resolved vs title-fallback.

131/131 in directly-impacted files (was 118, +13).
199/199 across the broader artifact suite. No regressions.

Pre-existing TypeScript errors in `a11y/`, `Agents/`, `Auth/`,
`Mermaid.tsx`, etc. are unrelated to this PR (verified by checking
`tsc --noEmit` on origin/dev — same errors).
---
 .../__tests__/useArtifactProps.test.ts        | 203 +++++++++
 .../src/hooks/Artifacts/useArtifactProps.ts   |  27 +-
 client/src/utils/__tests__/artifacts.test.ts  | 336 ++++++++++++++
 client/src/utils/artifacts.ts                 | 413 +++++++++++++++++-
 4 files changed, 953 insertions(+), 26 deletions(-)

diff --git a/client/src/hooks/Artifacts/__tests__/useArtifactProps.test.ts b/client/src/hooks/Artifacts/__tests__/useArtifactProps.test.ts
index 5ffd52879f..1f71e4c2c0 100644
--- a/client/src/hooks/Artifacts/__tests__/useArtifactProps.test.ts
+++ b/client/src/hooks/Artifacts/__tests__/useArtifactProps.test.ts
@@ -1,5 +1,6 @@
 import { renderHook } from '@testing-library/react';
 import useArtifactProps from '../useArtifactProps';
+import { TOOL_ARTIFACT_TYPES, wrapAsFencedCodeBlock } from '~/utils/artifacts';
 import type { Artifact } from '~/common';
 
 describe('useArtifactProps', () => {
@@ -207,4 +208,206 @@ describe('useArtifactProps', () => {
       expect(result.current.template).toBe('static');
     });
   });
+
+  describe('CODE artifacts', () => {
+    /* The CODE bucket reuses the static markdown pipeline — `marked`
+     * renders the wrapped fenced block as `
…`.
+     * These tests pin the wrap-then-render shape end-to-end so a future
+     * highlighter swap-in or CDN bump can't silently break the panel
+     * for code files. */
+    it('routes CODE artifacts to content.md / static template (markdown pipeline)', () => {
+      const artifact = createArtifact({
+        type: TOOL_ARTIFACT_TYPES.CODE,
+        title: 'simple_graph.py',
+        content: 'import matplotlib\nplt.savefig("foo.png")',
+      });
+      const { result } = renderHook(() => useArtifactProps({ artifact }));
+      expect(result.current.fileKey).toBe('content.md');
+      expect(result.current.template).toBe('static');
+    });
+
+    it('wraps the source as a fenced block with the language hint from the filename', () => {
+      const artifact = createArtifact({
+        type: TOOL_ARTIFACT_TYPES.CODE,
+        title: 'simple_graph.py',
+        content: 'x = 1\nprint(x)',
+      });
+      const { result } = renderHook(() => useArtifactProps({ artifact }));
+      const md = result.current.files['content.md'] as string;
+      /* Marked outputs `
` from this fence. */
+      expect(md.startsWith('```python\n')).toBe(true);
+      expect(md.endsWith('\n```')).toBe(true);
+      expect(md).toContain('x = 1');
+      expect(md).toContain('print(x)');
+    });
+
+    it('falls back to the raw extension as language hint for unknown extensions', () => {
+      const artifact = createArtifact({
+        type: TOOL_ARTIFACT_TYPES.CODE,
+        title: 'thing.qwerty',
+        content: 'data',
+      });
+      const { result } = renderHook(() => useArtifactProps({ artifact }));
+      const md = result.current.files['content.md'] as string;
+      expect(md.startsWith('```qwerty\n')).toBe(true);
+    });
+
+    it('uses an empty language hint when the title has no extension', () => {
+      const artifact = createArtifact({
+        type: TOOL_ARTIFACT_TYPES.CODE,
+        title: 'Makefile-style',
+        content: 'all: build',
+      });
+      const { result } = renderHook(() => useArtifactProps({ artifact }));
+      const md = result.current.files['content.md'] as string;
+      /* Empty hint = bare ` ``` ` opener (no language token). */
+      expect(md.startsWith('```\n')).toBe(true);
+    });
+
+    it('renders an index.html via the markdown template (visual parity with .md artifacts)', () => {
+      const artifact = createArtifact({
+        type: TOOL_ARTIFACT_TYPES.CODE,
+        title: 'app.js',
+        content: 'console.log("hi");',
+      });
+      const { result } = renderHook(() => useArtifactProps({ artifact }));
+      const html = result.current.files['index.html'] as string;
+      expect(html).toContain('');
+      expect(html).toContain('marked.parse');
+    });
+
+    /* Codex review P3: when `fileToArtifact` resolved a language via
+     * MIME fallback (extensionless title, useful MIME), the resolved
+     * value is stored on `artifact.language`. The hook reads it
+     * directly so the fence still gets the right `language-` class. */
+    it('uses pre-resolved artifact.language (covers MIME fallback for extensionless titles)', () => {
+      const artifact = createArtifact({
+        type: TOOL_ARTIFACT_TYPES.CODE,
+        title: 'noext',
+        language: 'python',
+        content: 'print(1)',
+      });
+      const { result } = renderHook(() => useArtifactProps({ artifact }));
+      const md = result.current.files['content.md'] as string;
+      expect(md.startsWith('```python\n')).toBe(true);
+    });
+
+    it('falls back to title-derived language when artifact.language is unset', () => {
+      /* Older callers that don't populate `language` still get a
+       * sensible fence via title-based lookup. */
+      const artifact = createArtifact({
+        type: TOOL_ARTIFACT_TYPES.CODE,
+        title: 'app.js',
+        content: 'console.log("hi");',
+      });
+      const { result } = renderHook(() => useArtifactProps({ artifact }));
+      const md = result.current.files['content.md'] as string;
+      expect(md.startsWith('```javascript\n')).toBe(true);
+    });
+  });
+});
+
+describe('wrapAsFencedCodeBlock', () => {
+  it('wraps source with the supplied language hint', () => {
+    expect(wrapAsFencedCodeBlock('print(1)', 'python')).toBe('```python\nprint(1)\n```');
+  });
+
+  it('emits a bare ``` opener when language is empty', () => {
+    expect(wrapAsFencedCodeBlock('hello', '')).toBe('```\nhello\n```');
+  });
+
+  it('trims a single trailing newline so the closing fence stays flush', () => {
+    /* Extractor output often ends with a newline; without trimming, the
+     * closing ``` would appear on its own line after a blank gap and
+     * marked would render an extra 
. */ + expect(wrapAsFencedCodeBlock('a\nb\n', 'go')).toBe('```go\na\nb\n```'); + /* Only ONE trailing newline gets trimmed — preserves explicit blank + * lines at end-of-file in the output. */ + expect(wrapAsFencedCodeBlock('a\nb\n\n', 'go')).toBe('```go\na\nb\n\n```'); + }); + + it('handles empty source cleanly (renders an empty fenced block)', () => { + expect(wrapAsFencedCodeBlock('', 'python')).toBe('```python\n\n```'); + }); + + /* Codex review P2: a hardcoded triple-backtick fence breaks when the + * source itself contains a line starting with ``` (e.g. a JS template + * literal embedding markdown). CommonMark closes the outer fence on + * a line whose backtick run matches-or-exceeds the opener, so we have + * to emit STRICTLY MORE backticks than any leading-backtick run in + * the payload. */ + it('uses a 4-backtick fence when source has a triple-backtick line at column 0', () => { + const src = 'const md = `\n```\nhello\n```\n`;'; + const wrapped = wrapAsFencedCodeBlock(src, 'js'); + expect(wrapped.startsWith('````js\n')).toBe(true); + expect(wrapped.endsWith('\n````')).toBe(true); + /* The payload's ``` lines are preserved verbatim — no escaping. */ + expect(wrapped).toContain('\n```\nhello\n```\n'); + }); + + it('uses a 5-backtick fence when source has a quadruple-backtick line', () => { + const src = 'before\n````\ninside\n````\nafter'; + const wrapped = wrapAsFencedCodeBlock(src, 'md'); + expect(wrapped.startsWith('`````md\n')).toBe(true); + expect(wrapped.endsWith('\n`````')).toBe(true); + }); + + it('keeps the 3-backtick fence when source has no leading-backtick lines', () => { + /* Ordinary code (no markdown-style fences) gets the conventional + * triple-backtick fence — readable and matches `marked`'s default + * expectations. */ + const wrapped = wrapAsFencedCodeBlock('x = 1\nprint(x)', 'python'); + expect(wrapped.startsWith('```python\n')).toBe(true); + expect(wrapped.endsWith('\n```')).toBe(true); + }); + + it('does not escalate the fence for backticks that are NOT at start-of-line', () => { + /* Inline backticks within a line don't count against the closing- + * fence rule, so they don't need escalation. Keeps the fence + * minimal for the common case (e.g. a Python file that uses + * markdown ` `code` ` in a docstring). */ + const src = 'x = `inline backticks ``` here`'; + const wrapped = wrapAsFencedCodeBlock(src, 'python'); + expect(wrapped.startsWith('```python\n')).toBe(true); + expect(wrapped.endsWith('\n```')).toBe(true); + }); + + it('escalates correctly when the source itself starts with a backtick run', () => { + /* Edge: backticks at column 0 of the very first line. The leading- + * run scan must catch this position (regex anchor allows + * start-of-string match too). */ + const src = '```already-fenced\nbody\n```'; + const wrapped = wrapAsFencedCodeBlock(src, 'md'); + expect(wrapped.startsWith('````md\n')).toBe(true); + }); + + /* Codex review P2 follow-up: CommonMark allows fence closers indented + * up to 3 spaces. The fence-length scan must catch backtick runs at + * columns 0–3, not just column 0 — otherwise an indented `\`\`\`` + * inside a JS/Python source still terminates our outer fence. */ + it.each([ + [' ```', 4], + [' ```', 4], + [' ```', 4], + [' ````', 5], + [' `````', 6], + ])('escalates fence to %s+1 backticks for indented closer "%s"', (line, expectedFenceLen) => { + const src = 'before\n' + line + '\nafter'; + const wrapped = wrapAsFencedCodeBlock(src, 'js'); + /* Fence prefix is `expectedFenceLen` backticks (one more than the + * indented run inside the source). */ + expect(wrapped.startsWith('`'.repeat(expectedFenceLen) + 'js\n')).toBe(true); + expect(wrapped.endsWith('\n' + '`'.repeat(expectedFenceLen))).toBe(true); + }); + + it('does NOT escalate for backticks indented 4+ spaces (CommonMark indented-code-block territory)', () => { + /* 4-space indentation makes the line an indented code block, not a + * fence closer. The CommonMark spec caps fence-closer indentation + * at 3 spaces, so backticks at column 4+ can't terminate an + * unindented opener. */ + const src = 'before\n ```\nafter'; + const wrapped = wrapAsFencedCodeBlock(src, 'js'); + /* No escalation — keeps the default 3-backtick fence. */ + expect(wrapped.startsWith('```js\n')).toBe(true); + }); }); diff --git a/client/src/hooks/Artifacts/useArtifactProps.ts b/client/src/hooks/Artifacts/useArtifactProps.ts index ce5e30cf5a..3a7b17c24a 100644 --- a/client/src/hooks/Artifacts/useArtifactProps.ts +++ b/client/src/hooks/Artifacts/useArtifactProps.ts @@ -2,7 +2,15 @@ import { useContext, useMemo } from 'react'; import { ThemeContext, isDark } from '@librechat/client'; import { removeNullishValues } from 'librechat-data-provider'; import type { Artifact } from '~/common'; -import { getKey, getProps, getTemplate, getArtifactFilename } from '~/utils/artifacts'; +import { + getKey, + getProps, + getTemplate, + getArtifactFilename, + languageForFilename, + wrapAsFencedCodeBlock, + TOOL_ARTIFACT_TYPES, +} from '~/utils/artifacts'; import { getMarkdownFiles } from '~/utils/markdown'; import { getMermaidFiles } from '~/utils/mermaid'; @@ -18,6 +26,21 @@ export default function useArtifactProps({ artifact }: { artifact: Artifact }) { return ['diagram.mmd', getMermaidFiles(artifact.content ?? '', isDarkMode)]; } + /* CODE bucket: source files render through the same static-markdown + * pipeline as .md artifacts, but the raw source is wrapped in a + * fenced code block first so `marked` outputs + * `
` instead of + * paragraph text. The language hint is computed once at artifact + * construction (`fileToArtifact` writes it to `artifact.language`) + * so the MIME fallback fires for extensionless-filename + useful- + * MIME inputs. Title-derived fallback covers older callers that + * didn't set `language`. */ + if (type === TOOL_ARTIFACT_TYPES.CODE) { + const lang = artifact.language ?? languageForFilename(artifact.title); + const wrapped = wrapAsFencedCodeBlock(artifact.content ?? '', lang); + return ['content.md', getMarkdownFiles(wrapped)]; + } + if (type === 'text/markdown' || type === 'text/md' || type === 'text/plain') { return ['content.md', getMarkdownFiles(artifact.content ?? '')]; } @@ -27,7 +50,7 @@ export default function useArtifactProps({ artifact }: { artifact: Artifact }) { [fileKey]: artifact.content, }); return [fileKey, files]; - }, [artifact.type, artifact.content, artifact.language, isDarkMode]); + }, [artifact.type, artifact.content, artifact.language, artifact.title, isDarkMode]); const template = useMemo( () => getTemplate(artifact.type ?? '', artifact.language), diff --git a/client/src/utils/__tests__/artifacts.test.ts b/client/src/utils/__tests__/artifacts.test.ts index 2d1216995c..02e7494c15 100644 --- a/client/src/utils/__tests__/artifacts.test.ts +++ b/client/src/utils/__tests__/artifacts.test.ts @@ -2,6 +2,7 @@ import { buildSandpackOptions, detectArtifactTypeFromFile, fileToArtifact, + languageForFilename, TOOL_ARTIFACT_TYPES, } from '../artifacts'; @@ -132,6 +133,246 @@ describe('detectArtifactTypeFromFile', () => { detectArtifactTypeFromFile({ filename: '.env', type: 'text/plain', text: 'KEY=value' }), ).toBeNull(); }); + + describe('CODE bucket (programming-language source files)', () => { + /* `.py` and other code files were previously inline-only — PR #12832 + * intentionally left them out of the side-panel pipeline. This bucket + * routes them through the markdown template with the source pre- + * wrapped as a fenced code block (`useArtifactProps`). */ + it.each([ + ['simple_graph.py', 'text/x-python'], + ['app.js', 'text/javascript'], + ['main.go', 'text/x-go'], + ['lib.rs', 'text/x-rust'], + ['style.css', 'text/css'], + ['build.sh', 'application/x-sh'], + ['query.sql', 'application/sql'], + ['Module.kt', 'text/x-kotlin'], + ])('routes %s (mime: %s) to the CODE bucket', (filename, type) => { + expect(detectArtifactTypeFromFile({ filename, type, text: 'x = 1' })).toBe( + TOOL_ARTIFACT_TYPES.CODE, + ); + }); + + it('routes by extension even when MIME is generic octet-stream', () => { + /* file-type / inferMimeType sometimes can't classify code files + * (Python has no magic bytes); the extension map still wins. */ + expect( + detectArtifactTypeFromFile({ + filename: 'data.py', + type: 'application/octet-stream', + text: 'print(1)', + }), + ).toBe(TOOL_ARTIFACT_TYPES.CODE); + }); + + it('keeps jsx/tsx on the React (sandpack) bucket, not CODE', () => { + /* `.jsx` and `.tsx` are React component sources — the existing + * sandpack live-preview should win over the static CODE bucket. */ + expect(detectArtifactTypeFromFile({ filename: 'App.jsx', type: '', text: 'x' })).toBe( + TOOL_ARTIFACT_TYPES.REACT, + ); + expect(detectArtifactTypeFromFile({ filename: 'App.tsx', type: '', text: 'x' })).toBe( + TOOL_ARTIFACT_TYPES.REACT, + ); + }); + + it('does NOT route data formats to CODE (CSV / JSON / YAML / TOML / XML)', () => { + /* These get dedicated viewers in follow-ups; for now they fall + * through to inline rendering (return null). */ + expect( + detectArtifactTypeFromFile({ filename: 'data.csv', type: 'text/csv', text: 'a,b' }), + ).toBeNull(); + expect( + detectArtifactTypeFromFile({ filename: 'data.json', type: 'application/json', text: '{}' }), + ).toBeNull(); + expect( + detectArtifactTypeFromFile({ + filename: 'config.yaml', + type: 'application/yaml', + text: 'a: 1', + }), + ).toBeNull(); + expect( + detectArtifactTypeFromFile({ + filename: 'pyproject.toml', + type: 'application/toml', + text: '', + }), + ).toBeNull(); + }); + + it('does NOT route config dotfiles to CODE (.env / .ini)', () => { + expect( + detectArtifactTypeFromFile({ filename: 'app.env', type: 'text/plain', text: 'KEY=val' }), + ).toBeNull(); + expect( + detectArtifactTypeFromFile({ + filename: 'config.ini', + type: 'text/plain', + text: '[section]', + }), + ).toBeNull(); + }); + + it('allows empty text for CODE files (an empty Python file is still a Python file)', () => { + expect( + detectArtifactTypeFromFile({ filename: 'empty.py', type: 'text/x-python', text: '' }), + ).toBe(TOOL_ARTIFACT_TYPES.CODE); + }); + + /* Codex review P2: extensionless build files like `Dockerfile` and + * `Makefile` have no `.` in their basename, so `extensionOf` returns + * `''` and the extension map can't match. Bare-name fallback + * recognizes the lowercased basename for these cases. */ + it.each([ + 'Dockerfile', + 'dockerfile', + 'Makefile', + 'makefile', + 'Gemfile', + 'Rakefile', + 'Vagrantfile', + 'Brewfile', + ])('routes extensionless build file %s to CODE via bare-name fallback', (filename) => { + expect(detectArtifactTypeFromFile({ filename, type: '', text: 'FROM alpine' })).toBe( + TOOL_ARTIFACT_TYPES.CODE, + ); + }); + + it('still recognizes nested-path Dockerfile (path-preserving sanitizer output)', () => { + /* The path-preserving artifact sanitizer can ship `proj/Dockerfile`. + * Bare-name lookup must use the basename, not the full string. */ + expect( + detectArtifactTypeFromFile({ filename: 'proj/Dockerfile', type: '', text: 'FROM alpine' }), + ).toBe(TOOL_ARTIFACT_TYPES.CODE); + }); + + it('does not bare-name match files that DO have an extension (no double-match)', () => { + /* `dockerfile.dev` has extension `dev` (not in the routing map), + * so it returns null. Bare-name lookup must skip files with a + * `.` so the extension path stays the source of truth for them. */ + expect( + detectArtifactTypeFromFile({ filename: 'dockerfile.dev', type: '', text: 'x' }), + ).toBeNull(); + }); + + it('does not bare-name match unknown extensionless filenames', () => { + expect(detectArtifactTypeFromFile({ filename: 'README', type: '', text: 'hi' })).toBeNull(); + expect(detectArtifactTypeFromFile({ filename: 'LICENSE', type: '', text: 'MIT' })).toBeNull(); + }); + + /* Codex review P3 companion: `extensionOf` used to consider the + * whole path string, so `pkg.v1/Dockerfile` yielded a path-laden + * "extension" that masked the bare-name fallback. The basename- + * first fix makes routing for these files work correctly. */ + it('routes nested-path Dockerfile under dotted directory to CODE', () => { + expect( + detectArtifactTypeFromFile({ filename: 'pkg.v1/Dockerfile', type: '', text: 'FROM x' }), + ).toBe(TOOL_ARTIFACT_TYPES.CODE); + }); + + it('still routes file extensions correctly under dotted directory', () => { + expect( + detectArtifactTypeFromFile({ filename: 'pkg.v1/main.go', type: '', text: 'package main' }), + ).toBe(TOOL_ARTIFACT_TYPES.CODE); + expect( + detectArtifactTypeFromFile({ + filename: 'a.b.c/script.py', + type: 'text/x-python', + text: 'x = 1', + }), + ).toBe(TOOL_ARTIFACT_TYPES.CODE); + }); + }); +}); + +describe('languageForFilename', () => { + it('returns the canonical language identifier for known extensions', () => { + expect(languageForFilename('foo.py')).toBe('python'); + expect(languageForFilename('foo.ts')).toBe('typescript'); + expect(languageForFilename('foo.go')).toBe('go'); + expect(languageForFilename('foo.rs')).toBe('rust'); + expect(languageForFilename('foo.kt')).toBe('kotlin'); + }); + + it('falls back to the raw extension for unknown ones (renders monospace)', () => { + expect(languageForFilename('foo.qwerty')).toBe('qwerty'); + }); + + it('returns the canonical language for extensionless build files (bare-name fallback)', () => { + /* Codex review P2 companion: language hint must follow the same + * bare-name fallback as the routing decision so the fenced block + * gets `language-dockerfile` / `language-makefile` etc. */ + expect(languageForFilename('Dockerfile')).toBe('dockerfile'); + expect(languageForFilename('Makefile')).toBe('makefile'); + expect(languageForFilename('Gemfile')).toBe('ruby'); + expect(languageForFilename('Rakefile')).toBe('ruby'); + }); + + it('handles nested-path filenames (uses basename)', () => { + expect(languageForFilename('proj/Dockerfile')).toBe('dockerfile'); + expect(languageForFilename('a/b/c.py')).toBe('python'); + }); + + it('returns empty string for filenames with no extension and no recognized bare name', () => { + expect(languageForFilename('README')).toBe(''); + expect(languageForFilename('')).toBe(''); + expect(languageForFilename(undefined)).toBe(''); + }); + + /* Codex review P3: `extensionOf` previously took `lastIndexOf('.')` + * across the FULL path, so `pkg.v1/Dockerfile` yielded the + * nonsensical "extension" `v1/dockerfile`. Since that's non-empty, + * `languageForFilename` returned it as the language hint instead of + * falling back to `bareNameOf`. The basename-first fix makes both + * helpers operate on the basename only. */ + it('correctly falls back to bare-name when path has dotted directory components', () => { + expect(languageForFilename('pkg.v1/Dockerfile')).toBe('dockerfile'); + expect(languageForFilename('a.b.c/Makefile')).toBe('makefile'); + expect(languageForFilename('proj.beta/Gemfile')).toBe('ruby'); + }); + + it('correctly identifies extension when path has dotted directory components', () => { + /* Dotted dir + dotted file: extension parsing should still find + * the file's extension, not concatenate dir+file fragments. */ + expect(languageForFilename('pkg.v1/main.go')).toBe('go'); + expect(languageForFilename('a.b.c/script.py')).toBe('python'); + }); + + /* Codex review P3: when a file routes to CODE via MIME-only (e.g. + * `noext` filename + `text/x-python` MIME), we still want a language + * hint on the fenced block so the future highlighter swap-in can + * apply syntax colors. Without the MIME fallback, `language-` is + * empty and the highlighter can't engage. */ + it('falls back to MIME when filename has no extension and no recognized bare name', () => { + expect(languageForFilename('noext', 'text/x-python')).toBe('python'); + expect(languageForFilename('noext', 'text/x-go')).toBe('go'); + expect(languageForFilename('noext', 'application/x-sh')).toBe('bash'); + expect(languageForFilename(undefined, 'text/x-rust')).toBe('rust'); + }); + + it('strips MIME parameters before lookup (charset, etc.)', () => { + expect(languageForFilename('noext', 'text/x-python; charset=utf-8')).toBe('python'); + expect(languageForFilename('noext', 'TEXT/X-PYTHON;charset=utf-8')).toBe('python'); + }); + + it('prefers extension over MIME when both are present (extension is more reliable)', () => { + /* `simple_graph.py` + a wrong/generic MIME → extension wins. */ + expect(languageForFilename('simple_graph.py', 'application/octet-stream')).toBe('python'); + expect(languageForFilename('main.go', 'text/x-python')).toBe('go'); + }); + + it('prefers bare-name over MIME for build files', () => { + /* `Dockerfile` + a generic MIME → bare-name wins. */ + expect(languageForFilename('Dockerfile', 'text/plain')).toBe('dockerfile'); + }); + + it('returns empty string when no signal yields a hint (extensionless + unknown MIME)', () => { + expect(languageForFilename('noext', 'application/octet-stream')).toBe(''); + expect(languageForFilename('noext', undefined)).toBe(''); + expect(languageForFilename('noext')).toBe(''); + }); }); describe('fileToArtifact', () => { @@ -159,6 +400,101 @@ describe('fileToArtifact', () => { expect(fileToArtifact({ ...baseFile, filename: 'data.csv', type: 'text/csv' })).toBeNull(); }); + /* End-to-end test for the CODE bucket. The classification path is + * covered separately in `detectArtifactTypeFromFile`'s describe block; + * this asserts that the full `Artifact` object (id / type / title / + * content / messageId / lastUpdateTime) is constructed correctly for + * a typical Python file. Locks in the empty-text gate exception for + * CODE and the title pass-through that `useArtifactProps` reads to + * derive the language hint. */ + it('builds a CODE-typed Artifact for a .py file with text', () => { + const artifact = fileToArtifact({ + ...baseFile, + filename: 'simple_graph.py', + type: 'text/x-python', + text: 'import matplotlib.pyplot as plt\nplt.savefig("foo.png")', + }); + expect(artifact).not.toBeNull(); + expect(artifact!.type).toBe(TOOL_ARTIFACT_TYPES.CODE); + expect(artifact!.title).toBe('simple_graph.py'); + expect(artifact!.content).toBe('import matplotlib.pyplot as plt\nplt.savefig("foo.png")'); + expect(artifact!.id).toBe('tool-artifact-fid-1'); + expect(artifact!.messageId).toBe('msg-1'); + }); + + it('builds a CODE-typed Artifact for an empty .py file (empty-text exception applies)', () => { + /* CODE joins MARKDOWN/PLAIN_TEXT in the empty-text exception so an + * empty Python file still surfaces in the side panel rather than + * silently disappearing. */ + const artifact = fileToArtifact({ + ...baseFile, + filename: 'empty.py', + type: 'text/x-python', + text: '', + }); + expect(artifact).not.toBeNull(); + expect(artifact!.type).toBe(TOOL_ARTIFACT_TYPES.CODE); + expect(artifact!.content).toBe(''); + }); + + it('builds a CODE-typed Artifact for an extensionless build file (Dockerfile)', () => { + const artifact = fileToArtifact({ + ...baseFile, + filename: 'Dockerfile', + type: '', + text: 'FROM alpine\nRUN apk add curl', + }); + expect(artifact).not.toBeNull(); + expect(artifact!.type).toBe(TOOL_ARTIFACT_TYPES.CODE); + expect(artifact!.title).toBe('Dockerfile'); + /* Bare-name resolved → `dockerfile` language hint stored on the + * artifact so `useArtifactProps` doesn't have to re-derive it. */ + expect(artifact!.language).toBe('dockerfile'); + }); + + /* Codex review P3: language is resolved AT CONSTRUCTION TIME so the + * MIME fallback fires for extensionless filenames. Without storing + * the language on the artifact, `useArtifactProps` would re-derive + * from `artifact.title` alone (which has no MIME context) and emit + * an empty `language-` class. */ + it('stores the language hint on CODE artifacts (filename-derived)', () => { + const artifact = fileToArtifact({ + ...baseFile, + filename: 'app.py', + type: 'text/x-python', + text: 'print(1)', + }); + expect(artifact!.language).toBe('python'); + }); + + it('stores the MIME-derived language on CODE artifacts when filename has no extension', () => { + const artifact = fileToArtifact({ + ...baseFile, + filename: 'noext', + type: 'text/x-python', + text: 'print(1)', + }); + expect(artifact).not.toBeNull(); + expect(artifact!.type).toBe(TOOL_ARTIFACT_TYPES.CODE); + expect(artifact!.language).toBe('python'); + }); + + it('does not set language on non-CODE artifacts', () => { + /* Markdown / HTML / etc. don't need a language hint — `useArtifactProps` + * uses different rendering paths for those. Keeping `language` + * undefined for them avoids confusing `getKey` which does include + * `language` in its cache key. */ + const html = fileToArtifact(baseFile); + expect(html!.language).toBeUndefined(); + const md = fileToArtifact({ + ...baseFile, + filename: 'README.md', + type: 'text/markdown', + text: '# hi', + }); + expect(md!.language).toBeUndefined(); + }); + it('returns null when an HTML/React/Mermaid file has no text', () => { expect(fileToArtifact({ ...baseFile, text: '' })).toBeNull(); expect(fileToArtifact({ ...baseFile, filename: 'App.tsx', type: '', text: '' })).toBeNull(); diff --git a/client/src/utils/artifacts.ts b/client/src/utils/artifacts.ts index 88fe1f688a..3a17c843b2 100644 --- a/client/src/utils/artifacts.ts +++ b/client/src/utils/artifacts.ts @@ -24,6 +24,7 @@ const artifactFilename = { const artifactTemplate: Record< | keyof typeof artifactFilename | 'application/vnd.mermaid' + | 'application/vnd.code' | 'text/markdown' | 'text/md' | 'text/plain', @@ -34,6 +35,12 @@ const artifactTemplate: Record< 'application/vnd.ant.react': 'react-ts', 'application/vnd.mermaid': 'react-ts', 'application/vnd.code-html': 'static', + /* CODE bucket reuses the static markdown pipeline — `useArtifactProps` + * pre-wraps the content in a fenced block and hands it to + * `getMarkdownFiles`, so the rendered HTML uses the same `marked` + * pipeline as `.md` artifacts. Keeping `static` (vs. `react-ts`) means + * the panel doesn't pay the sandpack-React boot cost for source files. */ + 'application/vnd.code': 'static', 'text/markdown': 'static', 'text/md': 'static', 'text/plain': 'static', @@ -112,6 +119,7 @@ const mermaidDependencies = { const dependenciesMap: Record< | keyof typeof artifactFilename | 'application/vnd.mermaid' + | 'application/vnd.code' | 'text/markdown' | 'text/md' | 'text/plain', @@ -122,6 +130,10 @@ const dependenciesMap: Record< 'application/vnd.ant.react': standardDependencies, 'text/html': standardDependencies, 'application/vnd.code-html': standardDependencies, + /* CODE renders in the static markdown template; no React or other + * runtime deps. Empty map skips the sandpack `package.json` install + * step entirely (same as MARKDOWN/PLAIN_TEXT). */ + 'application/vnd.code': {}, 'text/markdown': {}, 'text/md': {}, 'text/plain': {}, @@ -163,12 +175,88 @@ export function buildSandpackOptions( }; } +/** + * Strip path separators so extension/bare-name lookups operate on the + * basename only. Artifact filenames can carry nested directories + * through the path-preserving sanitizer in the backend, and a dotted + * directory name (e.g. `pkg.v1/Dockerfile`) would otherwise produce a + * nonsensical "extension" like `v1/dockerfile`. + */ +const basenameOf = (filename: string): string => { + const slash = Math.max(filename.lastIndexOf('/'), filename.lastIndexOf('\\')); + return slash >= 0 ? filename.slice(slash + 1) : filename; +}; + +/** + * Internal: derive the lowercased extension from a pre-computed basename. + * Returns `''` for hidden-files (`.env` → `env` in legacy code, but we + * deliberately preserve that for backwards compatibility with the + * extension-list lookup; the bare-name fallback handles the + * extensionless case separately). The shared variant lets + * `detectArtifactTypeFromFile` compute the basename once and reuse it + * across the extension AND bare-name lookups. + */ +const extensionFromBasename = (base: string): string => { + const dot = base.lastIndexOf('.'); + if (dot < 0 || dot === base.length - 1) { + return ''; + } + return base.slice(dot + 1).toLowerCase(); +}; + +/** + * Internal: derive the lowercased bare name for extensionless filenames + * from a pre-computed basename. Returns `''` for files that DO have an + * extension so `extensionOf` and `bareNameOf` are mutually exclusive. + */ +const bareNameFromBasename = (base: string): string => { + if (base.includes('.')) { + return ''; + } + return base.toLowerCase(); +}; + +/** + * Restrict the dot search to the basename — `pkg.v1/Dockerfile` should + * yield `''` (extensionless), not `v1/dockerfile`. Otherwise + * `languageForFilename` returns the path-laden string as the language + * hint and `marked` emits a `language-v1/dockerfile` class (broken). + * The bare-name fallback then can't fire because this returned + * non-empty. + */ +const extensionOf = (filename: string | undefined): string => { + if (!filename) return ''; + return extensionFromBasename(basenameOf(filename)); +}; + +/** + * Lowercased basename for extensionless filenames. Lets the routing map + * recognize `Dockerfile`, `Makefile`, `Gemfile`, etc. as code without + * a dotted extension. Returns `''` for files that DO have an extension + * (those go through `extensionOf`) so the two helpers don't double-match. + */ +const bareNameOf = (filename: string | undefined): string => { + if (!filename) return ''; + return bareNameFromBasename(basenameOf(filename)); +}; + +/** Strip charset / boundary parameters so we can do exact MIME comparisons. */ +const baseMime = (mime: string | undefined): string => { + if (!mime) { + return ''; + } + const semi = mime.indexOf(';'); + return (semi < 0 ? mime : mime.slice(0, semi)).trim().toLowerCase(); +}; + /** * Artifact MIME types we currently know how to render in the side panel * (or, for mermaid, inline). Plain text covers files we can show as raw * content even without a dedicated viewer (txt, docx-extracted text, …); * `useArtifactProps` routes `text/plain` through the markdown template - * so the panel renders them cleanly. + * so the panel renders them cleanly. `CODE` is the same idea for source + * files — `useArtifactProps` wraps the content in a fenced code block + * with a language hint before handing it to the markdown viewer. */ export const TOOL_ARTIFACT_TYPES = { HTML: 'text/html', @@ -176,13 +264,236 @@ export const TOOL_ARTIFACT_TYPES = { MARKDOWN: 'text/markdown', MERMAID: 'application/vnd.mermaid', PLAIN_TEXT: 'text/plain', + CODE: 'application/vnd.code', } as const; export type ToolArtifactType = (typeof TOOL_ARTIFACT_TYPES)[keyof typeof TOOL_ARTIFACT_TYPES]; +/** + * Extension → fenced-code-block language hint for the CODE bucket. The + * key is the lowercased file extension (no dot); the value is the + * identifier `marked` reads off the fence (e.g. ```` ```python ```` ). + * The map drives BOTH the `EXTENSION_TO_TOOL_ARTIFACT_TYPE` routing + * (presence in this map = code file) and the fenced-block emit in + * `useArtifactProps`, so adding a new language is one place. + * + * Identifiers follow the GitHub / `highlight.js` convention so a future + * highlighter swap-in (currently the markdown template uses plain + * `marked`) picks up syntax colors automatically. + * + * Scope: programming languages + stylesheets + shell/SQL/build files. + * Pure data formats (CSV/TSV/JSON/YAML/TOML/XML) and config files + * (`.env`/`.ini`/`.conf`) are intentionally NOT routed to CODE in this + * pass — they're better served by dedicated viewers (CSV table view, + * etc.) or remain inline. Adding them later is a one-entry change. + */ +const CODE_EXTENSION_TO_LANGUAGE: Record = { + // Web / scripting + js: 'javascript', + mjs: 'javascript', + cjs: 'javascript', + ts: 'typescript', + py: 'python', + pyi: 'python', + rb: 'ruby', + php: 'php', + pl: 'perl', + pm: 'perl', + lua: 'lua', + // Compiled / systems + go: 'go', + rs: 'rust', + c: 'c', + h: 'c', + cc: 'cpp', + cpp: 'cpp', + hpp: 'cpp', + cs: 'csharp', + m: 'objectivec', + mm: 'objectivec', + swift: 'swift', + java: 'java', + kt: 'kotlin', + kts: 'kotlin', + scala: 'scala', + // Functional / data + r: 'r', + jl: 'julia', + dart: 'dart', + ex: 'elixir', + exs: 'elixir', + erl: 'erlang', + hs: 'haskell', + clj: 'clojure', + cljs: 'clojure', + fs: 'fsharp', + fsx: 'fsharp', + // Shell + sh: 'bash', + bash: 'bash', + zsh: 'bash', + fish: 'bash', + ps1: 'powershell', + bat: 'batch', + cmd: 'batch', + // Build / query languages. The bare-name `Dockerfile` / `Makefile` / + // etc. cases are documented on `bareNameOf`. + sql: 'sql', + graphql: 'graphql', + gql: 'graphql', + proto: 'protobuf', + dockerfile: 'dockerfile', + makefile: 'makefile', + gemfile: 'ruby', + rakefile: 'ruby', + vagrantfile: 'ruby', + brewfile: 'ruby', + gradle: 'groovy', + tf: 'hcl', + hcl: 'hcl', + patch: 'diff', + diff: 'diff', + // Stylesheets + css: 'css', + scss: 'scss', + sass: 'sass', + less: 'less', +}; + +/** + * MIME → fenced-block language hint. Used as a fallback for files whose + * filename has no extension AND no recognized bare name — without this, + * an attachment like `{ filename: 'noext', type: 'text/x-python' }` + * would route to CODE (via the MIME bucket map above) but render with + * an empty `language-` class, losing the syntax-hint metadata the + * future highlighter swap-in needs. + * + * Best-effort: covers the language MIMEs the codeapi backend actually + * emits (mirrors `MIME_TO_TOOL_ARTIFACT_TYPE`'s CODE entries). MIMEs + * without an entry here just fall through to the empty hint, same as + * unknown extensions. + */ +const MIME_TO_LANGUAGE: Record = { + 'text/javascript': 'javascript', + 'application/javascript': 'javascript', + 'application/sql': 'sql', + 'application/x-sh': 'bash', + 'application/x-php': 'php', + 'application/x-powershell': 'powershell', + 'text/x-python': 'python', + 'text/x-typescript': 'typescript', + 'text/x-ruby': 'ruby', + 'text/x-go': 'go', + 'text/x-rust': 'rust', + 'text/x-c': 'c', + 'text/x-c++': 'cpp', + 'text/x-csharp': 'csharp', + 'text/x-java': 'java', + 'text/x-kotlin': 'kotlin', + 'text/x-scala': 'scala', + 'text/x-perl': 'perl', + 'text/x-r': 'r', + 'text/x-lua': 'lua', + 'text/x-swift': 'swift', + 'text/css': 'css', +}; + +/** + * Look up the fenced-block language hint for an attachment, trying (in + * order): the filename's extension, the lowercased basename for + * extensionless build files, and the MIME type. Falls back to the raw + * extension if not in the map (so a `.foo` source still gets + * ```` ```foo ```` and renders monospace, even without highlighting). + * Empty string when no signal yields a hint — the fence emits as + * ```` ``` ```` with no language token. + * + * The MIME fallback covers the case where codeapi reports a code file + * with a stripped or extensionless name AND a useful Content-Type + * (e.g. `{ filename: 'noext', type: 'text/x-python' }`). Without it, + * such files would route to CODE but render without `language-python` + * on the `` element. + */ +export function languageForFilename( + filename: string | undefined, + mime?: string | undefined, +): string { + const ext = extensionOf(filename); + if (ext) { + return CODE_EXTENSION_TO_LANGUAGE[ext] ?? ext; + } + /* Extensionless filename: try the basename. `Dockerfile` → + * `dockerfile` → `'dockerfile'` language hint. */ + const bare = bareNameOf(filename); + if (bare && Object.prototype.hasOwnProperty.call(CODE_EXTENSION_TO_LANGUAGE, bare)) { + return CODE_EXTENSION_TO_LANGUAGE[bare]; + } + /* MIME fallback for the extensionless-name + useful-MIME case. */ + if (mime) { + const stripped = baseMime(mime); + if (Object.prototype.hasOwnProperty.call(MIME_TO_LANGUAGE, stripped)) { + return MIME_TO_LANGUAGE[stripped]; + } + } + return ''; +} + +/** + * Find the longest run of backticks that could close a fenced code block + * inside `source`. CommonMark allows the closer to be indented up to 3 + * spaces, so runs at columns 0–3 all qualify. The fence emitted by + * `wrapAsFencedCodeBlock` must have STRICTLY MORE backticks than any + * such run inside the payload — otherwise a markdown snippet inside a + * JS template literal (or any source containing ` ``` ` near + * column 0) closes the outer fence early and the rest renders as + * markdown, corrupting the artifact. + */ +export function longestLeadingBacktickRun(source: string): number { + let max = 0; + /* `^ {0,3}(`+)` matches lines whose leading whitespace is ≤ 3 + * spaces (per CommonMark's fence-indent allowance) followed by one + * or more backticks. Tabs are not allowed — CommonMark expands them + * to 4 spaces, which is over the closer limit. The multiline + global + * flags scan every line in the payload. */ + const re = /^ {0,3}(`+)/gm; + for (let m = re.exec(source); m !== null; m = re.exec(source)) { + if (m[1].length > max) max = m[1].length; + } + return max; +} + +/** + * Wrap raw source as a fenced markdown code block so the CODE bucket + * can ride the existing markdown rendering pipeline — `marked` emits + * `
` and the future + * highlighter swap-in (currently the markdown template uses plain + * `marked` with no syntax colors) will pick up `language-` + * automatically. Pure; deterministic; safe on empty input (renders an + * empty fenced block which marked handles cleanly). + * + * Fence length is adaptive: 3 backticks by default, but bumped to + * (longest-leading-run-in-source + 1) when the source contains + * ` ``` ` (or longer) near the start of any line. This is the + * CommonMark-spec way to embed a fenced block inside another fenced + * block — see e.g. the markdown spec's nested-fence examples — and + * avoids the early-close attack on artifacts whose source is itself + * markdown-shaped. + */ +export function wrapAsFencedCodeBlock(source: string, lang: string): string { + const fenceLength = Math.max(3, longestLeadingBacktickRun(source) + 1); + const fence = '`'.repeat(fenceLength); + /* Trim a single trailing newline (common in extractor output) so the + * fence's closing ``` lands on its own line rather than after a blank + * gap that marked would render as an extra
. */ + const body = source.endsWith('\n') ? source.slice(0, -1) : source; + return fence + lang + '\n' + body + '\n' + fence; +} + const EXTENSION_TO_TOOL_ARTIFACT_TYPE: Record = { html: TOOL_ARTIFACT_TYPES.HTML, htm: TOOL_ARTIFACT_TYPES.HTML, + // jsx/tsx are React component sources — keep them on the React + // (sandpack) bucket rather than the new CODE bucket so the existing + // live-preview behavior survives. Plain JS/TS source goes through CODE. jsx: TOOL_ARTIFACT_TYPES.REACT, tsx: TOOL_ARTIFACT_TYPES.REACT, md: TOOL_ARTIFACT_TYPES.MARKDOWN, @@ -199,25 +510,19 @@ const EXTENSION_TO_TOOL_ARTIFACT_TYPE: Record = { pptx: TOOL_ARTIFACT_TYPES.PLAIN_TEXT, }; -const extensionOf = (filename: string | undefined): string => { - if (!filename) { - return ''; - } - const dot = filename.lastIndexOf('.'); - if (dot < 0 || dot === filename.length - 1) { - return ''; - } - return filename.slice(dot + 1).toLowerCase(); -}; - -/** Strip charset / boundary parameters so we can do exact MIME comparisons. */ -const baseMime = (mime: string | undefined): string => { - if (!mime) { - return ''; - } - const semi = mime.indexOf(';'); - return (semi < 0 ? mime : mime.slice(0, semi)).trim().toLowerCase(); -}; +/* Append every entry in `CODE_EXTENSION_TO_LANGUAGE` to the routing map + * pointing at the CODE bucket. Keeping the language map as the source + * of truth means a new language is one entry away from being routable. + * + * Skip extensions already claimed by a non-CODE bucket — `jsx`/`tsx` + * belong to React (sandpack) for the live-preview, and a future + * contributor adding them to `CODE_EXTENSION_TO_LANGUAGE` (a natural + * mistake — they ARE source code) shouldn't silently break the React + * routing path. The explicit map entries above always win. */ +for (const ext of Object.keys(CODE_EXTENSION_TO_LANGUAGE)) { + if (ext in EXTENSION_TO_TOOL_ARTIFACT_TYPE) continue; + EXTENSION_TO_TOOL_ARTIFACT_TYPE[ext] = TOOL_ARTIFACT_TYPES.CODE; +} const MIME_TO_TOOL_ARTIFACT_TYPE: Record = { 'text/html': TOOL_ARTIFACT_TYPES.HTML, @@ -227,6 +532,37 @@ const MIME_TO_TOOL_ARTIFACT_TYPE: Record = { 'application/vnd.react': TOOL_ARTIFACT_TYPES.REACT, 'application/vnd.ant.react': TOOL_ARTIFACT_TYPES.REACT, 'application/vnd.mermaid': TOOL_ARTIFACT_TYPES.MERMAID, + // Code MIME types — codeapi serves these via Content-Type for source + // files (`text/x-python`, `text/x-typescript`, etc.) so a file whose + // extension was stripped or renamed upstream still routes to CODE. + // This is a best-effort COMMON-CASE list, not an exhaustive mirror of + // `CODE_EXTENSION_TO_LANGUAGE` — extension-based routing is the + // primary path, and the MIME table only matters when the filename is + // missing/extensionless AND the upstream supplied a useful MIME. + // Adding more here is harmless; missing entries fall through to inline + // rendering (the same as today). + 'text/javascript': TOOL_ARTIFACT_TYPES.CODE, + 'application/javascript': TOOL_ARTIFACT_TYPES.CODE, + 'application/sql': TOOL_ARTIFACT_TYPES.CODE, + 'application/x-sh': TOOL_ARTIFACT_TYPES.CODE, + 'application/x-php': TOOL_ARTIFACT_TYPES.CODE, + 'application/x-powershell': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-python': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-typescript': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-ruby': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-go': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-rust': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-c': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-c++': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-csharp': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-java': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-kotlin': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-scala': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-perl': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-r': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-lua': TOOL_ARTIFACT_TYPES.CODE, + 'text/x-swift': TOOL_ARTIFACT_TYPES.CODE, + 'text/css': TOOL_ARTIFACT_TYPES.CODE, // Office MIME types fall through to the plain-text bucket here too — // matches the extension map so a file whose extension was stripped // somewhere upstream still routes to the panel. @@ -258,15 +594,30 @@ const MIME_TO_TOOL_ARTIFACT_TYPE: Record = { export function detectArtifactTypeFromFile( attachment: Partial>, ): ToolArtifactType | null { - const byExtension = EXTENSION_TO_TOOL_ARTIFACT_TYPE[extensionOf(attachment.filename)]; - const type = byExtension ?? MIME_TO_TOOL_ARTIFACT_TYPE[baseMime(attachment.type)] ?? null; + /* Compute the basename once and reuse it across the extension AND + * bare-name lookups. Both `extensionOf(filename)` and + * `bareNameOf(filename)` would otherwise split path separators + * twice on the same input. */ + const base = attachment.filename ? basenameOf(attachment.filename) : ''; + const byExtension = EXTENSION_TO_TOOL_ARTIFACT_TYPE[extensionFromBasename(base)]; + /* Bare-name fallback for extensionless build files (`Dockerfile`, + * `Makefile`, `Gemfile`, `Rakefile`, `Vagrantfile`, `Brewfile`). Only + * fires when the extension lookup missed AND the basename is in the + * routing map; everything else stays on the existing extension/MIME + * paths. */ + const byBareName = byExtension + ? undefined + : EXTENSION_TO_TOOL_ARTIFACT_TYPE[bareNameFromBasename(base)]; + const type = + byExtension ?? byBareName ?? MIME_TO_TOOL_ARTIFACT_TYPE[baseMime(attachment.type)] ?? null; if (type == null) { return null; } if ( !attachment.text && type !== TOOL_ARTIFACT_TYPES.PLAIN_TEXT && - type !== TOOL_ARTIFACT_TYPES.MARKDOWN + type !== TOOL_ARTIFACT_TYPES.MARKDOWN && + type !== TOOL_ARTIFACT_TYPES.CODE ) { return null; } @@ -346,10 +697,23 @@ export function fileToArtifact( if ( !attachment.text && type !== TOOL_ARTIFACT_TYPES.PLAIN_TEXT && - type !== TOOL_ARTIFACT_TYPES.MARKDOWN + type !== TOOL_ARTIFACT_TYPES.MARKDOWN && + type !== TOOL_ARTIFACT_TYPES.CODE ) { return null; } + /* For CODE artifacts, resolve the language hint at construction time + * (rather than re-deriving from `artifact.title` in `useArtifactProps`) + * so the MIME fallback fires for extensionless filenames. Without + * this, `{ filename: 'noext', type: 'text/x-python' }` would route + * to CODE via the MIME bucket but render with an empty `language-` + * class because the title carries no extension to classify. The + * resolved language is stored on `artifact.language`, which the + * hook reads directly. */ + const language = + type === TOOL_ARTIFACT_TYPES.CODE + ? languageForFilename(attachment.filename, attachment.type) + : undefined; return { id: toolArtifactKey(attachment), type, @@ -360,6 +724,7 @@ export function fileToArtifact( // placeholder. Only `null`/`undefined` fall through to the // placeholder, matching "no extraction has run yet." content: attachment.text ?? options?.placeholder ?? '', + language, messageId: attachment.messageId ?? undefined, lastUpdateTime: toLastUpdate(attachment), };