🧩 feat: Map Message Content Parts to a UI Parts Union (#16374)

* feat: map message content parts to a UI parts union

Adds a UIMessage-shaped view of TMessage modelled on the AI SDK
(ai@7.0.114): content parts map one-to-one onto text, reasoning,
tool-<name>, file and data-* parts, holes map to step-start, and the
reverse restores the stored content so a view round-trips.

* fix: carry text, files and metadata through the reverse message mapping

An empty content array with text now views as contentless, an edited text part updates the stored text, attachment file parts reconcile message files, and metadata fields restore a message rebuilt without a stored base.

* fix: read durable tool markers and apply view edits in the parts mapping

Tool parts fall back from an empty id to the host step id, surface approval pauses and background-cancel or validation failures, and treat an empty legacy function output as submitted. The reverse mapping writes file and attachment edits over the stored records, keeps an explicit null parent, carries non-derived message text and absent text fields, persists hand-built tool failures, and collects text, content and attachments in one pass. Web search sources include link references and take message-wide ids. UIMessage drops the system role it could not round-trip. Adds mock-harness scenarios that view persisted replies, reasoning and tool turns and map them back.

* fix: view persisted reasoning without its think tags

Persisted reasoning is stored inside <think> tags that every renderer strips; the reasoning part now exposes the text alone and keeps the exact wrapper so the reverse mapping restores it. The tool-turn scenario skips mobile, where the composer MCP picker is not offered.

* fix: give view attachments a stable identity and read legacy tool outcomes

Attachment file parts carry the stored path they were viewed with, so an edited url still matches its file. Reverse slots advance only for content parts, audio takes its format from an edited media type, a null model survives a base-less rebuild, legacy retrieval outputs count as submitted, and error-prefixed outputs map to output-error through isToolErrorOutput, the predicate the tool output renderers use.

* fix: keep attachments, empty content and hand-built tool results through a rebuild

A base-less rebuild restores the stored attachments and an explicit empty content array from metadata. Hand-built tool parts record a completed status for output-available and keep an array output as a code interpreter call. Files the view never showed keep their stored positions while visible files fill the others in view order.

* feat: let the caller supply tool outcome rules to the parts mapping

Some tool outcomes are decided by rules the client owns, such as memory failure prose and background status attachments. toUIMessage, toUIParts and toUIPart take a resolveToolFailure option instead of copying those rules. Text deltas keep their ownership metadata, an empty hand-built array output is marked completed, and fromUIMessage documents that a stored message round-trips only with its base.

* fix: map untyped lane placeholders to step-start

A dual-conversation turn seeds type '' placeholder slots, which fell through the part switch and crashed toUIMessage. Any slot without a known type now maps to step-start, and the reverse leaves a hole, the same thing persistence compaction does.

* fix: identify view attachments by position and keep content added to plain messages

Attachment parts carry their index in message.files beside the path, so two files sharing a path stay distinct, and a hand-built part claims the first unclaimed match. Content parts added to a view of a plain message are written as content. Tool arguments that parse to a JSON primitive stay their string, inside the declared input type.

* fix: derive fallback tool ids from the streamed index and drop stale annotations

A tool call with neither a provider id nor a step id takes its fallback id from the streamed index the settled message keeps, so its identity survives compaction. Annotations on object-form text are dropped once the view's text no longer matches the value they index into.

* fix: map tool call slots without a call to step-start and keep text delta object shape

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017snqTCuA2LHsoKuC335HFG

* fix: report background and caller-resolved tool failures by their reason

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017snqTCuA2LHsoKuC335HFG

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Marco Beretta 2026-09-26 15:54:58 +02:00 • committed by GitHub
parent 6d54d07ac1
commit 6ac941eff3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 2434 additions and 0 deletions

View file

@ -0,0 +1,133 @@
import { expect, test } from '@playwright/test';
import { fromUIMessage, toUIMessage } from 'librechat-data-provider';
import type { Page } from '@playwright/test';
import type { TMessage, UIMessage } from 'librechat-data-provider';
import {
sendMessageAndWaitForCompletion,
selectMockEndpoint,
getAccessToken,
MOCK_ENDPOINTS,
NEW_CHAT_PATH,
replyPrompt,
thinkPrompt,
replyText,
thinkText,
fetchJson,
} from '../helpers';
/** Non-spec endpoint without an activity label, so its MCP tool cards stay separate parts. */
const TOOL_ENDPOINT = { label: 'Mock Provider D', model: 'mock-model-d' };
const MCP_SERVER_TITLE = 'E2E Memory';
const uniqueLabel = (prefix: string) =>
`${prefix}-${Date.now()}-${Math.floor(Math.random() * 1e4)}`;
/** Reads the conversation the page is on, as persisted, through the messages API. */
async function readPersistedMessages(page: Page): Promise<TMessage[]> {
const conversationId = new URL(page.url()).pathname.replace('/c/', '');
expect(conversationId).not.toBe('new');
const token = await getAccessToken(page);
return fetchJson<TMessage[]>(page, `/api/messages/${encodeURIComponent(conversationId)}`, token);
}
/** Views every persisted message and requires each view to map back to the stored message. */
function viewAndRestore(messages: TMessage[]): UIMessage[] {
return messages.map((message) => {
const view = toUIMessage(message);
expect(fromUIMessage(view, message)).toStrictEqual(message);
return view;
});
}
async function selectMemoryServer(page: Page) {
await page.getByRole('button', { name: 'MCP Servers', exact: true }).click();
const serverItem = page.getByRole('menuitemcheckbox', { name: new RegExp(MCP_SERVER_TITLE) });
await expect(serverItem).toBeVisible();
await serverItem.click();
await expect(serverItem).toHaveAttribute('aria-checked', 'true');
await page.keyboard.press('Escape');
}
test.describe('UI parts view of persisted messages', () => {
test('views a persisted reply as a user and an assistant UI message @scenario:persisted-reply-views-as-user-and-assistant-messages', async ({
page,
}) => {
const label = uniqueLabel('parts-reply');
await page.goto(NEW_CHAT_PATH, { timeout: 10_000 });
await selectMockEndpoint(page, MOCK_ENDPOINTS[0]);
const response = await sendMessageAndWaitForCompletion(page, replyPrompt(label));
expect(response.ok()).toBeTruthy();
const [user, assistant] = viewAndRestore(await readPersistedMessages(page));
expect(user.role).toBe('user');
expect(user.parts).toContainEqual({ type: 'text', text: replyPrompt(label) });
expect(assistant.role).toBe('assistant');
expect(assistant.metadata?.parentMessageId).toBe(user.id);
const text = assistant.parts
.filter((part) => part.type === 'text')
.map((part) => part.text)
.join('');
expect(text).toBe(replyText(label));
});
test('views persisted reasoning before the answer, in content order @scenario:persisted-reasoning-views-before-its-answer', async ({
page,
}) => {
const label = uniqueLabel('parts-think');
await page.goto(NEW_CHAT_PATH, { timeout: 10_000 });
await selectMockEndpoint(page, MOCK_ENDPOINTS[0]);
const response = await sendMessageAndWaitForCompletion(page, thinkPrompt(label));
expect(response.ok()).toBeTruthy();
const messages = await readPersistedMessages(page);
const assistant = viewAndRestore(messages).find((message) => message.role === 'assistant');
const stored = messages.find((message) => !message.isCreatedByUser);
if (!assistant || !stored?.content) {
throw new Error('Persisted reasoning reply missing');
}
expect(assistant.parts.slice(0, stored.content.length).map((part) => part.type)).toEqual(
stored.content.map((part) => (part.type === 'think' ? 'reasoning' : part.type)),
);
const reasoning = assistant.parts.findIndex((part) => part.type === 'reasoning');
const answer = assistant.parts.findIndex((part) => part.type === 'text');
expect(reasoning).toBeGreaterThanOrEqual(0);
expect(reasoning).toBeLessThan(answer);
expect(assistant.parts[reasoning]).toMatchObject({ text: thinkText(label) });
});
test.describe('with an MCP tool turn', () => {
test.skip(({ isMobile }) => isMobile === true, 'composer MCP picker is desktop-only');
test('views persisted tool calls as completed tool parts @scenario:persisted-tool-calls-view-as-completed-tool-parts', async ({
page,
}) => {
test.setTimeout(90_000);
const label = uniqueLabel('parts-tool');
await page.goto(NEW_CHAT_PATH, { timeout: 10_000 });
await selectMockEndpoint(page, TOOL_ENDPOINT);
await selectMemoryServer(page);
const response = await sendMessageAndWaitForCompletion(page, `E2E_ACTIVITY_REPLY:${label}`, {
timeout: 60_000,
});
expect(response.ok()).toBeTruthy();
const messages = await readPersistedMessages(page);
const assistant = viewAndRestore(messages).find((message) => message.role === 'assistant');
if (!assistant) {
throw new Error('Persisted tool reply missing');
}
const tools = assistant.parts.filter((part) => part.type.startsWith('tool-'));
expect(tools.length).toBeGreaterThanOrEqual(2);
for (const tool of tools) {
expect(tool).toMatchObject({ state: 'output-available', toolCallId: expect.any(String) });
expect((tool as { toolCallId: string }).toolCallId).not.toBe('');
}
expect(new Set(tools.map((tool) => (tool as { toolCallId: string }).toolCallId)).size).toBe(
tools.length,
);
});
});
});

View file

@ -4,6 +4,11 @@ export function hasToolCallErrorPrefix(text: string): boolean {
return TOOL_CALL_ERROR_PREFIX.test(text);
}
/** Whether a tool output string is one the tool output renderers present as a failure. */
export function isToolErrorOutput(text: string): boolean {
return hasToolCallErrorPrefix(text) || text.startsWith('Error processing tool');
}
export function stripToolCallErrorPrefix(text: string): string {
return text.replace(TOOL_CALL_ERROR_PREFIX, '');
}

View file

@ -13,6 +13,8 @@ export * from './messages';
export * from './errors';
/* run steps */
export * from './runSteps';
/* ui parts */
export * from './parts';
/* artifacts */
export * from './artifacts';
/* schema helpers */

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff