From 3f442e4dbe633cad5ac08f30cb76b00b53501efd Mon Sep 17 00:00:00 2001 From: Danny Avila Date: Mon, 5 Oct 2026 07:42:45 -0400 Subject: [PATCH] =?UTF-8?q?=F0=9F=AA=B6=20perf:=20Load=20Tool-Call=20Previ?= =?UTF-8?q?ews=20and=20Fetch=20Full=20Parts=20on=20Demand=20(#16753)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * 🧾 perf: Load Tool-Call Previews and Fetch Full Parts on Demand Long agent conversations shipped every tool output, argument string and subagent transcript on load, though the cards render collapsed: 12-18 MB of message JSON per chat, enough to crash iPhone Safari tabs. A client that adds ?toolPreviews=1 to the conversation load now receives bounded previews of settled tool calls (start and end of long outputs, JSON arguments and outputs kept valid with only their longest strings shortened, exit-status trailers kept whole, subagent transcripts omitted) with explicit markers and full lengths. Opening a card or a subagent panel fetches the stored part from a new owner-scoped endpoint, GET /api/messages/:conversationId/:messageId/parts/:partIndex, cached under its own React Query key. Clients that do not ask, exports and share links keep the full payload; bounds are configurable under toolCallPreviews in librechat.yaml, and a client-authored message that carries previews is refused rather than saved. * 🧾 fix: Validate Part Reads in Parallel and Preview Fork Responses The full-part read now starts beside conversation-access validation instead of after it, and keeps the active-job fallback the conversation read has, so a conversation still live before its first save can expand its previews. Fork, duplicate and shared-link fork responses honor the preview parameter, so they no longer seed the conversation cache with the full history. Background-task results whose structure cannot fit a valid JSON preview go whole, image cards keep their prompts, and the subagent panel offers a retry when its stored part fails to load. * 🧾 fix: Key Hydrated Parts by Preview Revision and Call Identity A fetched part is now cached under the preview's revision and replaces only the fields the server shortened, so a call whose stored output changes later (a harvested background result) is fetched again rather than rolled back. Part lookups carry the run-step and agent identity and refuse an ambiguous fallback, since provider tool-call ids can repeat within one response. A subagent that finished without final text has its transcript omitted like any other settled call, and JSON nested past a fixed depth falls back to a text preview; a call whose preview fails is sent whole instead of failing the conversation load. * 🧾 fix: Release Hydrated Parts With Their History and Bound JSON Work Full tool-call parts fetched for a conversation are removed when its message history is released or cleared, so they cannot outlive it under their own key. The structure-preserving JSON preview now bails out before cloning when the value has more nodes or strings than could ever fit, keeps a __proto__ key as ordinary data, and the part endpoint accepts any safe-integer index instead of an arbitrary ceiling. * 🧾 fix: Count Property Names Before Cloning a JSON Preview Property names are never shortened, so the pre-check now adds their length to its budget and falls back to the text preview before any cloning when they alone could not fit. * 🧾 fix: Preview Artifact and Branch Responses and Rehydrate Subagent Prompts Artifact edits and parallel-response branches merge the message the server returns into the conversation cache, so both requests now ask for previews and their routes honor it. The subagent panel reads the prompt from the stored arguments once they load, and the part endpoint no longer caps identifier length, so any previewed call can be fetched. * 🧾 fix: Preview Only Tool Calls That Can Be Fetched Back by Identity A call without an id has nothing but its position to name it, and the position can shift on the client, so it is left whole instead of being previewed. The part endpoint now requires a tool-call id, so an indexed lookup always confirms the call it returns. * 🧾 fix: Fetch Parts by Exact Identity and Preview Only Unique Ones The part endpoint now finds a call by its exact identity (provider id, run-step id, agent) instead of trusting the client's index, and the preview pass only shortens calls whose identity is unique in their message, so every preview names exactly one stored part. Legacy web-search failures that carry their error phrase mid-output go whole, since the collapsed card reads that phrase anywhere. * 🧾 fix: Bound the JSON Fit Walk Before Queueing and Normalize Empty Identities The structure check now counts an array's or object's children against the node budget before queueing them, so a very wide value is refused without building a work list its size. An empty run-step or agent id is treated as absent by the part lookup, matching how the preview pass and the client already treat it. * 🧾 fix: Key Fetched Parts to the Stored Version and Hydrate Search Details Each preview now carries `previewRevision`, when its message last changed, and the client keys fetched parts to that plus a fingerprint of the whole preview, so any change to the stored call is fetched again. The web-search details card asks for the stored call when it opens, so its query is never left shortened. * 🧾 fix: Cache Fetched Parts by Identity and Hold Copy Until Hydrated The part query key drops the client-side index, which only rides along in the request; the server locates the call by message and exact identity, so a shifted index serves the cached copy instead of downloading it again. Copy controls inside a previewed part are disabled until the stored content arrives, so a slow or failed fetch cannot export shortened text. * 🧾 fix: Shape Shared Forks With the Recipient's Bounds and Keep JSON Numbers Exact A shared-link fork resolves the share owner's config for its content checks, but the forked conversation belongs to the requester, so its preview bounds now come from the requester's tenant and role (prepareResultToolCallPreviews). A structure-preserving JSON preview now requires every number in the text to survive a parse and re-serialization as the same value; an integer past 2^53, an out-of-range exponent or an over-precise decimal falls back to the text preview (or the whole output for parsed-output tools) instead of showing a changed value. --- api/server/middleware/index.js | 2 + .../__test-utils__/convos-route-mocks.js | 1 + .../__tests__/messages-content-edit.spec.js | 6 + .../routes/__tests__/messages-delete.spec.js | 6 + .../__tests__/messages-feedback.spec.js | 6 + .../messages-get-real-validation.spec.js | 6 + .../routes/__tests__/messages-get.spec.js | 6 + .../__tests__/messages-retention.spec.js | 6 + .../messages-subagent-thread.spec.js | 6 + .../__tests__/messages-tool-previews.spec.js | 396 +++++++++ .../routes/__tests__/messages-update.spec.js | 6 + api/server/routes/__tests__/share.spec.js | 29 + api/server/routes/convos.js | 5 +- api/server/routes/messages.js | 43 +- api/server/routes/share.js | 4 +- .../Chat/Messages/Content/AgentHandoff.tsx | 33 +- .../Chat/Messages/Content/ContentParts.tsx | 31 +- .../components/Chat/Messages/Content/Part.tsx | 11 + .../Chat/Messages/Content/Parts/BashCall.tsx | 9 +- .../Content/Parts/CodeWindowHeader.tsx | 14 +- .../Messages/Content/Parts/SubagentCall.tsx | 26 +- .../Chat/Messages/Content/Parts/command.ts | 6 +- .../Content/ToolOutput/OutputRenderer.tsx | 8 +- .../__tests__/OutputRenderer.test.tsx | 35 +- .../Chat/Messages/Content/WebSearch.tsx | 3 +- .../Content/__tests__/WebSearch.test.tsx | 23 + .../Content/__tests__/hydration.test.tsx | 286 +++++++ .../Chat/Messages/Content/disclosure.ts | 25 +- .../Chat/Messages/Content/hydration.tsx | 125 +++ .../Subagents/SubagentThreadPanel.test.tsx | 126 +++ .../Chat/Subagents/SubagentThreadPanel.tsx | 88 +- .../src/components/Chat/Subagents/prompt.ts | 25 + client/src/components/Chat/Subagents/state.ts | 10 + .../src/components/Chat/Trace/Inspector.tsx | 17 +- client/src/components/Chat/Trace/Viewer.tsx | 5 +- client/src/components/Chat/Trace/preview.ts | 14 +- .../ConvoOptions/ShareButton.tsx | 4 +- .../__tests__/ShareButton.test.tsx | 10 +- .../Messages/Content/ActionButton.tsx | 4 + .../Messages/Content/CopyButton.tsx | 14 +- .../Messages/__tests__/queries.test.ts | 16 +- .../Messages/__tests__/retention.spec.ts | 29 + client/src/data-provider/Messages/queries.ts | 47 +- .../src/data-provider/Messages/retention.ts | 12 + client/src/hooks/SSE/useResumableSSE.ts | 3 +- client/src/locales/en/translation.json | 2 + client/src/utils/approval.spec.ts | 26 + client/src/utils/tokens.spec.ts | 30 + client/src/utils/tokens.ts | 63 +- librechat.example.yaml | 12 + packages/api/src/index.ts | 2 + packages/api/src/messages/index.ts | 2 + packages/api/src/messages/parts.spec.ts | 419 ++++++++++ packages/api/src/messages/parts.ts | 234 ++++++ packages/api/src/messages/previews.spec.ts | 627 +++++++++++++++ packages/api/src/messages/previews.ts | 750 ++++++++++++++++++ .../api/src/middleware/messageValidation.ts | 31 +- packages/data-provider/src/api-endpoints.ts | 21 + packages/data-provider/src/config.ts | 21 + .../data-provider/src/createPayload.spec.ts | 55 ++ packages/data-provider/src/data-service.ts | 33 +- packages/data-provider/src/index.ts | 1 + packages/data-provider/src/keys.ts | 2 + packages/data-provider/src/previews.spec.ts | 47 ++ packages/data-provider/src/previews.ts | 75 ++ packages/data-provider/src/types/agents.ts | 15 + packages/data-provider/src/types/queries.ts | 21 + packages/data-schemas/src/app/service.ts | 14 + packages/data-schemas/src/types/app.ts | 3 + 69 files changed, 3949 insertions(+), 144 deletions(-) create mode 100644 api/server/routes/__tests__/messages-tool-previews.spec.js create mode 100644 client/src/components/Chat/Messages/Content/__tests__/hydration.test.tsx create mode 100644 client/src/components/Chat/Messages/Content/hydration.tsx create mode 100644 client/src/components/Chat/Subagents/prompt.ts create mode 100644 packages/api/src/messages/index.ts create mode 100644 packages/api/src/messages/parts.spec.ts create mode 100644 packages/api/src/messages/parts.ts create mode 100644 packages/api/src/messages/previews.spec.ts create mode 100644 packages/api/src/messages/previews.ts create mode 100644 packages/data-provider/src/createPayload.spec.ts create mode 100644 packages/data-provider/src/previews.spec.ts create mode 100644 packages/data-provider/src/previews.ts diff --git a/api/server/middleware/index.js b/api/server/middleware/index.js index 774ee44a2b..ca0ca7c367 100644 --- a/api/server/middleware/index.js +++ b/api/server/middleware/index.js @@ -9,6 +9,7 @@ const validateEmailLogin = require('./validateEmailLogin'); const validateMessageReq = require('./validateMessageReq'); const { canReadActiveJobConversation, + createMessageRequestValidation, prepareMessageRequestValidation, sendValidationResponse, } = require('./messageValidation'); @@ -67,6 +68,7 @@ module.exports = { validateMessageReq, canReadActiveJobConversation, sendValidationResponse, + createMessageRequestValidation, prepareMessageRequestValidation, buildEndpointOption, validateRegistration, diff --git a/api/server/routes/__test-utils__/convos-route-mocks.js b/api/server/routes/__test-utils__/convos-route-mocks.js index 34fd583f6e..bd86738d13 100644 --- a/api/server/routes/__test-utils__/convos-route-mocks.js +++ b/api/server/routes/__test-utils__/convos-route-mocks.js @@ -79,6 +79,7 @@ module.exports = { agents: () => ({ sleep: jest.fn() }), api: (overrides = {}) => ({ + withToolCallPreviews: (_req, result) => result, createGeneratedTitleHandler: jest.fn(() => jest.fn()), createRenameConversationHandler: jest.fn((deps) => { renameHandlerInputs.push(deps); diff --git a/api/server/routes/__tests__/messages-content-edit.spec.js b/api/server/routes/__tests__/messages-content-edit.spec.js index 48e2320c1f..16bc075022 100644 --- a/api/server/routes/__tests__/messages-content-edit.spec.js +++ b/api/server/routes/__tests__/messages-content-edit.spec.js @@ -7,6 +7,10 @@ jest.mock('@librechat/agents', () => ({ })); jest.mock('@librechat/api', () => ({ + prepareToolCallPreviews: jest.fn(() => (messages) => Promise.resolve(messages)), + createToolCallPartHandler: jest.fn(() => (_req, res) => res.status(404).end()), + rejectToolCallPreviewWrites: (_req, _res, next) => next(), + withMessageToolCallPreviews: (_req, message) => message, unescapeLaTeX: jest.fn((value) => value), countTokens: jest.fn().mockResolvedValue(2), createPrivateTextView: jest.fn(() => (_req, _res, next) => next()), @@ -26,6 +30,8 @@ jest.mock('@librechat/api', () => ({ applyForcedRetention: jest.fn(), })); +jest.mock('~/server/services/Config', () => ({ getAppConfig: jest.fn() })); + jest.mock('~/server/services/Endpoints/agents/subagentThreadStore', () => ({})); jest.mock('@librechat/data-schemas', () => ({ diff --git a/api/server/routes/__tests__/messages-delete.spec.js b/api/server/routes/__tests__/messages-delete.spec.js index c817425619..6dfe494d65 100644 --- a/api/server/routes/__tests__/messages-delete.spec.js +++ b/api/server/routes/__tests__/messages-delete.spec.js @@ -9,6 +9,10 @@ jest.mock('@librechat/agents', () => ({ })); jest.mock('@librechat/api', () => ({ + prepareToolCallPreviews: jest.fn(() => (messages) => Promise.resolve(messages)), + createToolCallPartHandler: jest.fn(() => (_req, res) => res.status(404).end()), + rejectToolCallPreviewWrites: (_req, _res, next) => next(), + withMessageToolCallPreviews: (_req, message) => message, createPrivateTextView: jest.fn(() => (_req, _res, next) => next()), createContentFilter: jest.fn(() => (req, res, next) => next()), inspectContent: jest.fn(() => null), @@ -24,6 +28,8 @@ jest.mock('@librechat/api', () => ({ requireFeedbackEnabled: (req, res, next) => next(), })); +jest.mock('~/server/services/Config', () => ({ getAppConfig: jest.fn() })); + jest.mock('~/server/services/Endpoints/agents/subagentThreadStore', () => ({})); jest.mock('@librechat/data-schemas', () => ({ diff --git a/api/server/routes/__tests__/messages-feedback.spec.js b/api/server/routes/__tests__/messages-feedback.spec.js index 1174518fc5..25064b188d 100644 --- a/api/server/routes/__tests__/messages-feedback.spec.js +++ b/api/server/routes/__tests__/messages-feedback.spec.js @@ -6,6 +6,10 @@ jest.mock('@librechat/agents', () => ({ })); jest.mock('@librechat/api', () => ({ + prepareToolCallPreviews: jest.fn(() => (messages) => Promise.resolve(messages)), + createToolCallPartHandler: jest.fn(() => (_req, res) => res.status(404).end()), + rejectToolCallPreviewWrites: (_req, _res, next) => next(), + withMessageToolCallPreviews: (_req, message) => message, unescapeLaTeX: jest.fn((value) => value), countTokens: jest.fn().mockResolvedValue(10), createPrivateTextView: jest.fn(() => (_req, _res, next) => next()), @@ -18,6 +22,8 @@ jest.mock('@librechat/api', () => ({ applyForcedRetention: jest.fn(), })); +jest.mock('~/server/services/Config', () => ({ getAppConfig: jest.fn() })); + jest.mock('~/server/services/Endpoints/agents/subagentThreadStore', () => ({})); jest.mock('@librechat/data-schemas', () => ({ diff --git a/api/server/routes/__tests__/messages-get-real-validation.spec.js b/api/server/routes/__tests__/messages-get-real-validation.spec.js index a7d7e4a4d9..fc2e02a21a 100644 --- a/api/server/routes/__tests__/messages-get-real-validation.spec.js +++ b/api/server/routes/__tests__/messages-get-real-validation.spec.js @@ -21,6 +21,10 @@ jest.mock('@librechat/agents', () => ({ })); jest.mock('@librechat/api', () => ({ + prepareToolCallPreviews: jest.fn(() => (messages) => Promise.resolve(messages)), + createToolCallPartHandler: jest.fn(() => (_req, res) => res.status(404).end()), + rejectToolCallPreviewWrites: (_req, _res, next) => next(), + withMessageToolCallPreviews: (_req, message) => message, createPrivateTextView: jest.fn(() => (_req, _res, next) => next()), createContentFilter: jest.fn(() => (req, res, next) => next()), inspectContent: jest.fn(() => null), @@ -43,6 +47,8 @@ jest.mock('@librechat/api', () => ({ requireFeedbackEnabled: (req, res, next) => next(), })); +jest.mock('~/server/services/Config', () => ({ getAppConfig: jest.fn() })); + jest.mock('~/server/services/Endpoints/agents/subagentThreadStore', () => ({})); jest.mock('@librechat/data-schemas', () => ({ diff --git a/api/server/routes/__tests__/messages-get.spec.js b/api/server/routes/__tests__/messages-get.spec.js index de858ac410..401b6a049f 100644 --- a/api/server/routes/__tests__/messages-get.spec.js +++ b/api/server/routes/__tests__/messages-get.spec.js @@ -51,6 +51,10 @@ jest.mock('@librechat/api', () => { }; return { + prepareToolCallPreviews: jest.fn(() => (messages) => Promise.resolve(messages)), + createToolCallPartHandler: jest.fn(() => (_req, res) => res.status(404).end()), + rejectToolCallPreviewWrites: (_req, _res, next) => next(), + withMessageToolCallPreviews: (_req, message) => message, /** The real helper, without loading the rest of the package this suite mocks around. */ withoutTraceRefs: jest.requireActual('../../../../packages/api/src/langfuse/trace.ts') .withoutTraceRefs, @@ -126,6 +130,8 @@ jest.mock('@librechat/api', () => { }; }); +jest.mock('~/server/services/Config', () => ({ getAppConfig: jest.fn() })); + jest.mock('~/server/services/Endpoints/agents/subagentThreadStore', () => ({})); jest.mock('@librechat/data-schemas', () => ({ diff --git a/api/server/routes/__tests__/messages-retention.spec.js b/api/server/routes/__tests__/messages-retention.spec.js index 9bbbc0ce9d..6aef3dbca9 100644 --- a/api/server/routes/__tests__/messages-retention.spec.js +++ b/api/server/routes/__tests__/messages-retention.spec.js @@ -7,6 +7,10 @@ jest.mock('@librechat/agents', () => ({ })); jest.mock('@librechat/api', () => ({ + prepareToolCallPreviews: jest.fn(() => (messages) => Promise.resolve(messages)), + createToolCallPartHandler: jest.fn(() => (_req, res) => res.status(404).end()), + rejectToolCallPreviewWrites: (_req, _res, next) => next(), + withMessageToolCallPreviews: (_req, message) => message, createPrivateTextView: jest.fn(() => (_req, _res, next) => next()), stripPrivateMessageFields: jest.requireActual( '../../../../packages/api/src/protection/private/view', @@ -39,6 +43,8 @@ jest.mock('@librechat/data-schemas', () => ({ }, })); +jest.mock('~/server/services/Config', () => ({ getAppConfig: jest.fn() })); + jest.mock('~/server/services/Endpoints/agents/subagentThreadStore', () => ({})); jest.mock('~/models', () => ({ diff --git a/api/server/routes/__tests__/messages-subagent-thread.spec.js b/api/server/routes/__tests__/messages-subagent-thread.spec.js index 574a1435e3..432eed0d7e 100644 --- a/api/server/routes/__tests__/messages-subagent-thread.spec.js +++ b/api/server/routes/__tests__/messages-subagent-thread.spec.js @@ -6,6 +6,10 @@ const mockIsSubagentThreadWriteBlocked = jest.fn(); jest.mock('@librechat/agents', () => ({ sleep: jest.fn() })); jest.mock('@librechat/api', () => ({ + prepareToolCallPreviews: jest.fn(() => (messages) => Promise.resolve(messages)), + createToolCallPartHandler: jest.fn(() => (_req, res) => res.status(404).end()), + rejectToolCallPreviewWrites: (_req, _res, next) => next(), + withMessageToolCallPreviews: (_req, message) => message, withoutTraceRefs: jest.fn((message) => message), createPrivateTextView: jest.fn(() => (_req, _res, next) => next()), createContentFilter: jest.fn(() => (_req, _res, next) => next()), @@ -52,6 +56,8 @@ jest.mock('~/models', () => ({ getMessagesByCursor: jest.fn(), })); +jest.mock('~/server/services/Config', () => ({ getAppConfig: jest.fn() })); + jest.mock('~/server/services/Endpoints/agents/subagentThreadStore', () => ({ isThreadActiveForOwner: jest.fn(), })); diff --git a/api/server/routes/__tests__/messages-tool-previews.spec.js b/api/server/routes/__tests__/messages-tool-previews.spec.js new file mode 100644 index 0000000000..4d58bb9883 --- /dev/null +++ b/api/server/routes/__tests__/messages-tool-previews.spec.js @@ -0,0 +1,396 @@ +const mongoose = require('mongoose'); +const express = require('express'); +const request = require('supertest'); +const { v4: uuidv4 } = require('uuid'); +const { MongoMemoryServer } = require('mongodb-memory-server'); + +const mockDb = { methods: null }; + +jest.mock('@librechat/api', () => ({ + ...jest.requireActual('@librechat/api'), + countTokens: jest.fn().mockResolvedValue(10), + applyForcedRetention: jest.fn().mockResolvedValue(undefined), + createContentFilter: jest.fn(() => (_req, _res, next) => next()), + createPrivateTextView: jest.fn(() => (_req, _res, next) => next()), + isSubagentThreadWriteBlocked: jest.fn().mockResolvedValue(false), + GenerationJobManager: { getJob: jest.fn().mockResolvedValue(null) }, +})); + +jest.mock('~/models', () => { + const methods = new Proxy( + {}, + { + get: + (_target, key) => + (...args) => + mockDb.methods[key](...args), + }, + ); + return methods; +}); + +jest.mock('~/server/services/Config', () => ({ + getAppConfig: jest.fn().mockResolvedValue({ + toolCallPreviews: { enabled: true, outputChars: 512, argsChars: 512 }, + }), +})); + +jest.mock('~/server/services/Endpoints/agents/subagentThreadStore', () => ({})); + +jest.mock('~/server/middleware', () => { + const validation = jest.requireActual('~/server/middleware/messageValidation'); + return { + ...validation, + requireJwtAuth: (_req, _res, next) => next(), + configMiddleware: (req, _res, next) => { + req.config = {}; + next(); + }, + }; +}); + +const OWNER = 'owner-user'; +const OTHER = 'other-user'; + +const fullOutput = `stdout:\n${'compiled module\n'.repeat(3_000)}[exit code: 0]`; +const fullArgs = JSON.stringify({ + command: 'npm run build', + intent: 'Build', + pad: 'p'.repeat(4_000), +}); +const transcript = [ + { type: 'text', text: 'child is reading' }, + { + type: 'tool_call', + tool_call: { + id: 'child_1', + name: 'read_file', + args: '{"path":"a"}', + output: 'z'.repeat(2_000), + }, + }, +]; + +describe('tool-call previews on the message routes', () => { + let mongoServer; + let app; + let conversationId; + const responseId = 'response-1'; + + const assistantContent = () => [ + { type: 'text', text: 'Building now', agentId: 'agent_a' }, + { + type: 'tool_call', + agentId: 'agent_a', + tool_call: { + id: 'call_bash', + type: 'tool_call', + name: 'bash_tool', + args: fullArgs, + output: fullOutput, + progress: 1, + }, + }, + { + type: 'tool_call', + agentId: 'agent_a', + tool_call: { + id: 'call_sub', + type: 'tool_call', + name: 'subagent', + args: '{"prompt":"look"}', + output: 'child summary', + progress: 1, + subagent_content: transcript, + }, + }, + ]; + + const storedResponse = async () => { + const [message] = await mockDb.methods.getMessages({ messageId: responseId, user: OWNER }); + return message; + }; + + const expectStoredContentIntact = async () => { + const message = await storedResponse(); + expect(message.content[1].tool_call.output).toBe(fullOutput); + expect(message.content[1].tool_call.args).toBe(fullArgs); + expect(message.content[1].tool_call).not.toHaveProperty('outputTruncated'); + expect(message.content[2].tool_call.subagent_content).toEqual(transcript); + expect(message.content[2].tool_call).not.toHaveProperty('subagentContentOmitted'); + }; + + beforeAll(async () => { + mongoServer = await MongoMemoryServer.create(); + const { createModels, createMethods } = jest.requireActual('@librechat/data-schemas'); + createModels(mongoose); + mockDb.methods = createMethods(mongoose); + await mongoose.connect(mongoServer.getUri()); + + const messagesRouter = require('../messages'); + app = express(); + app.use(express.json({ limit: '10mb' })); + app.use((req, _res, next) => { + req.user = { id: req.headers['x-user'] ?? OWNER }; + next(); + }); + app.use('/api/messages', messagesRouter); + }); + + afterAll(async () => { + await mongoose.disconnect(); + await mongoServer.stop(); + }); + + beforeEach(async () => { + await mongoose.models.Message.deleteMany({}); + await mongoose.models.Conversation.deleteMany({}); + conversationId = uuidv4(); + await mockDb.methods.saveConvo( + { userId: OWNER }, + { conversationId, endpoint: 'agents', title: 'Long chat' }, + ); + await mockDb.methods.saveMessage( + { userId: OWNER }, + { + messageId: 'user-1', + conversationId, + parentMessageId: '00000000-0000-0000-0000-000000000000', + isCreatedByUser: true, + text: 'build it', + }, + ); + await mockDb.methods.saveMessage( + { userId: OWNER }, + { + messageId: responseId, + conversationId, + parentMessageId: 'user-1', + isCreatedByUser: false, + endpoint: 'agents', + text: '', + content: assistantContent(), + }, + ); + }); + + const loadPreviews = async () => { + const response = await request(app).get(`/api/messages/${conversationId}?toolPreviews=1`); + expect(response.status).toBe(200); + return response.body.find((message) => message.messageId === responseId); + }; + + it('sends previews with markers to a client that asks for them', async () => { + const message = await loadPreviews(); + const bash = message.content[1].tool_call; + expect(bash).toMatchObject({ + id: 'call_bash', + outputTruncated: true, + outputLength: fullOutput.length, + argsTruncated: true, + argsLength: fullArgs.length, + }); + expect(bash.output.length).toBeLessThanOrEqual(512); + expect(bash.output.endsWith('[exit code: 0]')).toBe(true); + expect(JSON.parse(bash.args)).toMatchObject({ command: 'npm run build', intent: 'Build' }); + expect(message.content[2].tool_call).toMatchObject({ + subagentContentOmitted: true, + subagentContentParts: 2, + output: 'child summary', + }); + expect(message.content[2].tool_call).not.toHaveProperty('subagent_content'); + expect(message.content[0]).toEqual({ type: 'text', text: 'Building now', agentId: 'agent_a' }); + }); + + it('sends the full payload to an older client that does not ask', async () => { + const response = await request(app).get(`/api/messages/${conversationId}`); + const message = response.body.find((entry) => entry.messageId === responseId); + expect(message.content[1].tool_call.output).toBe(fullOutput); + expect(message.content[2].tool_call.subagent_content).toEqual(transcript); + expect(JSON.stringify(response.body)).not.toContain('Truncated'); + }); + + it('previews the cursor-paginated conversation read too', async () => { + const response = await request(app).get( + `/api/messages?conversationId=${conversationId}&toolPreviews=1&sortBy=createdAt&sortDirection=asc`, + ); + expect(response.status).toBe(200); + const message = response.body.messages.find((entry) => entry.messageId === responseId); + expect(message.content[1].tool_call.outputTruncated).toBe(true); + }); + + it('serves the full part to its owner and nobody else', async () => { + const url = `/api/messages/${conversationId}/${responseId}/parts/1?toolCallId=call_bash&agentId=agent_a`; + const owner = await request(app).get(url); + expect(owner.status).toBe(200); + expect(owner.body.tool_call.output).toBe(fullOutput); + expect(owner.body.tool_call.args).toBe(fullArgs); + + const other = await request(app).get(url).set('x-user', OTHER); + expect(other.status).toBe(404); + expect(JSON.stringify(other.body)).not.toContain('compiled module'); + }); + + describe('a conversation still only live as an active job', () => { + const { GenerationJobManager } = require('@librechat/api'); + let liveConversationId; + + beforeEach(async () => { + liveConversationId = uuidv4(); + await mockDb.methods.saveMessage( + { userId: OWNER }, + { + messageId: 'live-response', + conversationId: liveConversationId, + isCreatedByUser: false, + content: assistantContent(), + }, + ); + }); + + afterEach(() => { + GenerationJobManager.getJob.mockResolvedValue(null); + }); + + const partUrl = () => + `/api/messages/${liveConversationId}/live-response/parts/1?toolCallId=call_bash&agentId=agent_a`; + + it('serves the part to the job owner, as the conversation read does', async () => { + GenerationJobManager.getJob.mockResolvedValue({ + status: 'running', + metadata: { userId: OWNER }, + }); + const list = await request(app).get(`/api/messages/${liveConversationId}?toolPreviews=1`); + expect(list.status).toBe(200); + expect(list.body[0].content[1].tool_call.outputTruncated).toBe(true); + + const part = await request(app).get(partUrl()); + expect(part.status).toBe(200); + expect(part.body.tool_call.output).toBe(fullOutput); + }); + + it("refuses another user's job and a job from another tenant", async () => { + GenerationJobManager.getJob.mockResolvedValue({ + status: 'running', + metadata: { userId: OTHER }, + }); + expect((await request(app).get(partUrl())).status).toBe(404); + + GenerationJobManager.getJob.mockResolvedValue({ + status: 'running', + metadata: { userId: OWNER, tenantId: 'tenant-b' }, + }); + expect((await request(app).get(partUrl())).status).toBe(404); + }); + + it('refuses once the job is no longer active', async () => { + GenerationJobManager.getJob.mockResolvedValue({ + status: 'complete', + metadata: { userId: OWNER }, + }); + expect((await request(app).get(partUrl())).status).toBe(404); + }); + }); + + it('does not serve parts of a durable child thread', async () => { + const childId = uuidv4(); + await mongoose.models.Conversation.create({ + conversationId: childId, + user: OWNER, + endpoint: 'agents', + subagentThread: { + rootConversationId: conversationId, + parentConversationId: conversationId, + parentMessageId: responseId, + parentToolCallId: 'call_sub', + subagentType: 'self', + subagentKind: 'agent', + depth: 1, + }, + }); + await mockDb.methods.saveMessage( + { userId: OWNER }, + { messageId: 'child-msg', conversationId: childId, content: assistantContent() }, + ); + const response = await request(app).get( + `/api/messages/${childId}/child-msg/parts/1?toolCallId=call_bash&agentId=agent_a`, + ); + expect(response.status).toBe(404); + }); + + describe('a client holding previews never overwrites stored content', () => { + it('editing a text part of the previewed message keeps every tool call intact', async () => { + const preview = await loadPreviews(); + expect(preview.content[1].tool_call.outputTruncated).toBe(true); + + const response = await request(app) + .put(`/api/messages/${conversationId}/${responseId}`) + .send({ text: 'Building now, edited', index: 0, model: 'gpt' }); + expect(response.status).toBe(200); + + const stored = await storedResponse(); + expect(stored.content[0].text).toBe('Building now, edited'); + await expectStoredContentIntact(); + }); + + it('editing the message text keeps every tool call intact', async () => { + await loadPreviews(); + const response = await request(app) + .put(`/api/messages/${conversationId}/${responseId}`) + .send({ text: 'new text', model: 'gpt' }); + expect(response.status).toBe(200); + await expectStoredContentIntact(); + }); + + it('refuses to save a message body that carries previews', async () => { + const preview = await loadPreviews(); + const response = await request(app) + .post(`/api/messages/${conversationId}`) + .send({ ...preview, conversationId }); + expect(response.status).toBe(400); + await expectStoredContentIntact(); + }); + + it('branching from the previewed message copies the stored content, not the preview', async () => { + await loadPreviews(); + const response = await request(app) + .post('/api/messages/branch') + .send({ messageId: responseId, agentId: 'agent_a' }); + expect(response.status).toBe(201); + expect(response.body.content[1].tool_call.output).toBe(fullOutput); + expect(response.body.content[2].tool_call.subagent_content).toEqual(transcript); + await expectStoredContentIntact(); + }); + + it('returns a branched message as a preview to a client that asks, keeping storage full', async () => { + const response = await request(app) + .post('/api/messages/branch?toolPreviews=1') + .send({ messageId: responseId, agentId: 'agent_a' }); + expect(response.status).toBe(201); + expect(response.body.content[1].tool_call.outputTruncated).toBe(true); + expect(response.body.content[2].tool_call.subagentContentOmitted).toBe(true); + const [branched] = await mockDb.methods.getMessages({ + messageId: response.body.messageId, + user: OWNER, + }); + expect(branched.content[1].tool_call.output).toBe(fullOutput); + expect(branched.content[2].tool_call.subagent_content).toEqual(transcript); + }); + + it('forking copies the stored content, not the preview', async () => { + await loadPreviews(); + const { forkConversation } = require('~/server/utils/import/fork'); + const result = await forkConversation({ + originalConvoId: conversationId, + targetMessageId: responseId, + requestUserId: OWNER, + records: true, + }); + const forked = result.messages.find((message) => message.isCreatedByUser === false); + expect(forked.content[1].tool_call.output).toBe(fullOutput); + expect(forked.content[1].tool_call.args).toBe(fullArgs); + expect(forked.content[2].tool_call.subagent_content).toEqual(transcript); + await expectStoredContentIntact(); + }); + }); +}); diff --git a/api/server/routes/__tests__/messages-update.spec.js b/api/server/routes/__tests__/messages-update.spec.js index 5394b49cb4..8ff14345bf 100644 --- a/api/server/routes/__tests__/messages-update.spec.js +++ b/api/server/routes/__tests__/messages-update.spec.js @@ -7,6 +7,10 @@ jest.mock('@librechat/agents', () => ({ })); jest.mock('@librechat/api', () => ({ + prepareToolCallPreviews: jest.fn(() => (messages) => Promise.resolve(messages)), + createToolCallPartHandler: jest.fn(() => (_req, res) => res.status(404).end()), + rejectToolCallPreviewWrites: (_req, _res, next) => next(), + withMessageToolCallPreviews: (_req, message) => message, createPrivateTextView: jest.fn(() => (_req, _res, next) => next()), stripPrivateMessageFields: jest.requireActual( '../../../../packages/api/src/protection/private/view', @@ -39,6 +43,8 @@ jest.mock('@librechat/data-schemas', () => ({ }, })); +jest.mock('~/server/services/Config', () => ({ getAppConfig: jest.fn() })); + jest.mock('~/server/services/Endpoints/agents/subagentThreadStore', () => ({})); jest.mock('~/models', () => ({ diff --git a/api/server/routes/__tests__/share.spec.js b/api/server/routes/__tests__/share.spec.js index 060c463577..0d8bb9fb7f 100644 --- a/api/server/routes/__tests__/share.spec.js +++ b/api/server/routes/__tests__/share.spec.js @@ -21,6 +21,7 @@ const mockCanAccessSharedLink = jest.fn((req, _res, next) => { next(); }); const mockGetAppConfig = jest.fn(); +const mockPrepareResultToolCallPreviews = jest.fn(() => async (result) => result); const mockShareIpLimiter = jest.fn((_req, _res, next) => next()); const mockShareUserLimiter = jest.fn((_req, _res, next) => next()); const mockParseSharedLinksPageSize = jest.fn(() => 10); @@ -78,6 +79,7 @@ const mockCreateShareContentPreflight = jest.fn((filters, options = {}) => { }); jest.mock('@librechat/api', () => ({ + prepareResultToolCallPreviews: (...args) => mockPrepareResultToolCallPreviews(...args), resolveDownloadPath: (file) => file.storageKey || file.filepath, assertModelBoundContent: (...args) => mockAssertModelBoundContent(...args), assertSharedFileMetadataAllowed: (...args) => mockAssertSharedFileMetadataAllowed(...args), @@ -1503,6 +1505,33 @@ describe('share fork route', () => { }); }); + it("shapes a cross-tenant fork with the recipient's preview bounds, not the owner's", async () => { + const forkResult = { conversation: { conversationId: 'convo-456' }, messages: [{ id: 'm' }] }; + const previewed = { ...forkResult, messages: [{ id: 'previewed' }] }; + const previewResult = jest.fn().mockResolvedValue(previewed); + mockPrepareResultToolCallPreviews.mockImplementationOnce((req, deps) => { + expect(req.user).toMatchObject({ id: 'viewer', tenantId: 'tenant-viewer' }); + expect(req.config).toEqual({ owner: true }); + deps.getAppConfig({ userId: req.user.id, tenantId: req.user.tenantId }); + return previewResult; + }); + forkSharedConversation.mockResolvedValue(forkResult); + mockShareTenantId = 'tenant-owner'; + mockSharedLinkConfigMiddleware.mockImplementationOnce((req, _res, next) => { + req.config = { owner: true }; + next(); + }); + + const response = await request( + buildApp({ user: { id: 'viewer', role: 'USER', tenantId: 'tenant-viewer' } }), + ).post('/api/share/share-123/fork?toolPreviews=1'); + + expect(response.status).toBe(201); + expect(response.body).toEqual(previewed); + expect(previewResult).toHaveBeenCalledWith(forkResult); + expect(mockGetAppConfig).toHaveBeenCalledWith({ userId: 'viewer', tenantId: 'tenant-viewer' }); + }); + it('forces snapshotFiles=false into the fork when the file snapshot kill switch is active', async () => { isFileSnapshotKillSwitchActive.mockReturnValueOnce(true); forkSharedConversation.mockResolvedValue({ diff --git a/api/server/routes/convos.js b/api/server/routes/convos.js index a8e346f425..ebb02973b9 100644 --- a/api/server/routes/convos.js +++ b/api/server/routes/convos.js @@ -36,6 +36,7 @@ const { extractStoredMessageContent, GenerationJobManager, isStopConfirmed, + withToolCallPreviews, } = require('@librechat/api'); const { logger } = require('@librechat/data-schemas'); const { CacheKeys, EModelEndpoint } = require('librechat-data-provider'); @@ -800,7 +801,7 @@ router.post('/fork', forkIpLimiter, forkUserLimiter, configMiddleware, async (re : { legacyPii: req.config.messageFilter.pii }), }); - res.json(result); + res.json(withToolCallPreviews(req, result)); } catch (error) { if (isContentFilterError(error)) { return res.status(error.statusCode).json(error.body); @@ -833,7 +834,7 @@ router.post( ? {} : { legacyPii: req.config.messageFilter.pii }), }); - res.status(201).json(result); + res.status(201).json(withToolCallPreviews(req, result)); } catch (error) { if (isContentFilterError(error)) { return res.status(error.statusCode).json(error.body); diff --git a/api/server/routes/messages.js b/api/server/routes/messages.js index f652e8523b..89019d3d9f 100644 --- a/api/server/routes/messages.js +++ b/api/server/routes/messages.js @@ -30,8 +30,13 @@ const { createPrivateTextView, stripPrivateMessageFields, applyForcedRetention, + prepareToolCallPreviews, + createToolCallPartHandler, + rejectToolCallPreviewWrites, + withMessageToolCallPreviews, } = require('@librechat/api'); const subagentThreadTaskStore = require('~/server/services/Endpoints/agents/subagentThreadStore'); +const { getAppConfig } = require('~/server/services/Config'); const { findAllArtifacts, replaceArtifactContent } = require('~/server/services/Artifacts/update'); const { requireJwtAuth, @@ -39,6 +44,7 @@ const { configMiddleware, sendValidationResponse, canReadActiveJobConversation, + createMessageRequestValidation, prepareMessageRequestValidation, } = require('~/server/middleware'); const db = require('~/models'); @@ -60,9 +66,16 @@ const filterFeedbackContent = createContentFilter({ getFilters: (req) => req.config?.filters, extract: (req) => extractFeedbackContent(req.body), }); +const toolCallPreviewDeps = { getAppConfig }; +const readToolCallPart = createToolCallPartHandler({ + getMessages: db.getMessages, + validate: (req) => createMessageRequestValidation(req, { activeJobMessageRead: true }), + sendValidationResponse: (res, result) => sendValidationResponse(res, result), +}); const messageMutationMiddleware = [validateMessageReq, configMiddleware]; const storedMessageMutationMiddleware = [ validateMessageReq, + rejectToolCallPreviewWrites, configMiddleware, filterStoredMessageContent, ]; @@ -123,6 +136,7 @@ router.get('/', async (req, res) => { const sortOrder = sortDirection === 'asc' ? 1 : -1; let scopedMessageRead; + const previewToolCalls = prepareToolCallPreviews(req, toolCallPreviewDeps); if (typeof conversationId === 'string') { const ownershipRead = db.getConvoOwnership(user, conversationId); /** Client-facing reads never expose server-private fields such as `contextMeta`. */ @@ -155,13 +169,19 @@ router.get('/', async (req, res) => { throw messageResult.error; } const messages = messageResult.value; - response = { messages: messages?.length ? [messages[0]] : [], nextCursor: null }; + response = { + messages: await previewToolCalls(messages?.length ? [messages[0]] : []), + nextCursor: null, + }; } else if (conversationId) { const messageResult = await scopedMessageRead; if (!messageResult.ok) { throw messageResult.error; } - response = messageResult.value; + response = { + ...messageResult.value, + messages: await previewToolCalls(messageResult.value.messages), + }; } else if (search) { const searchResults = await db.searchMessages( search, @@ -393,7 +413,7 @@ router.post('/branch', configMiddleware, async (req, res) => { conversationId: sourceMessage.conversationId, }); - res.status(201).json(toClientMessage(savedMessage)); + res.status(201).json(withMessageToolCallPreviews(req, toClientMessage(savedMessage))); } catch (error) { if (isContentFilterError(error)) { return res.status(error.statusCode).json(error.body); @@ -498,11 +518,13 @@ router.post('/artifact/:messageId', configMiddleware, async (req, res) => { conversationId: message.conversationId, }); - res.status(200).json({ - conversationId: savedMessage.conversationId, - content: savedMessage.content, - text: savedMessage.text, - }); + res.status(200).json( + withMessageToolCallPreviews(req, { + conversationId: savedMessage.conversationId, + content: savedMessage.content, + text: savedMessage.text, + }), + ); } catch (error) { if (isContentFilterError(error)) { return res.status(error.statusCode).json(error.body); @@ -516,6 +538,7 @@ router.get('/:conversationId', prepareMessageRequestValidation, async (req, res) try { const { conversationId } = req.params; const validation = req.messageRequestValidation; + const previewToolCalls = prepareToolCallPreviews(req, toolCallPreviewDeps); // This intentionally starts a user-scoped read before validation resolves; // the response remains gated on validation success below. const messagesPromise = validation.shouldFetchMessages @@ -536,7 +559,7 @@ router.get('/:conversationId', prepareMessageRequestValidation, async (req, res) } const messages = messagesResult?.messages ?? []; - res.status(200).json(messages); + res.status(200).json(await previewToolCalls(messages)); } catch (error) { logger.error('Error fetching messages:', error); res.status(500).json({ error: 'Internal server error' }); @@ -604,6 +627,8 @@ router.get('/:conversationId/:messageId', validateMessageReq, async (req, res) = } }); +router.get('/:conversationId/:messageId/parts/:partIndex', readToolCallPart); + router.put('/:conversationId/:messageId', messageMutationMiddleware, async (req, res) => { try { const { conversationId, messageId } = req.params; diff --git a/api/server/routes/share.js b/api/server/routes/share.js index b64c278f70..ad09159a53 100644 --- a/api/server/routes/share.js +++ b/api/server/routes/share.js @@ -26,6 +26,7 @@ const { recordShareLinkRejection, traceIdForMessage, resolveDownloadPath, + prepareResultToolCallPreviews, } = require('@librechat/api'); const { logger, @@ -424,6 +425,7 @@ if (allowSharedLinks) { sharedLinkConfigMiddleware, async (req, res) => { try { + const previewToolCalls = prepareResultToolCallPreviews(req, { getAppConfig }); const result = await forkSharedConversation({ shareId: req.params.shareId, shareResourceId: req.shareResourceId, @@ -444,7 +446,7 @@ if (allowSharedLinks) { if (!result) { return res.status(404).json({ message: 'Shared conversation not found' }); } - return res.status(201).json(result); + return res.status(201).json(await previewToolCalls(result)); } catch (error) { if (isContentFilterError(error)) { return res.status(error.statusCode).json(error.body); diff --git a/client/src/components/Chat/Messages/Content/AgentHandoff.tsx b/client/src/components/Chat/Messages/Content/AgentHandoff.tsx index b4c0004273..6e77e64911 100644 --- a/client/src/components/Chat/Messages/Content/AgentHandoff.tsx +++ b/client/src/components/Chat/Messages/Content/AgentHandoff.tsx @@ -5,10 +5,14 @@ import { Button } from '@librechat/client'; import { EModelEndpoint, Constants } from 'librechat-data-provider'; import type { TMessage } from 'librechat-data-provider'; import type { TranslationKeys } from '~/hooks'; +import { + useToolContentRequest, + useToolContentPending, + toolPanelSpacingClassName, +} from './disclosure'; import CopyButton from '~/components/Messages/Content/CopyButton'; import { unescapeJsonString } from './Parts/parseJsonField'; import MessageIcon from '~/components/Share/MessageIcon'; -import { toolPanelSpacingClassName } from './disclosure'; import { useLocalize, useExpandCollapse } from '~/hooks'; import { useAgentsMapContext } from '~/Providers'; import { cn } from '~/utils'; @@ -107,6 +111,7 @@ const AgentHandoff: React.FC = ({ name, args: _args = '' }) = const localize = useLocalize(); const agentsMap = useAgentsMapContext(); const [showInfo, setShowInfo] = useState(false); + useToolContentRequest(showInfo); const [isCopied, setIsCopied] = useState(false); const copiedTimerRef = useRef>(); const contentId = useId(); @@ -147,21 +152,22 @@ const AgentHandoff: React.FC = ({ name, args: _args = '' }) = * non-secure origin (a LAN deployment served over http), where * dereferencing it throws before any rejection handler can run and the * button silently does nothing. */ + const contentPending = useToolContentPending(); const handleCopy = useCallback(() => { - if (!copy(copyText, { format: 'text/plain' })) { + if (contentPending || !copy(copyText, { format: 'text/plain' })) { return; } clearTimeout(copiedTimerRef.current); setIsCopied(true); copiedTimerRef.current = setTimeout(() => setIsCopied(false), 2000); - }, [copyText]); + }, [contentPending, copyText]); return (
+ {String(toolCall.output)} + + {String('outputTruncated' in toolCall || 'argsTruncated' in toolCall)} + + {String(pending)} +
+ ); +} + +function renderPart( + part: TMessageContentParts, + options: { sole?: boolean; partIndex?: number; queryClient?: QueryClient } = {}, +) { + const queryClient = + options.queryClient ?? new QueryClient({ defaultOptions: { queries: { retry: false } } }); + const partIndex = options.partIndex ?? 2; + const disclosures: ToolDisclosures = new Map(); + const tree = ( + + + + + + + {isPreviewedToolCallPart(part) ? ( + + {(rendered) => } + + ) : ( + + )} + + + + + + + ); + return { ...render(tree), queryClient }; +} + +describe('previewed tool-call parts', () => { + const getToolCallPart = dataService.getToolCallPart as jest.Mock; + + beforeEach(() => { + getToolCallPart.mockReset(); + }); + + it('loads nothing while the card stays collapsed', () => { + getToolCallPart.mockResolvedValue(stored()); + renderPart(previewPart()); + expect(screen.getByTestId('output')).toHaveTextContent('…'); + expect(getToolCallPart).not.toHaveBeenCalled(); + }); + + it('fetches the stored part when the card opens, showing a status until it arrives', async () => { + let resolve: (value: ToolCallPartResponse) => void = () => undefined; + getToolCallPart.mockReturnValue(new Promise((done) => (resolve = done))); + renderPart(previewPart()); + + fireEvent.click(screen.getByText('toggle')); + expect(await screen.findByRole('status')).toHaveTextContent('com_ui_tool_content_loading'); + expect(getToolCallPart).toHaveBeenCalledWith({ + conversationId: 'convo-1', + messageId: 'msg-1', + partIndex: 2, + toolCallId: 'call_1', + }); + + expect(screen.getByTestId('pending')).toHaveTextContent('true'); + await act(async () => resolve(stored())); + await waitFor(() => expect(screen.getByTestId('output').textContent).toBe(fullOutput)); + expect(screen.getByTestId('pending')).toHaveTextContent('false'); + expect(screen.getByTestId('markers')).toHaveTextContent('false'); + expect(screen.queryByRole('status')).not.toBeInTheDocument(); + }); + + it('shows an error with a retry that loads the part on the next attempt', async () => { + getToolCallPart + .mockRejectedValueOnce(new Error('offline')) + .mockRejectedValueOnce(new Error('offline')) + .mockResolvedValueOnce(stored()); + renderPart(previewPart()); + + fireEvent.click(screen.getByText('toggle')); + const alert = await screen.findByRole('alert', undefined, { timeout: 5_000 }); + expect(alert).toHaveTextContent('com_ui_tool_content_error'); + expect(screen.getByTestId('output')).toHaveTextContent('…'); + + fireEvent.click(screen.getByText('com_ui_retry')); + await waitFor(() => expect(screen.getByTestId('output').textContent).toBe(fullOutput)); + expect(screen.queryByRole('alert')).not.toBeInTheDocument(); + expect(getToolCallPart).toHaveBeenCalledTimes(3); + }); + + it('fetches once for a card that opens on its own, then serves reopenings from cache', async () => { + getToolCallPart.mockResolvedValue(stored()); + renderPart(previewPart(), { sole: true }); + await waitFor(() => expect(screen.getByTestId('output').textContent).toBe(fullOutput)); + + fireEvent.click(screen.getByText('toggle')); + fireEvent.click(screen.getByText('toggle')); + expect(getToolCallPart).toHaveBeenCalledTimes(1); + }); + + it('serves the same call from cache when client-only cards shift its index', async () => { + getToolCallPart.mockResolvedValue(stored()); + const first = renderPart(previewPart(), { sole: true }); + await waitFor(() => expect(screen.getByTestId('output').textContent).toBe(fullOutput)); + first.unmount(); + + renderPart(previewPart(), { sole: true, partIndex: 3, queryClient: first.queryClient }); + expect(screen.getByTestId('output').textContent).toBe(fullOutput); + expect(getToolCallPart).toHaveBeenCalledTimes(1); + }); + + it('leaves a part the server sent in full alone', () => { + const part = { + type: ContentTypes.TOOL_CALL, + tool_call: { id: 'call_1', name: 'bash_tool', args: '{}', output: 'done', progress: 1 }, + } as TMessageContentParts; + expect(isPreviewedToolCallPart(part)).toBe(false); + renderPart(part, { sole: true }); + expect(screen.getByTestId('output')).toHaveTextContent('done'); + expect(getToolCallPart).not.toHaveBeenCalled(); + }); +}); + +describe('previewed tool-call parts after the stored call changes', () => { + const getToolCallPart = dataService.getToolCallPart as jest.Mock; + + beforeEach(() => { + getToolCallPart.mockReset(); + }); + + it('fetches again when the preview it was fetched for is replaced', async () => { + getToolCallPart + .mockResolvedValueOnce(stored()) + .mockResolvedValueOnce(stored({ output: `${fullOutput}\nharvested` })); + const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + const disclosures: ToolDisclosures = new Map(); + const tree = (part: TMessageContentParts) => ( + + + + + + + + {(rendered) => } + + + + + + + + ); + const { rerender } = render(tree(previewPart())); + await waitFor(() => expect(screen.getByTestId('output').textContent).toBe(fullOutput)); + + rerender( + tree( + previewPart({ + output: 'stdout:\nfull line\n…\nharvested', + outputLength: fullOutput.length + 10, + }), + ), + ); + await waitFor(() => + expect(screen.getByTestId('output').textContent).toBe(`${fullOutput}\nharvested`), + ); + expect(getToolCallPart).toHaveBeenCalledTimes(2); + }); +}); + +describe('withFullToolCall', () => { + it('keeps fields the preview carried in full, so a fetched copy cannot roll them back', () => { + const part = previewPart({ output: 'current output', outputTruncated: undefined }); + const merged = withFullToolCall(part as never, { ...stored().tool_call, output: 'stale' }); + expect((merged.tool_call as { output: string }).output).toBe('current output'); + expect((merged.tool_call as { args: string }).args).toBe(fullArgs); + }); + + it('replaces only content fields and drops the markers', () => { + const part = previewPart({ subagentContentOmitted: true, subagentContentParts: 1 }); + const transcript = [{ type: ContentTypes.TEXT, text: 'child' }] as TMessageContentParts[]; + const merged = withFullToolCall(part as never, { + ...stored().tool_call, + subagent_content: transcript, + }); + expect(merged.tool_call).toEqual({ + id: 'call_1', + type: 'tool_call', + name: 'bash_tool', + args: fullArgs, + output: fullOutput, + progress: 1, + subagent_content: transcript, + }); + }); +}); diff --git a/client/src/components/Chat/Messages/Content/disclosure.ts b/client/src/components/Chat/Messages/Content/disclosure.ts index cc7cba43aa..ec08a1b99c 100644 --- a/client/src/components/Chat/Messages/Content/disclosure.ts +++ b/client/src/components/Chat/Messages/Content/disclosure.ts @@ -1,4 +1,4 @@ -import { createContext, useContext, useMemo } from 'react'; +import { createContext, useContext, useEffect, useMemo } from 'react'; import { atom, useAtom } from 'jotai'; import { useRecoilValue } from 'recoil'; import type { PrimitiveAtom } from 'jotai'; @@ -70,6 +70,28 @@ export function useToolAutoExpand() { return autoExpand || soleTool === true; } +/** Asks the enclosing tool-call part to load its stored content in place of a server preview. + * `null` outside a part that holds a preview. */ +export const ToolContentRequestContext = createContext<(() => void) | null>(null); + +/** True while the enclosing part still shows a server preview, so content actions (copy) would + * hand out shortened text. False outside a previewed part. */ +export const ToolContentPendingContext = createContext(false); + +export function useToolContentPending(): boolean { + return useContext(ToolContentPendingContext); +} + +/** Requests the part's full content while `active`: its card is open or its panel is showing. */ +export function useToolContentRequest(active: boolean) { + const request = useContext(ToolContentRequestContext); + useEffect(() => { + if (active) { + request?.(); + } + }, [active, request]); +} + /** A tool card's disclosure. The reader's explicit choice lives in the * per-tool atom, so it survives the card remounting when a live batch * regroups. Until they choose, the card follows `useToolAutoExpand` and @@ -78,5 +100,6 @@ export function useToolExpansion(canExpand: boolean) { const autoExpand = useToolAutoExpand(); const [override, setOverride] = useAtom(useToolDisclosure()); const expanded = override ?? (autoExpand && canExpand); + useToolContentRequest(expanded); return [expanded, setOverride] as const; } diff --git a/client/src/components/Chat/Messages/Content/hydration.tsx b/client/src/components/Chat/Messages/Content/hydration.tsx new file mode 100644 index 0000000000..d91dae867f --- /dev/null +++ b/client/src/components/Chat/Messages/Content/hydration.tsx @@ -0,0 +1,125 @@ +import { useCallback, useMemo, useState } from 'react'; +import { Spinner } from '@librechat/client'; +import { + ContentTypes, + hasToolCallPreview, + getToolCallPreviewRevision, +} from 'librechat-data-provider'; +import type { FullToolCall, TMessageContentParts } from 'librechat-data-provider'; +import type { ReactNode } from 'react'; +import { ToolContentRequestContext, ToolContentPendingContext } from './disclosure'; +import { useToolCallPartQuery } from '~/data-provider'; +import { useMessageContext } from '~/Providers'; +import { useLocalize } from '~/hooks'; + +type ToolCallPart = Extract; + +/** A tool-call part whose content the server sent as a preview. */ +export function isPreviewedToolCallPart(part: TMessageContentParts): part is ToolCallPart { + return ( + part.type === ContentTypes.TOOL_CALL && + part.tool_call != null && + hasToolCallPreview(part.tool_call as FullToolCall) + ); +} + +/** + * The part with its stored content in place of the preview. Only the fields the server shortened + * come from the fetched copy, and the preview markers go: everything else (progress, status, + * fields the preview already carried in full, client-only state) stays as the conversation cache + * has it, so a fetched copy can never roll back a newer cached value. + */ +export function withFullToolCall(part: ToolCallPart, full: FullToolCall): ToolCallPart { + const { + outputTruncated, + outputLength: _outputLength, + argsTruncated, + argsLength: _argsLength, + subagentContentOmitted, + subagentContentParts: _subagentContentParts, + previewRevision: _previewRevision, + ...toolCall + } = part.tool_call as FullToolCall; + return { + ...part, + tool_call: { + ...toolCall, + ...(argsTruncated === true && full.args != null ? { args: full.args } : {}), + ...(outputTruncated === true && full.output != null ? { output: full.output } : {}), + ...(subagentContentOmitted === true && full.subagent_content != null + ? { subagent_content: full.subagent_content } + : {}), + }, + } as ToolCallPart; +} + +/** + * Renders a previewed tool-call part. Nothing loads until the reader opens the card (or its + * subagent panel); then the stored part is fetched once, cached under its own key, and rendered + * in place of the preview, with a status line while it loads or if it fails. + */ +export function PreviewedToolCallPart({ + part, + children, +}: { + part: ToolCallPart; + children: (part: TMessageContentParts) => ReactNode; +}) { + const localize = useLocalize(); + const { messageId, conversationId, partIndex } = useMessageContext(); + const [requested, setRequested] = useState(false); + const request = useCallback(() => setRequested(true), []); + const toolCall = part.tool_call as FullToolCall; + const revision = useMemo(() => getToolCallPreviewRevision(toolCall), [toolCall]); + const canFetch = !!conversationId && !!messageId && partIndex != null; + const query = useToolCallPartQuery( + { + conversationId: conversationId ?? '', + messageId: messageId ?? '', + partIndex: partIndex ?? 0, + toolCallId: toolCall.id, + stepId: toolCall.stepId, + agentId: part.agentId, + }, + { enabled: requested && canFetch }, + revision, + ); + const full = query.data?.tool_call; + const effectivePart = useMemo( + () => (full == null ? part : withFullToolCall(part, full)), + [full, part], + ); + const retry = useCallback(() => { + query.refetch(); + }, [query]); + + return ( + + + {children(effectivePart)} + + {requested && full == null && query.isError && ( +
+ {localize('com_ui_tool_content_error')} + +
+ )} + {requested && full == null && !query.isError && canFetch && ( +
+
+ )} +
+ ); +} diff --git a/client/src/components/Chat/Subagents/SubagentThreadPanel.test.tsx b/client/src/components/Chat/Subagents/SubagentThreadPanel.test.tsx index 57e110b813..120bebc4e7 100644 --- a/client/src/components/Chat/Subagents/SubagentThreadPanel.test.tsx +++ b/client/src/components/Chat/Subagents/SubagentThreadPanel.test.tsx @@ -18,6 +18,14 @@ import SubagentThreadPanel from './SubagentThreadPanel'; import { getDraft } from '~/utils'; const mockUseSubagentThreadQuery = jest.fn(); +const mockUseToolCallPartQuery = jest.fn( + ( + ..._args: unknown[] + ): { data?: unknown; isError: boolean; refetch?: () => Promise } => ({ + data: undefined, + isError: false, + }), +); const mockUseSubagentActivityStream = jest.fn(); const mockForkMutate = jest.fn(); const mockControlMutate = jest.fn(); @@ -83,6 +91,7 @@ jest.mock('~/data-provider', () => ({ * (`useConfiguredFooter`), so the panel now reads the startup config. */ useGetStartupConfig: () => ({ data: undefined }), useSubagentThreadQuery: (...args: unknown[]) => mockUseSubagentThreadQuery(...args), + useToolCallPartQuery: (...args: unknown[]) => mockUseToolCallPartQuery(...args), subagentThreadHasTaskEvidence: (view: SubagentThreadView | undefined, taskId: string): boolean => view?.messages.some( (message) => @@ -2191,6 +2200,123 @@ describe('SubagentThreadPanel', () => { ).not.toBeInTheDocument(); }); + describe('a foreground call the server sent as a preview', () => { + const previewed: ActiveSubagentPanel = { + host: 'conversation', + parentConversationId: 'parent-conversation', + parentMessageId: 'parent-message', + toolCallId: 'foreground-call', + partIndex: 3, + subagentType: 'researcher', + prompt: 'Review this change.', + legacyOutput: 'Shortened final ans…', + contentPreview: { revision: 'rev-1', stepId: 'step-9', agentId: 'agent-2' }, + initialProgress: 1, + isSubmitting: false, + }; + + beforeEach(() => { + mockUseSubagentThreadQuery.mockReturnValue({ + data: undefined, + isLoading: false, + isError: false, + isReadinessPending: false, + }); + }); + + afterEach(() => { + mockUseToolCallPartQuery.mockImplementation(() => ({ data: undefined, isError: false })); + }); + + it('loads the stored part and never shows the shortened output as the activity', () => { + render( + + + , + ); + + expect(mockUseToolCallPartQuery).toHaveBeenCalledWith( + { + conversationId: 'parent-conversation', + messageId: 'parent-message', + partIndex: 3, + toolCallId: 'foreground-call', + stepId: 'step-9', + agentId: 'agent-2', + }, + { enabled: true }, + 'rev-1', + ); + expect(screen.getByTestId('subagent-conversation')).toHaveAttribute('data-state', 'loading'); + expect(screen.queryByText('Shortened final ans…')).not.toBeInTheDocument(); + }); + + it('renders the stored transcript once it arrives', () => { + mockUseToolCallPartQuery.mockImplementation(() => ({ + data: { + conversationId: 'parent-conversation', + messageId: 'parent-message', + partIndex: 3, + tool_call: { + id: 'foreground-call', + name: 'subagent', + args: '{"prompt":"Review this change, including the migration and its rollback."}', + output: 'Full final answer.', + subagent_content: [{ type: 'text', text: 'Stored review transcript.' }], + }, + }, + isError: false, + })); + + render( + + + , + ); + + expect(screen.getByText('Stored review transcript.')).toBeInTheDocument(); + expect(screen.getByTestId('subagent-conversation')).toHaveAttribute('data-state', 'ready'); + expect( + screen.getByText('Review this change, including the migration and its rollback.'), + ).toBeInTheDocument(); + expect(screen.queryByText('Review this change.')).not.toBeInTheDocument(); + }); + + it('reports a failed load with a retry instead of rendering the preview', () => { + const refetch = jest.fn().mockResolvedValue(undefined); + mockUseToolCallPartQuery.mockImplementation(() => ({ + data: undefined, + isError: true, + refetch, + })); + + render( + + + , + ); + + expect(screen.getByTestId('subagent-conversation')).toHaveAttribute('data-state', 'error'); + expect(screen.queryByText('Shortened final ans…')).not.toBeInTheDocument(); + expect(screen.getByRole('alert')).toHaveTextContent('com_ui_tool_content_error'); + fireEvent.click(screen.getByRole('button', { name: 'com_ui_retry' })); + expect(refetch).toHaveBeenCalledTimes(1); + }); + + it('reads nothing for a call that arrived in full', () => { + render( + + + , + ); + expect(mockUseToolCallPartQuery).toHaveBeenCalledWith( + expect.anything(), + { enabled: false }, + undefined, + ); + }); + }); + it('continues a completed durable agent task as an ordinary conversation snapshot', () => { mockUseSubagentThreadQuery.mockReturnValue({ data: completedView, diff --git a/client/src/components/Chat/Subagents/SubagentThreadPanel.tsx b/client/src/components/Chat/Subagents/SubagentThreadPanel.tsx index fcbc10745d..85eef79d30 100644 --- a/client/src/components/Chat/Subagents/SubagentThreadPanel.tsx +++ b/client/src/components/Chat/Subagents/SubagentThreadPanel.tsx @@ -22,6 +22,14 @@ import type { KeyboardEvent as ReactKeyboardEvent, ReactNode } from 'react'; import type { ActiveSubagentPanel, SubagentControlUiState } from './state'; import type { ComposerKeyAction } from '~/utils/shortcuts'; import type { OptionWithIcon } from '~/common'; +import { + ACTIVE_THREAD_REFRESH_MS, + subagentThreadHasTaskEvidence, + useForkConvoMutation, + useToolCallPartQuery, + useSubagentControlMutation, + useSubagentThreadQuery, +} from '~/data-provider'; import { adaptDurableThreadActivity, adaptDurableThreadConversation, @@ -29,13 +37,6 @@ import { mergeChildConversationTurns, retainBoundedMovingWindowTurns, } from './adapters'; -import { - ACTIVE_THREAD_REFRESH_MS, - subagentThreadHasTaskEvidence, - useForkConvoMutation, - useSubagentControlMutation, - useSubagentThreadQuery, -} from '~/data-provider'; import { agentAuthor, resolveSubagentAuthor, @@ -66,6 +67,7 @@ import { resolveSubagentAgentId } from './identity'; import { useAgentsMapContext } from '~/Providers'; import { isLiveSubagentStatus } from './status'; import { cn, renderAgentAvatar } from '~/utils'; +import { getSubagentPrompt } from './prompt'; import { useChatSurface } from './surface'; const EVENT_TASK_PAGE_SIZE = 3; @@ -808,22 +810,55 @@ export default function SubagentThreadPanel({ selection }: { selection: ActiveSu }; }, [isMobile]); + /** A previewed call renders nothing until its stored part arrives: the preview's output is a + * shortened copy and its transcript is absent, so neither may stand in for the activity. */ + const awaitsStoredPart = selection.contentPreview != null; + const storedPart = useToolCallPartQuery( + { + conversationId: selection.parentConversationId, + messageId: selection.parentMessageId, + partIndex: selection.partIndex, + toolCallId: selection.toolCallId || undefined, + stepId: selection.contentPreview?.stepId, + agentId: selection.contentPreview?.agentId, + }, + { enabled: awaitsStoredPart }, + selection.contentPreview?.revision, + ); + const storedToolCall = awaitsStoredPart ? storedPart.data?.tool_call : undefined; + const persistedContent = storedToolCall?.subagent_content ?? selection.persistedContent; + const prompt = + (storedToolCall == null ? undefined : getSubagentPrompt(storedToolCall.args)) ?? + selection.prompt; + let legacyOutput = selection.legacyOutput; + if (awaitsStoredPart && legacyOutput != null) { + legacyOutput = storedToolCall?.output; + } + const liveActivity = useMemo( () => adaptLivePersistedActivity({ title: foregroundTitle, - prompt: selection.prompt, + prompt, progress, - persistedContent: selection.persistedContent, + persistedContent, isDetached: selection.durable != null, - legacyOutput: selection.legacyOutput, + legacyOutput, // A detached parent tool step closes as soon as dispatch succeeds; // its terminal status does not describe the still-running child. initialProgress: selection.durable == null ? selection.initialProgress : 0, isSubmitting: selection.durable == null ? selection.isSubmitting : detachedLiveSubmitting, runStepStatus: selection.durable == null ? selection.runStepStatus : undefined, }), - [detachedLiveSubmitting, foregroundTitle, progress, selection], + [ + detachedLiveSubmitting, + foregroundTitle, + legacyOutput, + persistedContent, + progress, + prompt, + selection, + ], ); const activity = useMemo(() => { if (selection.durable == null) return liveActivity; @@ -995,7 +1030,7 @@ export default function SubagentThreadPanel({ selection }: { selection: ActiveSu selection.event == null ? ('parent_continuation' as const) : ('external_event' as const), - summary: selection.prompt ?? activity.prompt ?? '', + summary: prompt ?? activity.prompt ?? '', ...(selectedTaskCreatedAt == null ? {} : { createdAt: selectedTaskCreatedAt }), }, activity, @@ -1022,7 +1057,7 @@ export default function SubagentThreadPanel({ selection }: { selection: ActiveSu trigger: { kind: selection.event == null ? ('parent_dispatch' as const) : ('external_event' as const), - summary: selection.prompt ?? activity.prompt ?? '', + summary: prompt ?? activity.prompt ?? '', }, activity, }, @@ -1034,6 +1069,7 @@ export default function SubagentThreadPanel({ selection }: { selection: ActiveSu movingWindowTurns, olderTurns, postRebaseTurns, + prompt, rebaseTurns, retainedTurnsValid, threadId, @@ -1303,7 +1339,9 @@ export default function SubagentThreadPanel({ selection }: { selection: ActiveSu ], ); let panelState: 'ready' | 'loading' | 'error' = 'ready'; - if ( + if (awaitsStoredPart && selection.durable == null && storedToolCall == null) { + panelState = storedPart.isError ? 'error' : 'loading'; + } else if ( selection.durable != null && liveActivity.items.length === 0 && (isLoading || isReadinessPending) @@ -1381,11 +1419,27 @@ export default function SubagentThreadPanel({ selection }: { selection: ActiveSu )} ) : null; + const storedPartRetry = + awaitsStoredPart && storedToolCall == null && storedPart.isError ? ( +
+ {localize('com_ui_tool_content_error')} + +
+ ) : null; + const activityNotice = + droppedNotice == null && storedPartRetry == null ? null : ( + <> + {droppedNotice} + {storedPartRetry} + + ); let activityPanel: ReactNode; if (hasConversationProjection) { activityPanel = ( - {droppedNotice} + {activityNotice} {showUnavailableHistoryBoundary && (
- {droppedNotice} + {activityNotice}
{timelinePrefix} {visibleEventTasks.map(renderEventTask)} @@ -1461,7 +1515,7 @@ export default function SubagentThreadPanel({ selection }: { selection: ActiveSu state={panelState} showPrompt={false} headerInset - notice={droppedNotice} + notice={activityNotice} onCancelControl={ controlAvailable && !controlPending ? (controlId) => submitControl('cancel_message', controlId) diff --git a/client/src/components/Chat/Subagents/prompt.ts b/client/src/components/Chat/Subagents/prompt.ts new file mode 100644 index 0000000000..8a8f716e36 --- /dev/null +++ b/client/src/components/Chat/Subagents/prompt.ts @@ -0,0 +1,25 @@ +const PROMPT_KEYS = ['prompt', 'description', 'task', 'instructions'] as const; + +/** The task a subagent call was given, read from its arguments (JSON text or parsed). */ +export function getSubagentPrompt( + args: string | Record | null | undefined, +): string | undefined { + let parsed: unknown = args; + if (typeof args === 'string') { + try { + parsed = JSON.parse(args); + } catch { + return undefined; + } + } + if (parsed == null || typeof parsed !== 'object') { + return undefined; + } + for (const key of PROMPT_KEYS) { + const value = (parsed as Record)[key]; + if (typeof value === 'string' && value.trim().length > 0) { + return value; + } + } + return undefined; +} diff --git a/client/src/components/Chat/Subagents/state.ts b/client/src/components/Chat/Subagents/state.ts index 8373c3d6a3..3f396ec1bf 100644 --- a/client/src/components/Chat/Subagents/state.ts +++ b/client/src/components/Chat/Subagents/state.ts @@ -284,6 +284,12 @@ const eventKey = (event: SubagentUpdateEvent): string | undefined => { return hashString(`${event.subagentRunId}\u0000${activityEventId}`); }; +export type SubagentContentPreview = { + revision: string; + stepId?: string; + agentId?: string; +}; + /** One child invocation selected for the shared read-only activity panel. */ export type ActiveSubagentPanel = { host: 'conversation' | 'share'; @@ -297,6 +303,10 @@ export type ActiveSubagentPanel = { prompt?: string; legacyOutput?: string | null; persistedContent?: TMessageContentParts[]; + /** The tool call came from the server as a preview: its transcript, output or arguments are + * shortened or absent, so the panel loads the stored part before rendering activity. Carries + * what identifies the stored part and the preview's revision. */ + contentPreview?: SubagentContentPreview; initialProgress: number; isSubmitting: boolean; runStepStatus?: PartMetadata['runStepStatus']; diff --git a/client/src/components/Chat/Trace/Inspector.tsx b/client/src/components/Chat/Trace/Inspector.tsx index 8704ab99b2..1f503c4906 100644 --- a/client/src/components/Chat/Trace/Inspector.tsx +++ b/client/src/components/Chat/Trace/Inspector.tsx @@ -165,12 +165,15 @@ function RawContent({ const COPIED_MS = 2000; const CALL_CONTENT_LENGTH = 4000; -function toContent(value?: string): TTraceContent | undefined { +function toContent(value?: string, previewed = false): TTraceContent | undefined { if (value == null) { return undefined; } const truncated = value.length > CALL_CONTENT_LENGTH; - return { value: truncated ? value.slice(0, CALL_CONTENT_LENGTH) : value, truncated }; + return { + value: truncated ? value.slice(0, CALL_CONTENT_LENGTH) : value, + truncated: truncated || previewed, + }; } /** The saved agent a record ran: who it is, its id to copy, and a chat with it one click away. */ @@ -245,8 +248,14 @@ function ToolCalls({ )} - - + +
))} {fromConversation && ( diff --git a/client/src/components/Chat/Trace/Viewer.tsx b/client/src/components/Chat/Trace/Viewer.tsx index 1c9e207ae8..580ea6086b 100644 --- a/client/src/components/Chat/Trace/Viewer.tsx +++ b/client/src/components/Chat/Trace/Viewer.tsx @@ -2,7 +2,7 @@ import { useId, useRef, useMemo, useState, useEffect, useCallback, useDeferredVa import axios from 'axios'; import { useAtom } from 'jotai'; import { useQuery, useQueryClient } from '@tanstack/react-query'; -import { QueryKeys, dataService, resolveTraceViewerConfig } from 'librechat-data-provider'; +import { QueryKeys, resolveTraceViewerConfig } from 'librechat-data-provider'; import { Button, Spinner, EmptyState, FilterInput, buttonVariants } from '@librechat/client'; import { X, @@ -29,6 +29,7 @@ import type { TranslationKeys } from '~/hooks'; import { keepNewestTracePage, useGetStartupConfig, + fetchConversationMessages, useGetLangfuseSessionLinkQuery, useConversationTraceRecordsQuery, } from '~/data-provider'; @@ -112,7 +113,7 @@ export default function Viewer({ /** The chat's own messages, already loaded underneath the trace; they are only read, never fetched here. */ const { data: messages } = useQuery( [QueryKeys.messages, conversationId], - () => dataService.getMessagesByConvoId(conversationId), + () => fetchConversationMessages(conversationId), { enabled: false }, ); diff --git a/client/src/components/Chat/Trace/preview.ts b/client/src/components/Chat/Trace/preview.ts index f2165c9828..c20b183b0d 100644 --- a/client/src/components/Chat/Trace/preview.ts +++ b/client/src/components/Chat/Trace/preview.ts @@ -9,7 +9,15 @@ const PREVIEW_LENGTH = 160; const ELLIPSIS = '…'; /** `args` is the one-line preview; `input` and `output` are what the chat's own tool card holds. */ -export type ToolCallPreview = { name: string; args: string; input?: string; output?: string }; +export type ToolCallPreview = { + name: string; + args: string; + input?: string; + output?: string; + /** The conversation holds a server preview of the input or output, not the stored value. */ + inputTruncated?: boolean; + outputTruncated?: boolean; +}; /** What one model call of a response produced: the text it wrote and the tools it called. */ export type StepPreview = { text: string; toolCalls: ToolCallPreview[] }; @@ -89,6 +97,10 @@ function callOf(call: ToolCallPart | undefined): ToolCallPreview | null { args: argsPreview(call.args), input: contentOf(call.args), output: 'output' in call ? contentOf(call.output) : undefined, + ...('argsTruncated' in call && call.argsTruncated === true ? { inputTruncated: true } : {}), + ...('outputTruncated' in call && call.outputTruncated === true + ? { outputTruncated: true } + : {}), }; } return null; diff --git a/client/src/components/Conversations/ConvoOptions/ShareButton.tsx b/client/src/components/Conversations/ConvoOptions/ShareButton.tsx index 7bb8f3d966..466ac02e1b 100644 --- a/client/src/components/Conversations/ConvoOptions/ShareButton.tsx +++ b/client/src/components/Conversations/ConvoOptions/ShareButton.tsx @@ -17,8 +17,8 @@ import { OGDialogDescription, } from '@librechat/client'; import { useGetLatestMessage, useLatestMessageId } from '~/hooks/Messages/useLatestMessage'; +import { useGetStartupConfig, fetchConversationMessages } from '~/data-provider'; import SharedLinkCopyButton from './SharedLinkCopyButton'; -import { useGetStartupConfig } from '~/data-provider'; import SharedLinkButton from './SharedLinkButton'; import { buildShareLinkUrl } from '~/utils'; import { useLocalize } from '~/hooks'; @@ -65,7 +65,7 @@ export default function ShareButton({ if (!selectedMessageId) { await queryClient.fetchQuery( [QueryKeys.messages, conversationId], - () => dataService.getMessagesByConvoId(conversationId), + () => fetchConversationMessages(conversationId), { staleTime: 0 }, ); selectedMessageId = getActiveLatestMessage()?.messageId ?? null; diff --git a/client/src/components/Conversations/ConvoOptions/__tests__/ShareButton.test.tsx b/client/src/components/Conversations/ConvoOptions/__tests__/ShareButton.test.tsx index 5a2e476bec..f6309ca5b2 100644 --- a/client/src/components/Conversations/ConvoOptions/__tests__/ShareButton.test.tsx +++ b/client/src/components/Conversations/ConvoOptions/__tests__/ShareButton.test.tsx @@ -41,6 +41,8 @@ jest.mock('librechat-data-provider/react-query', () => ({ jest.mock('~/data-provider', () => ({ useGetStartupConfig: () => ({ data: { sharedLinksSnapshotFilesEnabled: true } }), + fetchConversationMessages: jest.requireActual('~/data-provider/Messages/queries') + .fetchConversationMessages, })); jest.mock('~/hooks/Messages/useLatestMessage', () => ({ @@ -249,7 +251,9 @@ describe('ShareButton', () => { renderShareButton(); await expect(mockResolveTargetMessageId?.()).resolves.toBe('persisted-message'); - expect(mockGetMessagesByConvoId).toHaveBeenCalledWith(ACTIVE_CONVERSATION_ID); + expect(mockGetMessagesByConvoId).toHaveBeenCalledWith(ACTIVE_CONVERSATION_ID, { + toolPreviews: true, + }); expect(mockGetMessageById).toHaveBeenCalledWith(ACTIVE_CONVERSATION_ID, 'persisted-message'); }); @@ -261,7 +265,9 @@ describe('ShareButton', () => { await expect(mockResolveTargetMessageId?.()).rejects.toMatchObject({ code: 'NO_MESSAGES', }); - expect(mockGetMessagesByConvoId).toHaveBeenCalledWith(ACTIVE_CONVERSATION_ID); + expect(mockGetMessagesByConvoId).toHaveBeenCalledWith(ACTIVE_CONVERSATION_ID, { + toolPreviews: true, + }); expect(mockGetMessageById).not.toHaveBeenCalled(); }); diff --git a/client/src/components/Messages/Content/ActionButton.tsx b/client/src/components/Messages/Content/ActionButton.tsx index dc7397e37e..452e5f4b32 100644 --- a/client/src/components/Messages/Content/ActionButton.tsx +++ b/client/src/components/Messages/Content/ActionButton.tsx @@ -14,6 +14,7 @@ interface ActionButtonProps { tabIndex?: number; className?: string; portalElement?: HTMLElement | null; + disabled?: boolean; } const ActionButton = React.forwardRef( @@ -28,6 +29,7 @@ const ActionButton = React.forwardRef( tabIndex, className, portalElement, + disabled = false, }, ref, ) => { @@ -38,11 +40,13 @@ const ActionButton = React.forwardRef( ref={ref} type="button" onClick={onClick} + disabled={disabled} tabIndex={tabIndex} aria-label={currentLabel} className={cn( 'text-text-secondary inline-flex items-center justify-center transition-all duration-200 ease-out select-none', 'hover:bg-surface-hover hover:text-text-primary', + 'disabled:pointer-events-none disabled:opacity-50', 'focus-visible:outline-border-heavy focus-visible:outline focus-visible:outline-2', iconOnly ? 'rounded-lg p-1.5' : 'ml-auto gap-2 rounded-md px-2 py-1', className, diff --git a/client/src/components/Messages/Content/CopyButton.tsx b/client/src/components/Messages/Content/CopyButton.tsx index 72ba649a6f..e703feb610 100644 --- a/client/src/components/Messages/Content/CopyButton.tsx +++ b/client/src/components/Messages/Content/CopyButton.tsx @@ -12,11 +12,22 @@ interface CopyButtonProps { label?: string; copiedLabel?: string; portalElement?: HTMLElement | null; + disabled?: boolean; } const CopyButton = React.forwardRef( ( - { isCopied, iconOnly = false, onClick, tabIndex, className, label, copiedLabel, portalElement }, + { + isCopied, + iconOnly = false, + onClick, + tabIndex, + className, + label, + copiedLabel, + portalElement, + disabled, + }, ref, ) => { const localize = useLocalize(); @@ -33,6 +44,7 @@ const CopyButton = React.forwardRef( tabIndex={tabIndex} className={className} portalElement={portalElement} + disabled={disabled} /> ); }, diff --git a/client/src/data-provider/Messages/__tests__/queries.test.ts b/client/src/data-provider/Messages/__tests__/queries.test.ts index 63c3b086c6..62c2ffb95b 100644 --- a/client/src/data-provider/Messages/__tests__/queries.test.ts +++ b/client/src/data-provider/Messages/__tests__/queries.test.ts @@ -372,7 +372,7 @@ describe('useGetMessagesByConvoId', () => { expect(result.current.data).toEqual(serverMessages); }); expect(mockGetMessagesByConvoId).toHaveBeenCalledTimes(1); - expect(mockGetMessagesByConvoId).toHaveBeenCalledWith(conversationId); + expect(mockGetMessagesByConvoId).toHaveBeenCalledWith(conversationId, { toolPreviews: true }); unmount(); }); @@ -397,7 +397,7 @@ describe('useGetMessagesByConvoId', () => { }); await waitFor(() => { - expect(mockGetMessagesByConvoId).toHaveBeenCalledWith(conversationId); + expect(mockGetMessagesByConvoId).toHaveBeenCalledWith(conversationId, { toolPreviews: true }); }); act(() => { @@ -443,7 +443,7 @@ describe('useGetMessagesByConvoId', () => { const refetchPromise = result.current.refetch(); await waitFor(() => { - expect(mockGetMessagesByConvoId).toHaveBeenCalledWith(conversationId); + expect(mockGetMessagesByConvoId).toHaveBeenCalledWith(conversationId, { toolPreviews: true }); }); act(() => { @@ -520,7 +520,7 @@ describe('useGetMessagesByConvoId', () => { const refetchPromise = result.current.refetch(); await waitFor(() => { - expect(mockGetMessagesByConvoId).toHaveBeenCalledWith(conversationId); + expect(mockGetMessagesByConvoId).toHaveBeenCalledWith(conversationId, { toolPreviews: true }); }); act(() => { @@ -579,7 +579,9 @@ describe('useGetMessagesByConvoId', () => { await waitFor(() => { expect(result.current.data).toBe(currentMessages); }); - expect(dataService.getMessagesByConvoId).toHaveBeenCalledWith(conversationId); + expect(dataService.getMessagesByConvoId).toHaveBeenCalledWith(conversationId, { + toolPreviews: true, + }); expect(queryClient.getQueryData([QueryKeys.messages, conversationId])).toBe(currentMessages); unmount(); @@ -623,7 +625,9 @@ describe('useGetMessagesByConvoId', () => { await waitFor(() => { expect(result.current.data).toBe(currentMessages); }); - expect(dataService.getMessagesByConvoId).toHaveBeenCalledWith(conversationId); + expect(dataService.getMessagesByConvoId).toHaveBeenCalledWith(conversationId, { + toolPreviews: true, + }); expect(queryClient.getQueryData([QueryKeys.messages, conversationId])).toBe(currentMessages); expect(logger.warn).toHaveBeenCalledWith( 'messages', diff --git a/client/src/data-provider/Messages/__tests__/retention.spec.ts b/client/src/data-provider/Messages/__tests__/retention.spec.ts index 4f6e1f800c..9b89506f86 100644 --- a/client/src/data-provider/Messages/__tests__/retention.spec.ts +++ b/client/src/data-provider/Messages/__tests__/retention.spec.ts @@ -244,6 +244,35 @@ describe('retainMessages', () => { void pending.catch(() => undefined); }); + it('releases the full tool-call parts fetched for a conversation along with its history', () => { + const part = (conversationId: string, index: number) => [ + QueryKeys.toolCallPart, + conversationId, + `${conversationId}-1`, + index, + 'call', + '', + '', + 'rev', + ]; + const isPartCached = (key: unknown[]) => queryClient.getQueryCache().find(key) != null; + visit(queryClient, 'a'); + queryClient.setQueryData(part('a', 1), { tool_call: { output: 'full' } }); + queryClient.setQueryData(part('b', 1), { tool_call: { output: 'other conversation' } }); + seed(queryClient, 'b'); + const stillOpen = observe(queryClient, 'b'); + + jest.advanceTimersByTime(TTL); + expect(isCached(queryClient, 'a')).toBe(false); + expect(isPartCached(part('a', 1))).toBe(false); + expect(isPartCached(part('b', 1))).toBe(true); + + queryClient.setQueryData(part('b', 2), { tool_call: { output: 'cleared' } }); + queryClient.removeQueries([QueryKeys.messages, 'b'], { exact: true }); + expect(isPartCached(part('b', 2))).toBe(false); + stillOpen(); + }); + it('restores the messages query defaults on cleanup', () => { expect(queryClient.getQueryDefaults([QueryKeys.messages])?.cacheTime).toBe(Infinity); stop(); diff --git a/client/src/data-provider/Messages/queries.ts b/client/src/data-provider/Messages/queries.ts index d02bc06137..760655ffcf 100644 --- a/client/src/data-provider/Messages/queries.ts +++ b/client/src/data-provider/Messages/queries.ts @@ -80,6 +80,51 @@ export function shouldPreserveMessagesOnNotFound({ return hasPendingAssistantTail(currentMessages); } +/** + * Loads a conversation's messages for the shared `[messages, id]` cache, with settled tool calls + * as bounded previews. Every writer of that cache uses this, so the cache never flips between + * preview and full payloads; a reader that needs every byte (export) asks for a full load. + */ +export function fetchConversationMessages(conversationId: string): Promise { + return dataService.getMessagesByConvoId(conversationId, { toolPreviews: true }); +} + +/** + * The stored content of one tool-call part, for a preview the reader opened. Kept under its own + * key and never merged into `[messages, id]`, so a refetch of the conversation cannot drop it and + * the conversation cache keeps only previews. `revision` (see `getToolCallPreviewRevision`) is + * part of the key, so a stored call that changes after it was fetched is fetched again. + */ +export const useToolCallPartQuery = ( + params: t.ToolCallPartParams, + config?: UseQueryOptions, + revision = '', +): QueryObserverResult => { + /** The index only rides along in the request; the server locates the call by identity, so the + * cache does too, and a shifted index cannot fetch the same part twice. */ + const { conversationId, messageId, toolCallId, stepId, agentId } = params; + return useQuery( + [ + QueryKeys.toolCallPart, + conversationId, + messageId, + toolCallId ?? '', + stepId ?? '', + agentId ?? '', + revision, + ], + () => dataService.getToolCallPart(params), + { + staleTime: Infinity, + retry: 1, + refetchOnWindowFocus: false, + refetchOnReconnect: false, + refetchOnMount: false, + ...config, + }, + ); +}; + function hasActiveJob(queryClient: QueryClient, id: string) { if (!id) { return false; @@ -119,7 +164,7 @@ export const useGetMessagesByConvoId = ( ); let result: t.TMessage[]; try { - result = await dataService.getMessagesByConvoId(id); + result = await fetchConversationMessages(id); } catch (error) { const currentMessages = queryClient.getQueryData(queryKey); if ( diff --git a/client/src/data-provider/Messages/retention.ts b/client/src/data-provider/Messages/retention.ts index 4ec0129325..c868fad15a 100644 --- a/client/src/data-provider/Messages/retention.ts +++ b/client/src/data-provider/Messages/retention.ts @@ -163,6 +163,15 @@ export function retainMessages( }); } + /** Full tool-call parts fetched for a conversation's previews leave with its history, so a + * released conversation does not keep its largest content alive under another key. */ + const releaseToolCallParts = (conversationId: string) => { + queryClient.removeQueries({ + queryKey: [QueryKeys.toolCallPart, conversationId], + predicate: (query) => query.getObserversCount() === 0, + }); + }; + const unsubscribe = cache.subscribe((event) => { if ( event.type !== 'added' && @@ -180,6 +189,9 @@ export function retainMessages( schedule(RELEASE_SETTLE_MS); return; } + if (event.type === 'removed') { + releaseToolCallParts(conversationId); + } if (event.type !== 'observerRemoved') { leftAt.delete(conversationId); return; diff --git a/client/src/hooks/SSE/useResumableSSE.ts b/client/src/hooks/SSE/useResumableSSE.ts index 9ed757faa2..cbdbd5d398 100644 --- a/client/src/hooks/SSE/useResumableSSE.ts +++ b/client/src/hooks/SSE/useResumableSSE.ts @@ -84,6 +84,7 @@ import { generationProtocolHeaders, getGenerationProtocolVersion, supportsGenerationProtocolV2, + fetchConversationMessages, GENERATION_PROTOCOL_VERSION, } from '~/data-provider'; import { @@ -3231,7 +3232,7 @@ export default function useResumableSSE( const fetched = await queryClient.fetchQuery({ queryKey: messageQueryKey, // The first stream can finish before CREATED mounts the saved-chat query. - queryFn: () => dataService.getMessagesByConvoId(convoId), + queryFn: () => fetchConversationMessages(convoId), }); if (!isCurrentSubscription()) { return; diff --git a/client/src/locales/en/translation.json b/client/src/locales/en/translation.json index 15b2897350..1b468eb66e 100644 --- a/client/src/locales/en/translation.json +++ b/client/src/locales/en/translation.json @@ -1627,6 +1627,8 @@ "com_ui_tool_calling": "Calling {{0}}", "com_ui_tool_preparation_time": "Preparation", "com_ui_tool_call_time": "Tool call", + "com_ui_tool_content_error": "Couldn't load the full tool call.", + "com_ui_tool_content_loading": "Loading the full tool call…", "com_ui_tool_total_time": "Total elapsed", "com_ui_during_run_actions": "More send options", "com_ui_edit": "Edit", diff --git a/client/src/utils/approval.spec.ts b/client/src/utils/approval.spec.ts index 099028673d..041bb9b7ed 100644 --- a/client/src/utils/approval.spec.ts +++ b/client/src/utils/approval.spec.ts @@ -16,6 +16,7 @@ import { isAnsweredAskUserQuestionPart, splitOtherOption, } from './approval'; +import { hasPendingApprovalInPart } from './groupToolCalls'; const toolCallPart = (id: string, extra: Record = {}): TMessageContentParts => ({ @@ -230,6 +231,31 @@ describe('applyPendingAction — subagent-nested tool calls', () => { expect(countTaggedApprovalParts(result, 'a1')).toBe(1); }); + it('still tags the nested pause in a message whose settled calls arrived as previews', () => { + /** The shape a preview-aware conversation load returns: a settled sibling shortened, and + * the running subagent's transcript kept because it holds an unresolved approval. */ + const message = msg({ + content: [ + toolCallPart('done-call', { + output: 'head…tail', + outputTruncated: true, + outputLength: 40_000, + }), + ...(subagentMsg('child-tc1').content as TMessageContentParts[]), + ], + }); + const result = applyPendingAction(message, childAction()); + const parentToolCall = getToolCall(result.content?.[1] as TMessageContentParts) as + | { subagent_content?: TMessageContentParts[] } + | undefined; + expect(getToolCall(parentToolCall?.subagent_content?.[0])?.approval).toMatchObject({ + actionId: 'a1', + }); + expect(hasPendingApprovalInPart(result.content?.[1] as TMessageContentParts)).toBe(true); + expect(hasPendingApprovalInPart(result.content?.[0] as TMessageContentParts)).toBe(false); + expect(countTaggedApprovalParts(result, 'a1')).toBe(1); + }); + it('returns the same message when no nested tool call matches', () => { const message = subagentMsg('child-other'); expect(applyPendingAction(message, childAction())).toBe(message); diff --git a/client/src/utils/tokens.spec.ts b/client/src/utils/tokens.spec.ts index 3b281b5cc4..f58b67fb66 100644 --- a/client/src/utils/tokens.spec.ts +++ b/client/src/utils/tokens.spec.ts @@ -202,6 +202,36 @@ describe('token index', () => { expect(totals.estTokens).toBe(5); }); + it('estimates a previewed tool call from its stored lengths, not the preview', () => { + buildIndex(CONVO, [ + msg('u1', Constants.NO_PARENT, true, 12), + { + messageId: 'a1', + parentMessageId: 'u1', + isCreatedByUser: false, + conversationId: CONVO, + text: '', + content: [ + { + type: 'tool_call', + tool_call: { + name: 'run', + args: 'aa', + argsTruncated: true, + argsLength: 17, + output: 'o…o', + outputTruncated: true, + outputLength: 20, + }, + }, + ], + } as unknown as TMessage, + ]); + + /** name 3 + stored args 17 + stored output 20 = 40 chars / 4 = 10. */ + expect(sumBranch(CONVO, 'a1').estTokens).toBe(10); + }); + it('exposes the count-less tail estimate so live output is not double-counted', () => { buildIndex(CONVO, [ msg('u1', Constants.NO_PARENT, true, 12), diff --git a/client/src/utils/tokens.ts b/client/src/utils/tokens.ts index d342fcfeab..d39be7ad8d 100644 --- a/client/src/utils/tokens.ts +++ b/client/src/utils/tokens.ts @@ -187,6 +187,33 @@ function addUsage(target: BranchUsage, usage?: BranchUsage): void { } } +type ToolCallLengthFields = { + args?: unknown; + output?: unknown; + argsLength?: unknown; + outputLength?: unknown; +}; + +/** Stored lengths of a tool call's arguments and output. A preview carries the full lengths in + * `argsLength`/`outputLength`, so estimates match the conversation the model actually sees. */ +function toolCallLengths(call: ToolCallLengthFields): { args: number; output: number } { + let args = 0; + if (typeof call.argsLength === 'number') { + args = call.argsLength; + } else if (typeof call.args === 'string') { + args = call.args.length; + } else if (call.args != null) { + args = JSON.stringify(call.args)?.length ?? 0; + } + let output = 0; + if (typeof call.outputLength === 'number') { + output = call.outputLength; + } else if (typeof call.output === 'string') { + output = call.output.length; + } + return { args, output }; +} + /** Chars of a content part's text, handling both the string and `{ value }` forms. * Reasoning (`think`) and error parts are excluded — the send path strips them * before counting, so they aren't part of the next call's context. */ @@ -199,24 +226,12 @@ function partTextChars(part: unknown): number { return 0; } if (type === 'tool_call') { - const call = (part as { tool_call?: { name?: unknown; args?: unknown; output?: unknown } }) - .tool_call; + const call = (part as { tool_call?: ToolCallLengthFields & { name?: unknown } }).tool_call; if (call == null) { return 0; } - let chars = typeof call.name === 'string' ? call.name.length : 0; - if (typeof call.args === 'string') { - chars += call.args.length; - } else if (call.args != null) { - const serialized = JSON.stringify(call.args); - if (typeof serialized === 'string') { - chars += serialized.length; - } - } - if (typeof call.output === 'string') { - chars += call.output.length; - } - return chars; + const lengths = toolCallLengths(call); + return (typeof call.name === 'string' ? call.name.length : 0) + lengths.args + lengths.output; } const text = (part as { text?: unknown }).text; if (typeof text === 'string') { @@ -253,20 +268,10 @@ function messageToolChars(message: Partial): { chars: number; resultCh if ('name' in call && typeof call.name === 'string') { chars += call.name.length; } - if ('args' in call) { - if (typeof call.args === 'string') { - chars += call.args.length; - } else { - const serialized = JSON.stringify(call.args ?? null); - if (typeof serialized === 'string') { - chars += serialized.length; - } - } - } - if ('output' in call && typeof call.output === 'string') { - chars += call.output.length; - resultChars += call.output.length; - } + const lengths = toolCallLengths(call as ToolCallLengthFields); + chars += 'args' in call && call.args == null ? 4 : lengths.args; + chars += lengths.output; + resultChars += lengths.output; } } return { chars, resultChars }; diff --git a/librechat.example.yaml b/librechat.example.yaml index 358374bcb0..0b69903cf7 100644 --- a/librechat.example.yaml +++ b/librechat.example.yaml @@ -17,6 +17,18 @@ projects: # Characters of the project description shown in the workspace and sidebar (maximum 10000). maxDescriptionLength: 1000 +# Tool-call previews: a long conversation's tool outputs, arguments and subagent transcripts +# can run to megabytes, though their cards render collapsed. The web client asks for bounded +# previews when it loads a conversation (the start and end of each long output, arguments +# with their long strings shortened, no subagent transcripts) and loads a call in full when +# the reader opens it. Clients that do not ask, exports and share links get full content. +# Every field is optional and defaults to the value shown; enabled: false always sends the +# full payload. +# toolCallPreviews: +# enabled: true +# outputChars: 512 # characters of each output kept, half from the start and half from the end +# argsChars: 512 # characters of serialized arguments kept; JSON stays valid JSON + # Langfuse connections are managed through Settings > Langfuse when available. # That flow verifies the credentials and stores the secret key encrypted; do not # place a plaintext langfuse.secretKey in this file. Environment-managed central diff --git a/packages/api/src/index.ts b/packages/api/src/index.ts index be7e6d7d0b..b5d05ebf3f 100644 --- a/packages/api/src/index.ts +++ b/packages/api/src/index.ts @@ -85,6 +85,8 @@ export * from './prompts'; export * from './projects'; /* Conversations */ export * from './conversations'; +/* Messages */ +export * from './messages'; /* Skills */ export * from './schedules'; export * from './schedules/service'; diff --git a/packages/api/src/messages/index.ts b/packages/api/src/messages/index.ts new file mode 100644 index 0000000000..4d62fa23e7 --- /dev/null +++ b/packages/api/src/messages/index.ts @@ -0,0 +1,2 @@ +export * from './parts'; +export * from './previews'; diff --git a/packages/api/src/messages/parts.spec.ts b/packages/api/src/messages/parts.spec.ts new file mode 100644 index 0000000000..5aff4bf694 --- /dev/null +++ b/packages/api/src/messages/parts.spec.ts @@ -0,0 +1,419 @@ +import express from 'express'; +import mongoose from 'mongoose'; +import request from 'supertest'; +import { ContentTypes } from 'librechat-data-provider'; +import { MongoMemoryServer } from 'mongodb-memory-server'; +import { + logger, + createMethods, + createModels, + tenantStorage, + CLIENT_MESSAGE_SELECT, +} from '@librechat/data-schemas'; +import type { AllMethods, IMessage } from '@librechat/data-schemas'; +import type { NextFunction, Request, Response } from 'express'; +import type { MessageValidationResult } from '~/middleware/messageValidation'; +import type { ToolCallPartDeps, ToolCallPartHandlerDeps } from './parts'; +import { previewMessagesToolCalls } from './previews'; +import { createToolCallPartHandler } from './parts'; + +let mongod: MongoMemoryServer; +let methods: AllMethods; +let app: express.Express; + +const OWNER = 'owner-user'; +const conversationId = '7f9c2b1e-4a5d-4c3b-9e8f-1a2b3c4d5e6f'; +const otherConversationId = '0b1c2d3e-4f50-4617-8293-a4b5c6d7e8f9'; +const messageId = 'msg-parts'; + +const longOutput = `stdout:\n${'line of build output\n'.repeat(2_000)}[exit code: 0]`; +const longArgs = JSON.stringify({ + command: 'npm run build', + intent: 'Build', + pad: 'p'.repeat(5_000), +}); +const transcript = [ + { type: ContentTypes.TEXT, text: 'child text' }, + { + type: ContentTypes.TOOL_CALL, + tool_call: { + id: 'child_1', + name: 'read_file', + args: '{"path":"a"}', + output: 'x'.repeat(3_000), + }, + }, +]; + +function content() { + return [ + { type: ContentTypes.TEXT, text: 'Working on it' }, + { + type: ContentTypes.TOOL_CALL, + tool_call: { + id: 'call_bash', + type: 'tool_call', + name: 'bash_tool', + args: longArgs, + output: longOutput, + progress: 1, + backgroundTask: { + version: 1, + taskId: 'task-1', + toolName: 'bash_tool', + status: 'completed', + settledAt: new Date('2026-10-01T00:00:00Z'), + resultClaim: { kind: 'manual', claimId: 'secret-claim', claimedAt: new Date() }, + }, + }, + }, + { + type: ContentTypes.TOOL_CALL, + tool_call: { + id: 'call_sub', + type: 'tool_call', + name: 'subagent', + args: '{"prompt":"look around"}', + output: 'child summary', + progress: 1, + subagent_content: transcript, + }, + }, + ]; +} + +async function seed(user = OWNER, overrides: Partial = {}) { + await methods.saveMessage( + { userId: user }, + { + messageId, + conversationId, + parentMessageId: 'parent', + isCreatedByUser: false, + text: '', + content: content(), + ...overrides, + }, + ); +} + +function withUser(req: Request, _res: Response, next: NextFunction) { + (req as Request & { user: { id: string } }).user = { + id: (req.headers['x-user'] as string | undefined) ?? OWNER, + }; + const tenantId = req.headers['x-tenant'] as string | undefined; + if (tenantId == null) { + next(); + return; + } + tenantStorage.run({ tenantId }, () => next()); +} + +const allow = (): Pick => ({ + validate: () => ({ + conversationId, + shouldFetchMessages: true, + promise: Promise.resolve({ ok: true }), + }), + sendValidationResponse: (res, result) => res.status(result.status).json(result.body), +}); + +const handler = ( + getMessages: ToolCallPartDeps['getMessages'], + validation: Partial = {}, +) => createToolCallPartHandler({ getMessages, ...allow(), ...validation }); + +const mount = (route: ReturnType) => { + const server = express(); + server.use(withUser); + server.get('/api/messages/:conversationId/:messageId/parts/:partIndex', route); + return server; +}; + +/** `toolCallId` defaults to the seeded bash call; `null` leaves the parameter out. */ +const partUrl = ( + index: number | string, + toolCallId: string | null = 'call_bash', + convo = conversationId, +) => + `/api/messages/${convo}/${messageId}/parts/${index}${ + toolCallId == null ? '' : `?toolCallId=${encodeURIComponent(toolCallId)}` + }`; + +beforeAll(async () => { + jest.spyOn(logger, 'error').mockImplementation(() => logger); + mongod = await MongoMemoryServer.create(); + createModels(mongoose); + methods = createMethods(mongoose); + await mongoose.connect(mongod.getUri()); + app = express(); + app.use(withUser); + app.get( + '/api/messages/:conversationId/:messageId/parts/:partIndex', + handler((filter, select) => methods.getMessages(filter, select)), + ); +}); + +afterAll(async () => { + await mongoose.disconnect(); + await mongod.stop(); +}); + +beforeEach(async () => { + await (mongoose.models.Message as mongoose.Model).deleteMany({}); +}); + +describe('GET /api/messages/:conversationId/:messageId/parts/:partIndex', () => { + it('returns the stored tool call in full, exactly as a full conversation load sends it', async () => { + await seed(); + const response = await request(app).get(partUrl(1, 'call_bash')); + expect(response.status).toBe(200); + + const [stored] = await methods.getMessages( + { conversationId, messageId, user: OWNER }, + CLIENT_MESSAGE_SELECT, + ); + const fullLoadPart = JSON.parse(JSON.stringify((stored.content as unknown[])[1])); + expect(response.body).toEqual({ + conversationId, + messageId, + partIndex: 1, + tool_call: fullLoadPart.tool_call, + }); + expect(response.body.tool_call.output).toBe(longOutput); + expect(response.body.tool_call.args).toBe(longArgs); + expect(response.body.tool_call.backgroundTask).toMatchObject({ taskId: 'task-1' }); + expect(response.body.tool_call.backgroundTask).not.toHaveProperty('resultClaim'); + }); + + it('restores everything a preview left out, subagent transcript included', async () => { + await seed(); + const [stored] = await methods.getMessages( + { conversationId, messageId, user: OWNER }, + CLIENT_MESSAGE_SELECT, + ); + const [previewed] = previewMessagesToolCalls([stored], { outputChars: 512, argsChars: 512 }); + const previewParts = previewed.content as Array<{ tool_call?: Record }>; + expect(previewParts[1].tool_call).toMatchObject({ outputTruncated: true, argsTruncated: true }); + expect(previewParts[2].tool_call).toMatchObject({ + subagentContentOmitted: true, + subagentContentParts: 2, + }); + + const sub = await request(app).get(partUrl(2, 'call_sub')); + expect(sub.status).toBe(200); + expect(sub.body.tool_call.subagent_content).toEqual(transcript); + expect(sub.body.tool_call).not.toHaveProperty('subagentContentOmitted'); + }); + + it('finds the part by tool-call id when the client copy is out of position', async () => { + await seed(); + const response = await request(app).get(partUrl(0, 'call_sub')); + expect(response.status).toBe(200); + expect(response.body.partIndex).toBe(2); + expect(response.body.tool_call.id).toBe('call_sub'); + }); + + it('tells repeated provider ids apart by step and agent, refusing an ambiguous match', async () => { + const repeated = (agentId: string, stepId: string, output: string) => ({ + type: ContentTypes.TOOL_CALL, + agentId, + tool_call: { + id: 'call_dup', + type: 'tool_call', + name: 'read_file', + stepId, + output, + args: '{}', + }, + }); + await seed(OWNER, { + content: [ + { type: ContentTypes.TEXT, text: 'two agents' }, + repeated('agent_a', 'step_a', 'from agent a'), + repeated('agent_b', 'step_b', 'from agent b'), + ], + }); + const identity = (agentId: string, stepId: string) => + `${partUrl(0, 'call_dup')}&stepId=${stepId}&agentId=${agentId}`; + const agentB = await request(app).get(identity('agent_b', 'step_b')); + expect(agentB.status).toBe(200); + expect(agentB.body).toMatchObject({ partIndex: 2, tool_call: { output: 'from agent b' } }); + + const agentA = await request(app).get(identity('agent_a', 'step_a')); + expect(agentA.body).toMatchObject({ partIndex: 1, tool_call: { output: 'from agent a' } }); + + /** An index pointing at the other agent's call does not win over the identity. */ + const shifted = await request(app).get( + `${partUrl(1, 'call_dup')}&stepId=step_b&agentId=agent_b`, + ); + expect(shifted.body).toMatchObject({ partIndex: 2 }); + + /** Without the step and agent, both calls carry the requested identity loosely; neither does + * exactly, so nothing is returned. */ + expect((await request(app).get(partUrl(1, 'call_dup'))).status).toBe(404); + }); + + it('refuses an identity that repeats exactly, even when the index points at one occurrence', async () => { + const twin = (output: string) => ({ + type: ContentTypes.TOOL_CALL, + tool_call: { id: 'call_twin', type: 'tool_call', name: 'read_file', output, args: '{}' }, + }); + await seed(OWNER, { content: [twin('first'), twin('second')] }); + expect((await request(app).get(partUrl(0, 'call_twin'))).status).toBe(404); + expect((await request(app).get(partUrl(1, 'call_twin'))).status).toBe(404); + }); + + it('treats an empty step or agent as absent, so such a preview stays fetchable', async () => { + await seed(OWNER, { + content: [ + { + type: ContentTypes.TOOL_CALL, + agentId: '', + tool_call: { + id: 'call_empty', + stepId: '', + type: 'tool_call', + name: 'bash_tool', + output: 'done', + args: '{}', + }, + }, + ], + }); + const response = await request(app).get(partUrl(0, 'call_empty')); + expect(response.status).toBe(200); + expect(response.body.tool_call.id).toBe('call_empty'); + }); + + it('serves a call whose provider id is unusually long', async () => { + const longId = `call_${'x'.repeat(2_000)}`; + await seed(OWNER, { + content: [ + { + type: ContentTypes.TOOL_CALL, + tool_call: { + id: longId, + type: 'tool_call', + name: 'bash_tool', + output: 'done', + args: '{}', + }, + }, + ], + }); + const response = await request(app).get(partUrl(0, longId)); + expect(response.status).toBe(200); + expect(response.body.tool_call.id).toBe(longId); + }); + + it('answers not found for a missing part, a non-tool part, or an unknown id', async () => { + await seed(); + expect((await request(app).get(partUrl(9, 'call_missing'))).status).toBe(404); + expect((await request(app).get(partUrl(250_000, 'call_missing'))).status).toBe(404); + expect((await request(app).get(partUrl(0, 'call_missing'))).status).toBe(404); + expect((await request(app).get(partUrl(1, 'call_missing'))).status).toBe(404); + }); + + it("never returns another user's message", async () => { + await seed(); + const response = await request(app).get(partUrl(1)).set('x-user', 'intruder'); + expect(response.status).toBe(404); + expect(JSON.stringify(response.body)).not.toContain('line of build output'); + }); + + it('never returns a message through a different conversation id', async () => { + await seed(); + const response = await request(app).get(partUrl(1, 'call_bash', otherConversationId)); + expect(response.status).toBe(404); + }); + + it('never crosses tenants, even for the same user id', async () => { + await tenantStorage.run({ tenantId: 'tenant-a' }, () => seed()); + const sameTenant = await request(app).get(partUrl(1)).set('x-tenant', 'tenant-a'); + expect(sameTenant.status).toBe(200); + const otherTenant = await request(app).get(partUrl(1)).set('x-tenant', 'tenant-b'); + expect(otherTenant.status).toBe(404); + }); + + it('rejects malformed coordinates before reading', async () => { + const getMessages = jest.fn(); + const strict = mount(handler(getMessages)); + expect((await request(strict).get(partUrl(1, null))).status).toBe(400); + expect((await request(strict).get(partUrl('-1'))).status).toBe(400); + expect((await request(strict).get(partUrl('1.5'))).status).toBe(400); + expect((await request(strict).get(partUrl('99999999999999999999'))).status).toBe(400); + expect(getMessages).not.toHaveBeenCalled(); + }); + + it('starts the part read beside access validation and answers only once it passes', async () => { + await seed(); + const events: string[] = []; + let release: (result: MessageValidationResult) => void = () => undefined; + const pending = new Promise((resolve) => (release = resolve)); + const server = mount( + handler( + (filter, select) => { + events.push('read'); + return methods.getMessages(filter, select); + }, + { validate: () => ({ conversationId, shouldFetchMessages: true, promise: pending }) }, + ), + ); + const response = request(server) + .get(partUrl(1)) + .then((res) => res); + await new Promise((resolve) => setTimeout(resolve, 50)); + events.push('validated'); + release({ ok: true }); + expect((await response).status).toBe(200); + expect(events).toEqual(['read', 'validated']); + }); + + it('sends the validation verdict, never the part, when access is refused', async () => { + await seed(); + const server = mount( + handler((filter, select) => methods.getMessages(filter, select), { + validate: () => ({ + conversationId, + shouldFetchMessages: true, + promise: Promise.resolve({ + ok: false, + status: 403, + body: { error: 'User not authorized for this conversation' }, + }), + }), + }), + ); + const response = await request(server).get(partUrl(1)); + expect(response.status).toBe(403); + expect(JSON.stringify(response.body)).not.toContain('line of build output'); + }); + + it('answers not found for the placeholder conversation without reading', async () => { + const getMessages = jest.fn(); + const server = mount( + handler(getMessages, { + validate: () => ({ + conversationId: 'new', + shouldFetchMessages: false, + promise: Promise.resolve({ + ok: false, + status: 200, + body: [], + send: true, + }), + }), + }), + ); + expect((await request(server).get(partUrl(1))).status).toBe(404); + expect(getMessages).not.toHaveBeenCalled(); + }); + + it('reports a read failure as a server error without leaking it', async () => { + const failing = mount(handler(jest.fn().mockRejectedValue(new Error('db down')))); + const response = await request(failing).get(partUrl(1)); + expect(response.status).toBe(500); + expect(response.body).toEqual({ error: 'Internal server error' }); + }); +}); diff --git a/packages/api/src/messages/parts.ts b/packages/api/src/messages/parts.ts new file mode 100644 index 0000000000..0802463116 --- /dev/null +++ b/packages/api/src/messages/parts.ts @@ -0,0 +1,234 @@ +import { ContentTypes } from 'librechat-data-provider'; +import { logger, CLIENT_MESSAGE_SELECT } from '@librechat/data-schemas'; +import type { FullToolCall, ToolCallPartResponse } from 'librechat-data-provider'; +import type { Request, Response } from 'express'; +import type { + MessageRequestValidation, + FailedMessageValidationResult, +} from '~/middleware/messageValidation'; + +/** Scopes a read to one message the authenticated user owns. */ +export interface ToolCallPartFilter { + user: string; + conversationId: string; + messageId: string; +} + +interface StoredMessage { + content?: unknown[] | null; +} + +export interface ToolCallPartDeps { + /** + * The data-schemas `getMessages` method. The projection is supplied here, so the part leaves + * with exactly the fields a conversation load sends for it, private fields excluded. + */ + getMessages: (filter: ToolCallPartFilter, select: string) => Promise; +} + +export interface ToolCallPartInput extends ToolCallPartFilter { + partIndex: number; + toolCallId?: string; + /** Host run-step id of the call; provider tool-call ids can repeat within one response. */ + stepId?: string; + /** Agent that produced the part, for responses where parallel agents reuse call ids. */ + agentId?: string; +} + +type PartIdentity = Pick; + +export type ToolCallPartErrorCode = 'message_not_found' | 'part_not_found'; + +export type ToolCallPartResult = + | { ok: true; value: { partIndex: number; toolCall: FullToolCall } } + | { ok: false; error: { code: ToolCallPartErrorCode } }; + +interface StoredToolCallPart { + type?: string; + agentId?: string; + tool_call?: FullToolCall; +} + +const PART_INDEX_PATTERN = /^\d+$/; + +function getAgentToolCall(part: unknown): FullToolCall | undefined { + if (part == null || typeof part !== 'object') { + return undefined; + } + const { type, tool_call: toolCall } = part as StoredToolCallPart; + if (type !== ContentTypes.TOOL_CALL || toolCall == null || typeof toolCall !== 'object') { + return undefined; + } + return toolCall.type == null || toolCall.type === 'tool_call' ? toolCall : undefined; +} + +/** An empty or missing discriminator is the same absent value, on both sides. */ +const discriminator = (value: unknown): string | undefined => + typeof value === 'string' && value !== '' ? value : undefined; + +/** Exact match: an absent step or agent matches only a call that has none, as the preview pass + * assumed when it checked the identity was unique. */ +function matchesIdentity(part: unknown, toolCall: FullToolCall, identity: PartIdentity): boolean { + if (toolCall.id !== identity.toolCallId) { + return false; + } + if (discriminator(toolCall.stepId) !== discriminator(identity.stepId)) { + return false; + } + return discriminator((part as StoredToolCallPart).agentId) === discriminator(identity.agentId); +} + +/** + * Finds the one part carrying the requested identity. The client's index is not trusted for + * this: its copy of the content can be shifted by client-only cards. The preview pass only + * shortens calls whose identity is unique in their message, so a request for a preview always + * names exactly one part; a repeated identity resolves to not found, never to another call. + */ +function locatePart( + content: unknown[], + identity: PartIdentity, +): { partIndex: number; toolCall: FullToolCall } | undefined { + let found: { partIndex: number; toolCall: FullToolCall } | undefined; + for (let i = 0; i < content.length; i++) { + const toolCall = getAgentToolCall(content[i]); + if (toolCall == null || !matchesIdentity(content[i], toolCall, identity)) { + continue; + } + if (found != null) { + return undefined; + } + found = { partIndex: i, toolCall }; + } + return found; +} + +/** Reads one stored tool-call part in full, scoped to the requesting user's message. */ +export async function readToolCallPart( + deps: ToolCallPartDeps, + input: ToolCallPartInput, +): Promise { + const { user, conversationId, messageId, toolCallId, stepId, agentId } = input; + const messages = await deps.getMessages( + { user, conversationId, messageId }, + CLIENT_MESSAGE_SELECT, + ); + const content = messages?.[0]?.content; + if (!Array.isArray(content)) { + return { ok: false, error: { code: 'message_not_found' } }; + } + const located = locatePart(content, { toolCallId, stepId, agentId }); + if (located == null) { + return { ok: false, error: { code: 'part_not_found' } }; + } + return { ok: true, value: located }; +} + +export type ToolCallPartRequest = Request< + { conversationId?: string; messageId?: string; partIndex?: string }, + ToolCallPartResponse | { error: string }, + unknown, + { toolCallId?: unknown; stepId?: unknown; agentId?: unknown } +> & { user?: { id?: string; tenantId?: string | null } }; + +export interface ToolCallPartHandlerDeps extends ToolCallPartDeps { + /** + * Starts the conversation access check the message routes use (ownership, the active-job + * fallback, child-thread refusal). The part read runs beside it, and nothing is sent until it + * passes. + */ + validate: (req: ToolCallPartRequest) => MessageRequestValidation; + sendValidationResponse: (res: Response, result: FailedMessageValidationResult) => unknown; +} + +type SettledRead = { ok: true; value: ToolCallPartResult } | { ok: false; error: unknown }; + +function parsePartIndex(value: string | undefined): number | undefined { + if (value == null || !PART_INDEX_PATTERN.test(value)) { + return undefined; + } + const index = Number(value); + return Number.isSafeInteger(index) ? index : undefined; +} + +/** `undefined` when absent, `null` when present but unusable. */ +function parseIdentifier(value: unknown): string | undefined | null { + if (value == null) { + return undefined; + } + if (typeof value !== 'string' || value === '') { + return null; + } + return value; +} + +/** + * `GET /api/messages/:conversationId/:messageId/parts/:partIndex[?toolCallId=]`, mounted behind + * authentication. Conversation access is validated as the message routes do it, with the + * user-scoped part read started alongside so an expansion costs one round trip of latency; the + * read is itself scoped to the authenticated user, so a message id from another conversation or + * account resolves to not found. + */ +export function createToolCallPartHandler( + deps: ToolCallPartHandlerDeps, +): (req: ToolCallPartRequest, res: Response) => Promise { + return async (req, res) => { + const userId = req.user?.id; + const { conversationId, messageId } = req.params; + const partIndex = parsePartIndex(req.params.partIndex); + const toolCallId = parseIdentifier(req.query.toolCallId); + const stepId = parseIdentifier(req.query.stepId); + const agentId = parseIdentifier(req.query.agentId); + if (!userId || !conversationId || !messageId) { + res.status(404).json({ error: 'Tool call not found' }); + return; + } + /** Previews are only produced for calls with an id, so a request without one has no part + * it could stably name: an index alone may point at another call once content shifted. */ + if (partIndex == null || toolCallId == null || stepId === null || agentId === null) { + res.status(400).json({ error: 'Invalid tool call part' }); + return; + } + try { + const validation = deps.validate(req); + if (!validation.shouldFetchMessages) { + res.status(404).json({ error: 'Tool call not found' }); + return; + } + const read: Promise = readToolCallPart(deps, { + user: userId, + conversationId, + messageId, + partIndex, + toolCallId, + stepId, + agentId, + }).then( + (value) => ({ ok: true, value }), + (error: unknown) => ({ ok: false, error }), + ); + const verdict = await validation.promise; + if (!verdict.ok) { + deps.sendValidationResponse(res, verdict); + return; + } + const settled = await read; + if (!settled.ok) { + throw settled.error; + } + const result = settled.value; + if (!result.ok) { + res.status(404).json({ error: 'Tool call not found' }); + return; + } + res.status(200).json({ + conversationId, + messageId, + partIndex: result.value.partIndex, + tool_call: result.value.toolCall, + }); + } catch (error) { + logger.error('[toolCallPart] Failed to read tool call part', error); + res.status(500).json({ error: 'Internal server error' }); + } + }; +} diff --git a/packages/api/src/messages/previews.spec.ts b/packages/api/src/messages/previews.spec.ts new file mode 100644 index 0000000000..2975e7240f --- /dev/null +++ b/packages/api/src/messages/previews.spec.ts @@ -0,0 +1,627 @@ +import { ContentTypes, toolCallPreviewsConfigSchema } from 'librechat-data-provider'; +import type { AppConfig } from '@librechat/data-schemas'; +import { + previewToolCall, + previewOutputText, + previewToolCallArgs, + wantsToolCallPreviews, + containsToolCallPreviews, + prepareToolCallPreviews, + previewMessagesToolCalls, + prepareResultToolCallPreviews, + rejectToolCallPreviewWrites, + withToolCallPreviews, + withMessageToolCallPreviews, + TOOL_CALL_PREVIEW_ELISION, +} from './previews'; + +const limits = { outputChars: 1_024, argsChars: 1_024 }; + +type TestToolCall = { + type?: string; + id?: string; + name?: string; + args?: unknown; + output?: string; + executor?: string; + progress?: number; + runStepStatus?: string; + stepId?: string; + approval?: unknown; + subagent_content?: unknown[]; + outputTruncated?: true; + argsTruncated?: true; + subagentContentOmitted?: true; + function?: unknown; +}; + +type TestMessage = { messageId?: string; content?: unknown[]; updatedAt?: Date }; + +const toolPart = (toolCall: TestToolCall) => ({ + type: ContentTypes.TOOL_CALL, + tool_call: { type: 'tool_call', id: 'call_1', name: 'bash_tool', ...toolCall } as TestToolCall, +}); + +const longOutput = (head: string, tail: string, fill = 'x'.repeat(20_000)) => + `${head}${fill}${tail}`; + +describe('previewOutputText', () => { + it('keeps short output untouched', () => { + expect(previewOutputText('done', 1_024)).toBe('done'); + }); + + it('keeps the start and the end within the bound', () => { + const output = longOutput('stdout:\nhello', '\n[exit code: 2]'); + const preview = previewOutputText(output, 1_024); + expect(preview.length).toBeLessThanOrEqual(1_024); + expect(preview.startsWith('stdout:\nhello')).toBe(true); + expect(preview.endsWith('\n[exit code: 2]')).toBe(true); + expect(preview).toContain(TOOL_CALL_PREVIEW_ELISION); + }); + + it('never splits a surrogate pair at either cut', () => { + const output = '😀'.repeat(5_000); + const preview = previewOutputText(output, 1_024); + const [head, tail] = preview.split(TOOL_CALL_PREVIEW_ELISION); + expect(head).toBe('😀'.repeat(head.length / 2)); + expect(tail).toBe('😀'.repeat(tail.length / 2)); + }); +}); + +describe('previewToolCallArgs', () => { + it('returns nothing for arguments within the bound', () => { + expect(previewToolCallArgs('{"command":"ls"}', 1_024)).toBeUndefined(); + expect(previewToolCallArgs({ command: 'ls' }, 1_024)).toBeUndefined(); + }); + + it('keeps JSON string arguments parseable, with every field still present', () => { + const args = JSON.stringify({ + command: 'cat big.log', + intent: 'Reading the build log', + content: 'y'.repeat(50_000), + }); + const preview = previewToolCallArgs(args, 1_024); + expect(preview?.length).toBe(args.length); + expect(typeof preview?.args).toBe('string'); + const parsed = JSON.parse(preview?.args as string); + expect(parsed.command).toBe('cat big.log'); + expect(parsed.intent).toBe('Reading the build log'); + expect(parsed.content.startsWith('yyy')).toBe(true); + expect(parsed.content.endsWith('…')).toBe(true); + expect((preview?.args as string).length).toBeLessThanOrEqual(1_024); + }); + + it('shortens only the longest strings, keeping every short field exact', () => { + const args = JSON.stringify({ + intent: 'Patch the route so the part endpoint reuses the ownership probe', + path: 'api/server/routes/messages.js', + replace: 'r'.repeat(4_000), + search: 's'.repeat(3_000), + flags: { dryRun: false, count: 3 }, + tags: ['a', 'b'], + }); + const parsed = JSON.parse(previewToolCallArgs(args, 512)?.args as string); + expect(parsed.intent).toBe('Patch the route so the part endpoint reuses the ownership probe'); + expect(parsed.path).toBe('api/server/routes/messages.js'); + expect(parsed.flags).toEqual({ dryRun: false, count: 3 }); + expect(parsed.tags).toEqual(['a', 'b']); + expect(parsed.replace.length).toBe(parsed.search.length); + }); + + it('keeps object arguments as objects', () => { + const args = { path: 'src/index.ts', content: 'z'.repeat(10_000) }; + const preview = previewToolCallArgs(args, 1_024); + expect(preview?.args).toEqual({ path: 'src/index.ts', content: expect.any(String) }); + expect(preview?.length).toBe(JSON.stringify(args).length); + }); + + it('falls back to a bounded prefix for text that is not JSON', () => { + const args = 'q'.repeat(5_000); + const preview = previewToolCallArgs(args, 1_024); + expect(preview).toEqual({ args: 'q'.repeat(1_024), length: 5_000 }); + }); + + it('falls back to a bounded prefix when even short strings cannot fit', () => { + const args = JSON.stringify( + Object.fromEntries(Array.from({ length: 400 }, (_, i) => [`k${i}`, i])), + ); + const preview = previewToolCallArgs(args, 1_024); + expect((preview?.args as string).length).toBe(1_024); + }); +}); + +describe('previewToolCall', () => { + it('shortens a settled call and records what it shortened', () => { + const output = longOutput('Error: failed', '\n Please fix your mistakes.'); + const args = JSON.stringify({ command: 'run', content: 'a'.repeat(5_000) }); + const toolCall = toolPart({ output, args, progress: 1 }).tool_call; + const preview = previewToolCall(toolCall, limits); + + expect(preview).not.toBe(toolCall); + expect(preview).toMatchObject({ + id: 'call_1', + name: 'bash_tool', + progress: 1, + outputTruncated: true, + outputLength: output.length, + argsTruncated: true, + argsLength: args.length, + }); + expect(preview.output).toMatch(/^Error: failed/); + expect(preview.output).toMatch(/\n Please fix your mistakes\.$/); + expect(toolCall.output).toBe(output); + }); + + it('keeps JSON output parseable, so result cards still read their fields', () => { + const output = JSON.stringify({ + status: 'completed', + task_id: 'task_1', + output: 'line\n'.repeat(5_000), + }); + const preview = previewToolCall( + toolPart({ name: 'check_background_task', output }).tool_call, + limits, + ); + expect(preview.outputTruncated).toBe(true); + const parsed = JSON.parse(preview.output as string); + expect(parsed).toMatchObject({ status: 'completed', task_id: 'task_1' }); + expect((preview.output as string).length).toBeLessThanOrEqual(limits.outputChars); + }); + + it('keeps an attached-workspace exit trailer whole even past half the bound', () => { + const trailer = + '\n[exit code: 1][timed out]\nCommand reached timeoutMs: 120000. ' + + 'r'.repeat(300) + + '\n[directory hint: ' + + 'h'.repeat(300) + + ']'; + const output = `[starting directory: "workspace/"]\nstdout:\n${'o'.repeat(20_000)}\n${trailer}`; + const attached = previewToolCall( + toolPart({ output, executor: 'attached_workspace' }).tool_call, + limits, + ); + expect(attached.output?.endsWith(trailer)).toBe(true); + expect(attached.output?.startsWith('[starting directory: "workspace/"]\nstdout:\n')).toBe(true); + + const sandbox = previewToolCall(toolPart({ output }).tool_call, limits); + expect(sandbox.output?.length).toBeLessThanOrEqual(limits.outputChars); + expect(sandbox.output?.endsWith(trailer)).toBe(false); + }); + + it('leaves a call without an id whole, since it could not be fetched back by identity', () => { + const anonymous = toolPart({ id: '', output: 'o'.repeat(5_000) }).tool_call; + expect(previewToolCall(anonymous, limits)).toBe(anonymous); + const missing = { ...toolPart({ output: 'o'.repeat(5_000) }).tool_call, id: undefined }; + expect(previewToolCall(missing, limits)).toBe(missing); + }); + + it('returns the same object when nothing exceeds the bounds', () => { + const toolCall = toolPart({ output: 'ok', args: '{}' }).tool_call; + expect(previewToolCall(toolCall, limits)).toBe(toolCall); + }); + + it('leaves calls without output untouched, pending approvals included', () => { + const args = JSON.stringify({ command: 'rm -rf build', content: 'a'.repeat(5_000) }); + const pending = toolPart({ + args, + approval: { actionId: 'a1', allowed_decisions: ['edit'] }, + }).tool_call; + expect(previewToolCall(pending, limits)).toBe(pending); + const running = toolPart({ args, output: '' }).tool_call; + expect(previewToolCall(running, limits)).toBe(running); + }); + + it('omits a settled subagent transcript and counts its parts', () => { + const transcript = [ + { type: ContentTypes.TEXT, text: 'child says hi' }, + toolPart({ id: 'child_call', output: 'done' }), + ]; + const toolCall = toolPart({ + name: 'subagent', + args: '{"prompt":"look"}', + output: 'summary', + subagent_content: transcript, + }).tool_call; + const preview = previewToolCall(toolCall, limits); + expect(preview).not.toHaveProperty('subagent_content'); + expect(preview).toMatchObject({ subagentContentOmitted: true, subagentContentParts: 2 }); + expect(preview.output).toBe('summary'); + }); + + it('keeps a subagent transcript that still holds an unresolved approval', () => { + const transcript = [ + toolPart({ + id: 'child_call', + args: '{}', + approval: { actionId: 'a1', allowed_decisions: ['approve'] }, + }), + ]; + const toolCall = toolPart({ + name: 'subagent', + output: 'partial', + subagent_content: transcript, + }).tool_call; + expect(previewToolCall(toolCall, limits)).toBe(toolCall); + }); + + it('omits the transcript of a subagent that finished without final text', () => { + const transcript = [{ type: ContentTypes.TEXT, text: 'searched and found nothing' }]; + for (const finished of [{ progress: 1 }, { runStepStatus: 'completed' }]) { + const toolCall = toolPart({ + name: 'subagent', + subagent_content: transcript, + ...finished, + }).tool_call; + const preview = previewToolCall(toolCall, limits); + expect(preview).toMatchObject({ subagentContentOmitted: true, subagentContentParts: 1 }); + expect(preview).not.toHaveProperty('outputTruncated'); + } + }); + + it('keeps a subagent transcript while the subagent is still running', () => { + const transcript = [{ type: ContentTypes.TEXT, text: 'working' }]; + const toolCall = toolPart({ name: 'subagent', subagent_content: transcript }).tool_call; + expect(previewToolCall(toolCall, limits)).toBe(toolCall); + }); + + it('leaves image cards whole, since their details dialog shows the prompt directly', () => { + for (const name of ['image_gen_oai', 'image_edit_oai', 'gemini_image_gen']) { + const image = toolPart({ + name, + args: JSON.stringify({ prompt: 'p'.repeat(4_000) }), + output: 'o'.repeat(4_000), + }).tool_call; + expect(previewToolCall(image, limits)).toBe(image); + } + }); + + it('sends a background-task result whole when no valid JSON preview fits', () => { + const tasks = Array.from({ length: 200 }, (_, i) => ({ + task_id: `task_${i}`, + status: i === 7 ? 'error' : 'completed', + tool: 'bash_tool', + })); + const output = JSON.stringify({ + tasks, + partial: true, + warning: 'Some tasks are still running', + }); + const check = toolPart({ name: 'check_background_task', output }).tool_call; + expect(previewToolCall(check, limits).output).toBe(output); + + const other = toolPart({ name: 'list_issues_mcp_linear', output }).tool_call; + const otherPreview = previewToolCall(other, limits); + expect(otherPreview.outputTruncated).toBe(true); + expect(otherPreview.output?.length).toBeLessThanOrEqual(limits.outputChars); + }); + + it('keeps a legacy web-search error whole, since its card reads the phrase anywhere', () => { + const output = `${'result '.repeat(500)}Error processing request${' tail'.repeat(500)}`; + const search = toolPart({ name: 'web_search', output }).tool_call; + expect(previewToolCall(search, limits).output).toBe(output); + const plain = toolPart({ name: 'web_search', output: 'result '.repeat(2_000) }).tool_call; + expect(previewToolCall(plain, limits).outputTruncated).toBe(true); + }); + + it('leaves the question-and-answer record and legacy Assistants calls whole', () => { + const ask = toolPart({ name: 'ask_user_question', output: 'a'.repeat(5_000) }).tool_call; + expect(previewToolCall(ask, limits)).toBe(ask); + const legacy = { type: 'function', function: { name: 'x', output: 'a'.repeat(5_000) } }; + expect(previewToolCall(legacy, limits)).toBe(legacy); + }); +}); + +describe('wide or unusual JSON', () => { + it('falls back to text for a wide array without cloning it', () => { + const output = JSON.stringify(Array.from({ length: 200_000 }, (_, i) => i)); + const stringify = jest.spyOn(JSON, 'stringify'); + const preview = previewToolCall(toolPart({ output }).tool_call, limits); + expect(stringify).not.toHaveBeenCalled(); + stringify.mockRestore(); + expect(preview.outputTruncated).toBe(true); + expect(preview.output?.length).toBeLessThanOrEqual(limits.outputChars); + }); + + it('falls back to text for an oversized property name without cloning it', () => { + const output = `{"${'k'.repeat(200_000)}":1}`; + const stringify = jest.spyOn(JSON, 'stringify'); + const preview = previewToolCall(toolPart({ output }).tool_call, limits); + expect(stringify).not.toHaveBeenCalled(); + stringify.mockRestore(); + expect(preview.outputTruncated).toBe(true); + expect(preview.output?.length).toBeLessThanOrEqual(limits.outputChars); + }); + + it('refuses a wide array without queueing its elements', () => { + const output = JSON.stringify(Array.from({ length: 300_000 }, () => 1)); + const push = jest.spyOn(Array.prototype, 'push'); + const preview = previewToolCall(toolPart({ output }).tool_call, limits); + const pushes = push.mock.calls.length; + push.mockRestore(); + expect(pushes).toBeLessThan(1_000); + expect(preview.outputTruncated).toBe(true); + }); + + it('keeps a __proto__ key as ordinary data', () => { + const args = `{"__proto__":{"mode":"x"},"pad":"${'p'.repeat(4_000)}"}`; + const preview = previewToolCallArgs(args, 512); + const parsed = JSON.parse(preview?.args as string); + expect(Object.prototype.hasOwnProperty.call(parsed, '__proto__')).toBe(true); + expect(parsed.__proto__).toEqual({ mode: 'x' }); + expect(Object.prototype.hasOwnProperty.call(parsed, 'mode')).toBe(false); + }); +}); + +describe('JSON numbers in previews', () => { + const pad = 'p'.repeat(5_000); + + it.each([ + ['an integer past 2^53', '9007199254740993'], + ['an exponent past the double range', '1e400'], + ['a decimal finer than a double holds', '0.1000000000000000000001'], + ['an underflowing exponent', '1e-400'], + ])('falls back to text rather than change %s', (_label, token) => { + const args = `{"id":${token},"pad":"${pad}"}`; + const preview = previewToolCallArgs(args, 1_024); + expect(preview?.args).toBe(args.slice(0, 1_024)); + expect(preview?.args).toContain(token); + + const output = `{"id":${token},"items":[${token}],"pad":"${pad}"}`; + const call = previewToolCall(toolPart({ name: 'list_issues_mcp', output }).tool_call, limits); + expect(call.output).toContain(token); + expect(call.output).toContain(TOOL_CALL_PREVIEW_ELISION); + }); + + it('keeps the structure for numbers a double holds exactly, however they are written', () => { + const args = `{"a":1.50,"b":15e-1,"c":-0,"d":9007199254740991,"e":"1e400","pad":"${pad}"}`; + const preview = previewToolCallArgs(args, 1_024); + const parsed = JSON.parse(preview?.args as string); + expect(parsed).toMatchObject({ a: 1.5, b: 1.5, c: 0, d: 9007199254740991, e: '1e400' }); + }); + + it('sends a background-task result whole rather than as unparseable text', () => { + const output = `{"status":"completed","task_id":"task_1","bytes":9007199254740993,"output":"${pad}"}`; + const check = toolPart({ name: 'check_background_task', output }).tool_call; + expect(previewToolCall(check, limits).output).toBe(output); + }); +}); + +describe('deeply nested JSON output', () => { + it('falls back to a bounded text preview instead of exhausting the stack', () => { + const output = `${'['.repeat(20_000)}${']'.repeat(20_000)}`; + const preview = previewToolCall(toolPart({ output }).tool_call, limits); + expect(preview.outputTruncated).toBe(true); + expect(preview.output?.length).toBeLessThanOrEqual(limits.outputChars); + }); +}); + +describe('previewMessagesToolCalls', () => { + it('copies only the messages and parts that change, keeping content positions', () => { + const untouched = { messageId: 'm1', content: [{ type: ContentTypes.TEXT, text: 'hello' }] }; + const textPart = { type: ContentTypes.TEXT, text: 'before' }; + const bigPart = toolPart({ output: 'o'.repeat(10_000) }); + const smallPart = toolPart({ id: 'call_2', output: 'small' }); + const changed = { messageId: 'm2', content: [textPart, bigPart, smallPart] }; + const messages: TestMessage[] = [untouched, changed, { messageId: 'm3' }]; + + const result = previewMessagesToolCalls(messages, limits); + expect(result).not.toBe(messages); + expect(result[0]).toBe(untouched); + expect(result[2]).toBe(messages[2]); + expect(result[1].content).toHaveLength(3); + expect(result[1].content?.[0]).toBe(textPart); + expect(result[1].content?.[2]).toBe(smallPart); + expect(result[1].content?.[1]).toMatchObject({ tool_call: { outputTruncated: true } }); + expect(bigPart.tool_call.output).toHaveLength(10_000); + }); + + it('previews a call only when its identity is unique in the message', () => { + const repeated = (agentId: string) => ({ + ...toolPart({ id: 'call_dup', stepId: 'step_1', output: 'o'.repeat(10_000) }), + agentId, + }); + const content = [repeated('a'), repeated('a'), repeated('a'), repeated('b')]; + const [message] = previewMessagesToolCalls([{ content }], limits); + const parts = message.content as Array<{ tool_call: TestToolCall }>; + expect(parts[0]).toBe(content[0]); + expect(parts[1]).toBe(content[1]); + expect(parts[2]).toBe(content[2]); + expect(parts[3].tool_call.outputTruncated).toBe(true); + }); + + it('stamps each preview with when its message last changed', () => { + const updatedAt = new Date('2026-10-05T01:00:00Z'); + const [message] = previewMessagesToolCalls( + [ + { + updatedAt, + content: [toolPart({ output: 'o'.repeat(10_000) }), toolPart({ id: 'b', output: 'ok' })], + }, + ], + limits, + ); + const parts = message.content as Array<{ tool_call: Record }>; + expect(parts[0].tool_call.previewRevision).toBe(String(updatedAt.getTime())); + expect(parts[1].tool_call).not.toHaveProperty('previewRevision'); + }); + + it('returns the same array when nothing needs shortening', () => { + const messages: TestMessage[] = [{ content: [toolPart({ output: 'ok' })] }]; + expect(previewMessagesToolCalls(messages, limits)).toBe(messages); + }); +}); + +describe('prepareToolCallPreviews', () => { + const messages: TestMessage[] = [{ content: [toolPart({ output: 'o'.repeat(10_000) })] }]; + const config = (toolCallPreviews: Partial) => + ({ + toolCallPreviews: toolCallPreviewsConfigSchema.parse(toolCallPreviews), + }) as AppConfig; + + it('sends the full payload to a client that did not ask, without reading config', async () => { + const getAppConfig = jest.fn(); + const preview = prepareToolCallPreviews({ query: {} }, { getAppConfig }); + await expect(preview(messages)).resolves.toBe(messages); + expect(getAppConfig).not.toHaveBeenCalled(); + }); + + it('ignores an unknown preview version', () => { + expect(wantsToolCallPreviews({ toolPreviews: '2' })).toBe(false); + expect(wantsToolCallPreviews({ toolPreviews: ['1'] })).toBe(false); + expect(wantsToolCallPreviews({ toolPreviews: '1' })).toBe(true); + }); + + it('starts the config read immediately and applies its bounds', async () => { + const getAppConfig = jest.fn().mockResolvedValue(config({ outputChars: 512 })); + const preview = prepareToolCallPreviews( + { query: { toolPreviews: '1' }, user: { id: 'u1', role: 'USER', tenantId: 't1' } }, + { getAppConfig }, + ); + expect(getAppConfig).toHaveBeenCalledWith( + expect.objectContaining({ userId: 'u1', role: 'USER', tenantId: 't1' }), + ); + const [message] = await preview(messages); + const part = message.content?.[0] as ReturnType; + expect(part.tool_call.output?.length).toBeLessThanOrEqual(512); + }); + + it('sends in full when the deployment turned previews off', async () => { + const getAppConfig = jest.fn().mockResolvedValue(config({ enabled: false })); + const preview = prepareToolCallPreviews({ query: { toolPreviews: '1' } }, { getAppConfig }); + await expect(preview(messages)).resolves.toBe(messages); + }); + + it('uses the default bounds when config cannot be read', async () => { + const getAppConfig = jest.fn().mockRejectedValue(new Error('config down')); + const preview = prepareToolCallPreviews({ query: { toolPreviews: '1' } }, { getAppConfig }); + const [message] = await preview(messages); + expect(message.content?.[0]).toMatchObject({ tool_call: { outputTruncated: true } }); + }); +}); + +describe('prepareResultToolCallPreviews', () => { + const result = () => ({ + conversation: { conversationId: 'fork-1' }, + messages: [{ messageId: 'm1', content: [toolPart({ output: 'o'.repeat(10_000) })] }], + }); + const config = (enabled: boolean) => + ({ toolCallPreviews: toolCallPreviewsConfigSchema.parse({ enabled }) }) as AppConfig; + + it("reads the requester's bounds, ignoring the config the route resolved", async () => { + const getAppConfig = jest.fn().mockResolvedValue(config(true)); + const req = { + query: { toolPreviews: '1' }, + user: { id: 'viewer', role: 'USER', tenantId: 'tenant-viewer' }, + config: config(false), + }; + const preview = prepareResultToolCallPreviews(req, { getAppConfig }); + expect(getAppConfig).toHaveBeenCalledWith( + expect.objectContaining({ userId: 'viewer', tenantId: 'tenant-viewer' }), + ); + const original = result(); + const previewed = await preview(original); + expect(previewed.conversation).toBe(original.conversation); + expect(previewed.messages[0].content[0]).toMatchObject({ + tool_call: { outputTruncated: true }, + }); + }); + + it('returns the result untouched when the requester opted out or did not ask', async () => { + const original = result(); + const optedOut = prepareResultToolCallPreviews( + { query: { toolPreviews: '1' } }, + { getAppConfig: jest.fn().mockResolvedValue(config(false)) }, + ); + await expect(optedOut(original)).resolves.toBe(original); + + const getAppConfig = jest.fn(); + const notAsked = prepareResultToolCallPreviews({ query: {} }, { getAppConfig }); + await expect(notAsked(original)).resolves.toBe(original); + expect(getAppConfig).not.toHaveBeenCalled(); + }); +}); + +describe('withToolCallPreviews', () => { + const result = () => ({ + conversation: { conversationId: 'fork-1' }, + messages: [{ messageId: 'm1', content: [toolPart({ output: 'o'.repeat(10_000) })] }], + }); + const config = (toolCallPreviews: Partial = {}) => ({ + toolCallPreviews: toolCallPreviewsConfigSchema.parse(toolCallPreviews), + }); + + it('previews a fork or duplicate response for a client that asked', () => { + const original = result(); + const previewed = withToolCallPreviews( + { query: { toolPreviews: '1' }, config: config() }, + original, + ); + expect(previewed).not.toBe(original); + expect(previewed.conversation).toBe(original.conversation); + expect(previewed.messages[0].content[0]).toMatchObject({ + tool_call: { outputTruncated: true }, + }); + expect(original.messages[0].content[0].tool_call.output).toHaveLength(10_000); + }); + + it('returns the response untouched otherwise', () => { + const original = result(); + expect(withToolCallPreviews({ query: {}, config: config() }, original)).toBe(original); + expect( + withToolCallPreviews( + { query: { toolPreviews: '1' }, config: config({ enabled: false }) }, + original, + ), + ).toBe(original); + const noMessages: { conversation: object; messages?: TestMessage[] } = { + conversation: { conversationId: 'x' }, + }; + expect(withToolCallPreviews({ query: { toolPreviews: '1' } }, noMessages)).toBe(noMessages); + }); +}); + +describe('withMessageToolCallPreviews', () => { + it('previews one returned message for a client that asked, and only then', () => { + const message = { + conversationId: 'c1', + text: '', + content: [toolPart({ output: 'o'.repeat(10_000) })], + }; + const config = { toolCallPreviews: toolCallPreviewsConfigSchema.parse({}) }; + const previewed = withMessageToolCallPreviews( + { query: { toolPreviews: '1' }, config }, + message, + ); + expect(previewed).toMatchObject({ conversationId: 'c1', text: '' }); + expect(previewed.content[0]).toMatchObject({ tool_call: { outputTruncated: true } }); + expect(withMessageToolCallPreviews({ query: {}, config }, message)).toBe(message); + }); +}); + +describe('rejectToolCallPreviewWrites', () => { + const run = (content: unknown) => { + const json = jest.fn(); + const status = jest.fn(() => ({ json })); + const next = jest.fn(); + rejectToolCallPreviewWrites({ body: { content } }, { status }, next); + return { status, json, next }; + }; + + it('refuses content carrying a preview, nested transcripts included', () => { + const top = run([toolPart({ output: 'x', outputTruncated: true })]); + expect(top.status).toHaveBeenCalledWith(400); + expect(top.next).not.toHaveBeenCalled(); + + const nested = run([ + toolPart({ name: 'subagent', subagent_content: [toolPart({ argsTruncated: true })] }), + ]); + expect(nested.status).toHaveBeenCalledWith(400); + expect(containsToolCallPreviews([toolPart({ subagentContentOmitted: true })])).toBe(true); + }); + + it('passes full content through', () => { + const result = run([toolPart({ output: 'x'.repeat(10_000) })]); + expect(result.next).toHaveBeenCalled(); + expect(result.status).not.toHaveBeenCalled(); + }); +}); diff --git a/packages/api/src/messages/previews.ts b/packages/api/src/messages/previews.ts new file mode 100644 index 0000000000..a7de540b3f --- /dev/null +++ b/packages/api/src/messages/previews.ts @@ -0,0 +1,750 @@ +import { logger } from '@librechat/data-schemas'; +import { + Tools, + Constants, + ContentTypes, + TOOL_CALL_PREVIEWS_PARAM, + TOOL_CALL_PREVIEWS_VERSION, + COMMAND_RESULT_TRAILER, + hasToolCallPreview, + toolCallPreviewsConfigSchema, +} from 'librechat-data-provider'; +import type { TToolCallPreviewsConfig, ToolCallPreviewMarkers } from 'librechat-data-provider'; +import type { AppConfig } from '@librechat/data-schemas'; +import type { AppConfigUserLike, GetAppConfigOptions } from '~/app/service'; +import { getAppConfigOptionsFromUser } from '~/app/service'; + +export type ToolCallPreviewLimits = Pick; + +/** Separates the kept start and end of an output preview. */ +export const TOOL_CALL_PREVIEW_ELISION = '\n…\n'; + +/** Share of a text output preview taken from the start; the rest keeps the end. */ +const OUTPUT_HEAD_SHARE = 0.5; + +/** Shortest a string inside a JSON preview gets before the structure is abandoned for text. */ +const MIN_JSON_STRING_CHARS = 8; + +/** Deepest JSON nesting a structure-preserving preview walks; deeper values fall back to text. */ +const MAX_JSON_DEPTH = 64; + +/** Ceiling on a kept exit-status trailer, so a malformed one cannot defeat the bound. */ +const MAX_TRAILER_CHARS = 2_048; + +/** + * Tools whose cards show their full record without going through a card disclosure, so a + * preview would be what the reader sees: the question-and-answer record reads both fields + * directly, and an image card's details dialog shows the prompt from its arguments. + */ +const FULL_CONTENT_TOOLS: ReadonlySet = new Set([ + 'ask_user_question', + 'image_gen_oai', + 'image_edit_oai', + 'gemini_image_gen', +]); + +/** + * Tools whose collapsed card parses its JSON output for the row's verdict (task failures, + * partial results, warnings). When even a structure-preserving preview cannot fit, their output + * goes in full rather than as text that no longer parses. + */ +const PARSED_OUTPUT_TOOLS: ReadonlySet = new Set([ + Constants.CHECK_BACKGROUND_TASK as string, +]); + +/** + * Older web-search failures carry only this phrase, anywhere in the output, and the collapsed + * card reads it to label the search as not completed; such output goes whole. + */ +const LEGACY_SEARCH_ERROR = /error processing/i; + +type JsonValue = string | number | boolean | null | JsonValue[] | { [key: string]: JsonValue }; + +/** The stored fields this module reads; everything else on the tool call passes through. */ +interface StoredToolCall extends ToolCallPreviewMarkers { + type?: string; + id?: unknown; + stepId?: unknown; + name?: string; + executor?: string; + progress?: unknown; + runStepStatus?: unknown; + output?: unknown; + args?: unknown; + approval?: unknown; + subagent_content?: unknown; +} + +interface StoredToolCallPart { + type?: string; + agentId?: unknown; + tool_call?: StoredToolCall; +} + +/** A message as read for a client; only `content` is inspected. */ +export interface PreviewableMessage { + content?: unknown[] | null; + /** Stamped onto each preview as `previewRevision`, so clients key fetched parts to it. */ + updatedAt?: Date | string | number | null; +} + +const isObject = (value: unknown): value is object => value != null && typeof value === 'object'; + +const isToolCallPart = ( + part: unknown, +): part is StoredToolCallPart & { tool_call: StoredToolCall } => + isObject(part) && + (part as StoredToolCallPart).type === ContentTypes.TOOL_CALL && + isObject((part as StoredToolCallPart).tool_call); + +/** Agent tool calls only; legacy Assistants shapes keep their own nested output fields. */ +const isAgentToolCall = (toolCall: StoredToolCall): boolean => + toolCall.type == null || toolCall.type === 'tool_call'; + +const hasOutput = (toolCall: StoredToolCall): boolean => + typeof toolCall.output === 'string' && toolCall.output.length > 0; + +/** Only a call with an id can be fetched back by identity, so only such calls are previewed. */ +const hasIdentity = (toolCall: StoredToolCall): boolean => + typeof toolCall.id === 'string' && toolCall.id !== ''; + +/** + * The identity the client sends to fetch a part back: provider call id, host run-step id and + * the part's agent, matched exactly by the part endpoint. Provider ids can repeat within a + * response, so a call is previewed only when this identity is unique in its message; otherwise + * no request could name it without also naming another call. + */ +function toolCallIdentity(part: StoredToolCallPart & { tool_call: StoredToolCall }): string | null { + const { id, stepId } = part.tool_call; + if (typeof id !== 'string' || id === '') { + return null; + } + const step = typeof stepId === 'string' ? stepId : ''; + const agent = typeof part.agentId === 'string' ? part.agentId : ''; + return `${id}\u0000${step}\u0000${agent}`; +} + +/** + * A call whose content no card still acts on: it has output, or its run step closed or reached + * full progress, and it is not waiting on an approval. A subagent can finish with no final text, + * and its transcript is as large as any other. + */ +const isSettled = (toolCall: StoredToolCall): boolean => { + if (toolCall.approval != null && !hasOutput(toolCall)) { + return false; + } + return ( + hasOutput(toolCall) || + toolCall.runStepStatus != null || + (typeof toolCall.progress === 'number' && toolCall.progress >= 1) + ); +}; + +/** An unresolved approval anywhere in the parts, including inside nested subagent runs. */ +function hasPendingApproval(parts: unknown[]): boolean { + const stack: unknown[][] = [parts]; + while (stack.length > 0) { + const current = stack.pop() ?? []; + for (const part of current) { + if (!isToolCallPart(part)) { + continue; + } + const toolCall = part.tool_call; + if (toolCall.approval != null && !hasOutput(toolCall)) { + return true; + } + if (Array.isArray(toolCall.subagent_content)) { + stack.push(toolCall.subagent_content); + } + } + } + return false; +} + +/** Cuts at `end` without splitting a surrogate pair. */ +function sliceStart(text: string, end: number): string { + const code = text.charCodeAt(end - 1); + return text.slice(0, code >= 0xd800 && code <= 0xdbff ? end - 1 : end); +} + +/** Keeps the last `count` code units without starting inside a surrogate pair. */ +function sliceEnd(text: string, count: number): string { + if (count <= 0) { + return ''; + } + const start = text.length - count; + const code = text.charCodeAt(start); + return text.slice(code >= 0xdc00 && code <= 0xdfff ? start + 1 : start); +} + +/** + * Keeps the start and the end. Error prefixes and first error lines live at the start; + * validation feedback, the last line of a traceback and exit-status trailers live at the end. + * `minTailChars` widens the end to keep a trailer whole. + */ +export function previewOutputText(output: string, maxChars: number, minTailChars = 0): string { + if (output.length <= maxChars) { + return output; + } + const budget = Math.max(0, maxChars - TOOL_CALL_PREVIEW_ELISION.length); + const tailChars = Math.max(budget - Math.floor(budget * OUTPUT_HEAD_SHARE), minTailChars); + const headChars = Math.max(0, budget - tailChars); + if (headChars + tailChars + TOOL_CALL_PREVIEW_ELISION.length >= output.length) { + return output; + } + return `${sliceStart(output, headChars)}${TOOL_CALL_PREVIEW_ELISION}${sliceEnd(output, tailChars)}`; +} + +function capJsonStrings(value: JsonValue, maxChars: number): JsonValue { + if (typeof value === 'string') { + return value.length > maxChars ? `${sliceStart(value, maxChars)}…` : value; + } + if (Array.isArray(value)) { + return value.map((item) => capJsonStrings(item, maxChars)); + } + if (value == null || typeof value !== 'object') { + return value; + } + /** Null-prototype, so a `__proto__` key stays an ordinary data key. */ + const result: { [key: string]: JsonValue } = Object.create(null); + for (const key in value) { + result[key] = capJsonStrings(value[key], maxChars); + } + return result; +} + +/** + * Whether the JSON could fit `maxChars` once its strings are capped: every value serializes to + * at least two characters with its separator, every string to at least `MIN_JSON_STRING_CHARS`, + * and property names are never shortened, so a structure with more values, strings or key + * characters than that allows cannot fit at any cap. The walk is iterative and counts children + * before queueing them, so its work list never outgrows the node budget; it stops as soon as any + * bound is passed and treats nesting deeper than `MAX_JSON_DEPTH` as unfit, so the cloning and + * serialization that follow only ever run on small, shallow values. + */ +function mayFitJson(value: JsonValue, maxChars: number): boolean { + const maxNodes = Math.floor(maxChars / 2); + const maxStrings = Math.floor(maxChars / MIN_JSON_STRING_CHARS); + let nodes = 0; + let strings = 0; + let keyChars = 0; + const stack: Array<[JsonValue, number]> = [[value, 0]]; + while (stack.length > 0) { + const [current, depth] = stack.pop() as [JsonValue, number]; + nodes++; + if (nodes > maxNodes || depth > MAX_JSON_DEPTH) { + return false; + } + if (typeof current === 'string') { + strings++; + if (strings > maxStrings) { + return false; + } + } else if (Array.isArray(current)) { + if (nodes + stack.length + current.length > maxNodes) { + return false; + } + for (const item of current) { + stack.push([item, depth + 1]); + } + } else if (current != null && typeof current === 'object') { + for (const key in current) { + keyChars += key.length + 3; + if (keyChars > maxChars || nodes + stack.length + 1 > maxNodes) { + return false; + } + stack.push([current[key], depth + 1]); + } + } + } + return true; +} + +function parseJsonContainer(text: string): JsonValue | undefined { + const first = text.trimStart()[0]; + if (first !== '{' && first !== '[') { + return undefined; + } + try { + return JSON.parse(text) as JsonValue; + } catch { + return undefined; + } +} + +const isDigitCode = (code: number): boolean => code >= 48 && code <= 57; + +/** `0-9 . e E + -`: the characters a JSON number token is made of. */ +const isNumberCode = (code: number): boolean => + isDigitCode(code) || code === 46 || code === 101 || code === 69 || code === 43 || code === 45; + +/** A decimal as `[-]digits e exponent` with no leading or trailing zeros, so `1.50` and `15e-1` + * compare equal and only a change of value shows. */ +function canonicalDecimal(token: string): string { + const negative = token.startsWith('-'); + const unsigned = negative ? token.slice(1) : token; + const exponentAt = unsigned.search(/[eE]/); + const mantissa = exponentAt === -1 ? unsigned : unsigned.slice(0, exponentAt); + const pointAt = mantissa.indexOf('.'); + const fraction = pointAt === -1 ? '' : mantissa.slice(pointAt + 1); + const digits = (pointAt === -1 ? mantissa : mantissa.slice(0, pointAt)) + fraction; + const significant = digits.replace(/^0+/, ''); + if (significant === '') { + return '0'; + } + const trimmed = significant.replace(/0+$/, ''); + const exponent = + (exponentAt === -1 ? 0 : Number(unsigned.slice(exponentAt + 1))) - + fraction.length + + (significant.length - trimmed.length); + return `${negative ? '-' : ''}${trimmed}e${exponent}`; +} + +/** Whether the number survives a parse and re-serialization as the same value. */ +function numberRoundTrips(token: string): boolean { + const value = Number(token); + return Number.isFinite(value) && canonicalDecimal(token) === canonicalDecimal(String(value)); +} + +/** + * Whether every number in the (already valid) JSON text comes back as the same value once + * parsed into doubles and serialized again. An integer past 2^53, a decimal with more precision + * than a double holds, or an exponent past its range would otherwise show a different value + * (or `null`) in the preview than the tool actually produced. + */ +function numbersRoundTrip(text: string): boolean { + let i = 0; + while (i < text.length) { + const code = text.charCodeAt(i); + if (code === 34) { + i++; + while (i < text.length && text.charCodeAt(i) !== 34) { + i += text.charCodeAt(i) === 92 ? 2 : 1; + } + i++; + continue; + } + if (code !== 45 && !isDigitCode(code)) { + i++; + continue; + } + const start = i; + while (i < text.length && isNumberCode(text.charCodeAt(i))) { + i++; + } + if (!numberRoundTrips(text.slice(start, i))) { + return false; + } + } + return true; +} + +/** + * The structure-preserving preview of JSON text, or `undefined` when the caller should fall + * back: the structure cannot fit, or a number in it would not survive the round trip. + */ +function shrinkJsonText(text: string, parsed: JsonValue, maxChars: number): string | undefined { + const shrunk = shrinkJsonToFit(parsed, maxChars); + return shrunk !== undefined && numbersRoundTrip(text) ? shrunk : undefined; +} + +/** + * Serializes the JSON within `maxChars` by shortening only its longest strings: the largest + * per-string cap that fits wins, so short fields (a command, a path, an intent, a status) stay + * exact while a file body or log absorbs the cut. Every key, array item and scalar is kept, so + * readers that parse the value still find their fields. Returns `undefined` when even the + * shortest cap cannot fit, and the caller falls back to text. + */ +export function shrinkJsonToFit(value: JsonValue, maxChars: number): string | undefined { + if (!mayFitJson(value, maxChars)) { + return undefined; + } + const capped = capJsonStrings(value, maxChars); + let low = MIN_JSON_STRING_CHARS; + let high = maxChars; + let best: string | undefined; + while (low <= high) { + const cap = Math.floor((low + high) / 2); + const serialized = JSON.stringify(capJsonStrings(capped, cap)); + if (serialized.length <= maxChars) { + best = serialized; + low = cap + 1; + } else { + high = cap - 1; + } + } + return best; +} + +type ArgsPreview = { args: string | JsonValue; length: number }; + +export function previewToolCallArgs(args: unknown, maxChars: number): ArgsPreview | undefined { + if (typeof args === 'string') { + if (args.length <= maxChars) { + return undefined; + } + const parsed = parseJsonContainer(args); + const shrunk = parsed === undefined ? undefined : shrinkJsonText(args, parsed, maxChars); + return { args: shrunk ?? sliceStart(args, maxChars), length: args.length }; + } + if (!isObject(args)) { + return undefined; + } + const serialized = JSON.stringify(args); + if (serialized.length <= maxChars) { + return undefined; + } + const shrunk = shrinkJsonToFit(args as JsonValue, maxChars); + return { + args: + shrunk === undefined ? sliceStart(serialized, maxChars) : (JSON.parse(shrunk) as JsonValue), + length: serialized.length, + }; +} + +/** + * Length of the exit-status trailer the attached-workspace `bash_tool` appends. Only a call the + * server stamped with that executor carries one, and its collapsed card reads the verdict there. + */ +function commandTrailerLength(toolCall: StoredToolCall, output: string): number { + if (toolCall.executor !== 'attached_workspace') { + return 0; + } + const match = COMMAND_RESULT_TRAILER.exec(output.slice(-MAX_TRAILER_CHARS)); + return match == null ? 0 : match[0].length; +} + +/** JSON output stays parseable (background-task results, handles); other text keeps both ends. */ +export function previewToolCallOutput( + toolCall: StoredToolCall, + output: string, + maxChars: number, +): string { + if (output.length <= maxChars) { + return output; + } + const parsed = parseJsonContainer(output); + const shrunk = parsed === undefined ? undefined : shrinkJsonText(output, parsed, maxChars); + if (shrunk != null) { + return shrunk; + } + if ( + parsed !== undefined && + typeof toolCall.name === 'string' && + PARSED_OUTPUT_TOOLS.has(toolCall.name) + ) { + return output; + } + if (toolCall.name === Tools.web_search && LEGACY_SEARCH_ERROR.test(output)) { + return output; + } + return previewOutputText(output, maxChars, commandTrailerLength(toolCall, output)); +} + +/** + * Returns the tool call with long content replaced by previews, or the same object when nothing + * needed shortening. Only settled calls change: a call still waiting on output, approval or a + * reader's answer keeps every field, since its card acts on them. + */ +export function previewToolCall( + toolCall: T, + limits: ToolCallPreviewLimits, + revision = '', +): T { + if (!isAgentToolCall(toolCall) || !isSettled(toolCall) || !hasIdentity(toolCall)) { + return toolCall; + } + if (typeof toolCall.name === 'string' && FULL_CONTENT_TOOLS.has(toolCall.name)) { + return toolCall; + } + + const output = typeof toolCall.output === 'string' ? toolCall.output : ''; + const outputPreview = previewToolCallOutput(toolCall, output, limits.outputChars); + const argsPreview = previewToolCallArgs(toolCall.args, limits.argsChars); + const subagentContent = toolCall.subagent_content; + const omitSubagentContent = + Array.isArray(subagentContent) && + subagentContent.length > 0 && + !hasPendingApproval(subagentContent); + + if (outputPreview === output && argsPreview == null && !omitSubagentContent) { + return toolCall; + } + + const next: T = { ...toolCall }; + if (outputPreview !== output) { + next.output = outputPreview; + next.outputTruncated = true; + next.outputLength = output.length; + } + if (argsPreview != null) { + next.args = argsPreview.args; + next.argsTruncated = true; + next.argsLength = argsPreview.length; + } + if (omitSubagentContent) { + delete next.subagent_content; + next.subagentContentOmitted = true; + next.subagentContentParts = (subagentContent as unknown[]).length; + } + if (revision !== '') { + next.previewRevision = revision; + } + return next; +} + +/** + * A call that cannot be previewed is sent as stored rather than failing the whole conversation + * load; previews are an optimization, and the full value is what the client would otherwise get. + */ +function previewToolCallSafely( + toolCall: T, + limits: ToolCallPreviewLimits, + revision: string, +): T { + try { + return previewToolCall(toolCall, limits, revision); + } catch (error) { + logger.warn('[toolCallPreviews] Sending a tool call in full; its preview failed', error); + return toolCall; + } +} + +/** Previews every tool-call part of one content array; returns the same array when unchanged. */ +export function previewContentToolCalls( + content: unknown[], + limits: ToolCallPreviewLimits, + revision = '', +): unknown[] { + let result: unknown[] | undefined; + /** First index of each identity; a repeat sends both calls whole (see `toolCallIdentity`). */ + const firstIndex = new Map(); + for (let i = 0; i < content.length; i++) { + const part = content[i]; + if (!isToolCallPart(part)) { + continue; + } + const identity = toolCallIdentity(part); + if (identity == null) { + continue; + } + const first = firstIndex.get(identity); + if (first != null) { + if (result != null && first >= 0) { + result[first] = content[first]; + } + firstIndex.set(identity, -1); + continue; + } + firstIndex.set(identity, i); + const toolCall = previewToolCallSafely(part.tool_call, limits, revision); + if (toolCall === part.tool_call) { + continue; + } + result ??= content.slice(); + result[i] = { ...part, tool_call: toolCall }; + } + return result ?? content; +} + +function messageRevision(message: PreviewableMessage): string { + const { updatedAt } = message; + if (updatedAt instanceof Date) { + return String(updatedAt.getTime()); + } + return updatedAt == null ? '' : String(updatedAt); +} + +/** Applies previews across messages, copying only the messages whose content changed. */ +export function previewMessagesToolCalls( + messages: T[], + limits: ToolCallPreviewLimits, +): T[] { + let result: T[] | undefined; + for (let i = 0; i < messages.length; i++) { + const message = messages[i]; + if (!Array.isArray(message?.content)) { + continue; + } + const content = previewContentToolCalls(message.content, limits, messageRevision(message)); + if (content === message.content) { + continue; + } + result ??= messages.slice(); + result[i] = { ...message, content }; + } + return result ?? messages; +} + +/** True when the request asked for this preview format. */ +export function wantsToolCallPreviews(query: unknown): boolean { + if (!isObject(query)) { + return false; + } + return ( + (query as Record)[TOOL_CALL_PREVIEWS_PARAM] === TOOL_CALL_PREVIEWS_VERSION + ); +} + +export interface ToolCallPreviewRequest { + query?: unknown; + user?: AppConfigUserLike | null; +} + +export interface ToolCallPreviewDeps { + getAppConfig: (options: GetAppConfigOptions) => Promise; +} + +type MessagesPreviewer = (messages: T[]) => Promise; + +/** A response carrying the messages a fork-style route returns. */ +type MessagesResult = { messages?: PreviewableMessage[] | null }; + +export type ResultPreviewer = (result: T) => Promise; + +const sendInFull: MessagesPreviewer = (messages) => Promise.resolve(messages); + +async function resolveLimits( + req: ToolCallPreviewRequest, + deps: ToolCallPreviewDeps, +): Promise { + try { + const config = await deps.getAppConfig({ + ...getAppConfigOptionsFromUser(req.user), + skipRuntimeAugmentation: true, + }); + return config?.toolCallPreviews ?? toolCallPreviewsConfigSchema.parse({}); + } catch (error) { + logger.warn('[toolCallPreviews] Config unavailable; using default preview bounds', error); + return toolCallPreviewsConfigSchema.parse({}); + } +} + +/** + * Starts resolving the preview bounds for a message load, so the config read runs beside the + * message read instead of ahead of it. The returned function shapes the messages once both are + * in. A request without the preview parameter, or a deployment that turned previews off, + * receives the messages untouched. + */ +export function prepareToolCallPreviews( + req: ToolCallPreviewRequest, + deps: ToolCallPreviewDeps, +): MessagesPreviewer { + if (!wantsToolCallPreviews(req.query)) { + return sendInFull; + } + const limitsPromise = resolveLimits(req, deps); + return async (messages) => { + const limits = await limitsPromise; + if (!limits.enabled) { + return messages; + } + return previewMessagesToolCalls(messages, limits); + }; +} + +/** + * `prepareToolCallPreviews` for a route that returns `{ messages }` and whose `req.config` is not + * the requester's: a shared-link fork resolves the share owner's config for its content checks, + * while the forked conversation belongs to the requester, so the bounds come from the + * requester's own tenant and role. + */ +export function prepareResultToolCallPreviews( + req: ToolCallPreviewRequest, + deps: ToolCallPreviewDeps, +): ResultPreviewer { + const preview = prepareToolCallPreviews(req, deps); + return async (result) => { + if (!Array.isArray(result?.messages)) { + return result; + } + const messages = await preview(result.messages); + return messages === result.messages ? result : { ...result, messages }; + }; +} + +/** True when any tool call in the content, nested subagent runs included, carries preview markers. */ +export function containsToolCallPreviews(content: unknown): boolean { + if (!Array.isArray(content)) { + return false; + } + const stack: unknown[][] = [content]; + while (stack.length > 0) { + const current = stack.pop() ?? []; + for (const part of current) { + if (!isToolCallPart(part)) { + continue; + } + if (hasToolCallPreview(part.tool_call)) { + return true; + } + if (Array.isArray(part.tool_call.subagent_content)) { + stack.push(part.tool_call.subagent_content); + } + } + } + return false; +} + +interface PreviewableResultRequest { + query?: unknown; + config?: { toolCallPreviews?: TToolCallPreviewsConfig } | null; +} + +/** + * Previews the messages a fork, duplicate or shared-link fork returns, so a client that asked + * for previews does not seed its conversation cache with the full history. Reads the bounds + * from the config the route already resolved. + */ +export function withToolCallPreviews( + req: PreviewableResultRequest, + result: T, +): T { + if (!wantsToolCallPreviews(req.query) || !Array.isArray(result?.messages)) { + return result; + } + const limits = req.config?.toolCallPreviews ?? toolCallPreviewsConfigSchema.parse({}); + if (!limits.enabled) { + return result; + } + const messages = previewMessagesToolCalls(result.messages, limits); + return messages === result.messages ? result : { ...result, messages }; +} + +/** + * Previews one message a mutation returns (an artifact edit's content, a branched response), so + * a client that asked for previews merges a bounded copy into its conversation cache. + */ +export function withMessageToolCallPreviews( + req: PreviewableResultRequest, + message: T, +): T { + if (message == null || !Array.isArray(message.content)) { + return message; + } + return withToolCallPreviews(req, { messages: [message] }).messages[0]; +} + +interface MessageWriteRequest { + body?: { content?: unknown } | null; +} + +interface MessageWriteResponse { + status: (code: number) => { json: (body: { error: string }) => unknown }; +} + +/** + * Refuses a client-authored message whose content holds tool-call previews. A preview is a + * display copy; saving it would replace the stored output, arguments or transcript with it. + */ +export function rejectToolCallPreviewWrites( + req: MessageWriteRequest, + res: MessageWriteResponse, + next: () => void, +): void { + if (containsToolCallPreviews(req.body?.content)) { + res.status(400).json({ error: 'Tool call previews cannot be saved as message content' }); + return; + } + next(); +} diff --git a/packages/api/src/middleware/messageValidation.ts b/packages/api/src/middleware/messageValidation.ts index ba39849755..3ff6bab12a 100644 --- a/packages/api/src/middleware/messageValidation.ts +++ b/packages/api/src/middleware/messageValidation.ts @@ -71,12 +71,25 @@ export type MessageValidationDeps = { logger: MessageValidationLogger; }; +export type MessageRequestValidationOptions = { + /** + * The route reads one stored message of the conversation (a tool-call part) rather than + * addressing a message for mutation, so a conversation that is still only an active job + * before its first save stays readable to the job's owner, as the conversation read is. + */ + activeJobMessageRead?: boolean; +}; + export type MessageRequestMiddleware = { canReadActiveJobConversation: ( req: MessageValidationRequest, conversationId?: string, + options?: MessageRequestValidationOptions, ) => Promise; - createMessageRequestValidation: (req: MessageValidationRequest) => MessageRequestValidation; + createMessageRequestValidation: ( + req: MessageValidationRequest, + options?: MessageRequestValidationOptions, + ) => MessageRequestValidation; prepareMessageRequestValidation: ( req: MessageValidationRequest, res: Response, @@ -105,8 +118,12 @@ export function createMessageRequestMiddleware( async function canReadActiveJobConversation( req: MessageValidationRequest, conversationId?: string, + options?: MessageRequestValidationOptions, ): Promise { - if (!isPublicReadMethod(req.method) || req.params?.messageId) { + if (!isPublicReadMethod(req.method)) { + return false; + } + if (req.params?.messageId && options?.activeJobMessageRead !== true) { return false; } @@ -143,11 +160,12 @@ export function createMessageRequestMiddleware( async function validateConversationAccess( req: MessageValidationRequest, conversationId?: string, + options?: MessageRequestValidationOptions, ): Promise { const conversation = await deps.getConvo(req.user.id, conversationId); if (!conversation) { - if (await canReadActiveJobConversation(req, conversationId)) { + if (await canReadActiveJobConversation(req, conversationId, options)) { return { ok: true }; } @@ -172,7 +190,10 @@ export function createMessageRequestMiddleware( return { ok: true }; } - function createMessageRequestValidation(req: MessageValidationRequest): MessageRequestValidation { + function createMessageRequestValidation( + req: MessageValidationRequest, + options?: MessageRequestValidationOptions, + ): MessageRequestValidation { const body = req.body ?? {}; const paramConversationId = req.params?.conversationId; const bodyConversationId = body.conversationId; @@ -207,7 +228,7 @@ export function createMessageRequestMiddleware( return { conversationId, shouldFetchMessages: true, - promise: validateConversationAccess(req, conversationId), + promise: validateConversationAccess(req, conversationId, options), }; } diff --git a/packages/data-provider/src/api-endpoints.ts b/packages/data-provider/src/api-endpoints.ts index 7b182030c8..7432e3e506 100644 --- a/packages/data-provider/src/api-endpoints.ts +++ b/packages/data-provider/src/api-endpoints.ts @@ -76,6 +76,27 @@ export const messages = (params: q.MessagesListParams) => { export const messagesArtifacts = (messageId: string) => `${messagesRoot}/artifact/${messageId}`; +/** One tool-call part of a stored message, in full. */ +export const messageToolCallPart = ({ + conversationId, + messageId, + partIndex, + toolCallId, + stepId, + agentId, +}: q.ToolCallPartParams) => { + const path = `${messagesRoot}/${encodeURIComponent(conversationId)}/${encodeURIComponent(messageId)}/parts/${partIndex}`; + const query = [ + ['toolCallId', toolCallId], + ['stepId', stepId], + ['agentId', agentId], + ] + .filter((entry): entry is [string, string] => typeof entry[1] === 'string' && entry[1] !== '') + .map(([key, value]) => `${key}=${encodeURIComponent(value)}`) + .join('&'); + return query === '' ? path : `${path}?${query}`; +}; + export const messagesBranch = () => `${messagesRoot}/branch`; const shareRoot = `${BASE_URL}/api/share`; diff --git a/packages/data-provider/src/config.ts b/packages/data-provider/src/config.ts index 3a4e7d2500..9dccc8295f 100644 --- a/packages/data-provider/src/config.ts +++ b/packages/data-provider/src/config.ts @@ -3735,6 +3735,24 @@ export const conversationListConfigSchema = z.object({ export type TConversationListConfig = z.infer; +/** + * Bounded previews of settled tool calls on conversation loads. A client that asks for them + * receives the start and end of each long output and argument string, without subagent + * transcripts, and fetches a part in full only when the reader opens it. `enabled: false` sends + * every load in full, as before previews existed. + */ +export const toolCallPreviewsConfigSchema = z.object({ + enabled: z.boolean().default(true), + /** Characters of `output` kept per tool call, half from the start and half from the end. + * JSON output stays valid JSON, and an exit-status trailer is always kept whole. */ + outputChars: z.number().int().min(256).max(1_000_000).default(512), + /** Characters of serialized `args` kept per tool call. JSON arguments stay valid JSON with + * every field present; only the longest strings are shortened. */ + argsChars: z.number().int().min(256).max(1_000_000).default(512), +}); + +export type TToolCallPreviewsConfig = z.infer; + export const configSchema = z.object({ version: z.string(), permissions: z.object({ maxWriteAttempts: permissionWriteAttemptsSchema }).optional(), @@ -3751,6 +3769,9 @@ export const configSchema = z.object({ conversationList: conversationListConfigSchema.default(() => conversationListConfigSchema.parse({}), ), + toolCallPreviews: toolCallPreviewsConfigSchema.default(() => + toolCallPreviewsConfigSchema.parse({}), + ), includedTools: z.array(z.string()).optional(), filteredTools: z.array(z.string()).optional(), mcpServers: MCPServersSchema.optional(), diff --git a/packages/data-provider/src/createPayload.spec.ts b/packages/data-provider/src/createPayload.spec.ts new file mode 100644 index 0000000000..29065582f2 --- /dev/null +++ b/packages/data-provider/src/createPayload.spec.ts @@ -0,0 +1,55 @@ +import type { TSubmission } from './types'; +import type { TMessage } from './schemas'; +import createPayload from './createPayload'; +import { EModelEndpoint } from './schemas'; + +const previewedResponse = { + messageId: 'response-1', + conversationId: 'convo-1', + parentMessageId: 'user-1', + isCreatedByUser: false, + text: '', + content: [ + { + type: 'tool_call', + tool_call: { + id: 'call_1', + name: 'bash_tool', + args: '{"command":"make"}', + output: 'head…tail', + outputTruncated: true, + outputLength: 40_000, + }, + }, + ], +} as unknown as TMessage; + +describe('createPayload', () => { + it.each([ + ['regenerate', { isRegenerate: true }], + ['continue', { isEdited: true, isContinued: true }], + ['edit and rerun', { isEdited: true, editedContent: { index: 0, type: 'text', text: 'x' } }], + ])('never carries cached tool-call content on %s', (_name, flags) => { + const submission = { + userMessage: { + messageId: 'user-1', + conversationId: 'convo-1', + parentMessageId: '00000000-0000-0000-0000-000000000000', + isCreatedByUser: true, + text: 'build it', + }, + conversation: { conversationId: 'convo-1', endpoint: EModelEndpoint.agents }, + endpointOption: { endpoint: EModelEndpoint.agents, agent_id: 'agent_1' }, + messages: [previewedResponse], + initialResponse: previewedResponse, + ...flags, + } as unknown as TSubmission; + + const { payload } = createPayload(submission); + const serialized = JSON.stringify(payload); + expect(serialized).not.toContain('tool_call'); + expect(serialized).not.toContain('outputTruncated'); + expect(payload).not.toHaveProperty('content'); + expect(payload).not.toHaveProperty('messages'); + }); +}); diff --git a/packages/data-provider/src/data-service.ts b/packages/data-provider/src/data-service.ts index 6c4b14efa8..3fb895a94a 100644 --- a/packages/data-provider/src/data-service.ts +++ b/packages/data-provider/src/data-service.ts @@ -11,6 +11,7 @@ import type { ScheduleMCPConsentView, ConfirmScheduleMCPConsent } from './types/ import type { TFileConfig } from './file-config'; import type * as tl from './types/tools'; import type * as t from './types'; +import { TOOL_CALL_PREVIEWS_PARAM, TOOL_CALL_PREVIEWS_VERSION } from './previews'; import * as permissions from './accessPermissions'; import * as endpoints from './api-endpoints'; import { uploadEventStream } from './upload'; @@ -969,14 +970,20 @@ export const getCustomConfigSpeech = (): Promise /* conversations */ +/** Asks for tool-call previews in a response whose messages seed the conversation cache. */ +function withToolCallPreviews(url: string): string { + const separator = url.includes('?') ? '&' : '?'; + return `${url}${separator}${TOOL_CALL_PREVIEWS_PARAM}=${TOOL_CALL_PREVIEWS_VERSION}`; +} + export function duplicateConversation( payload: t.TDuplicateConvoRequest, ): Promise { - return request.post(endpoints.duplicateConversation(), payload); + return request.post(withToolCallPreviews(endpoints.duplicateConversation()), payload); } export function forkConversation(payload: t.TForkConvoRequest): Promise { - return request.post(endpoints.forkConversation(), payload); + return request.post(withToolCallPreviews(endpoints.forkConversation()), payload); } export function forkSharedConversation( @@ -984,7 +991,7 @@ export function forkSharedConversation( targetMessageIndex?: number, shareRevision?: string, ): Promise { - return request.post(endpoints.forkSharedMessages(shareId), { + return request.post(withToolCallPreviews(endpoints.forkSharedMessages(shareId)), { targetMessageIndex, shareRevision, }); @@ -1138,13 +1145,13 @@ export const editArtifact = async ({ messageId, ...params }: m.TEditArtifactRequest): Promise => { - return request.post(endpoints.messagesArtifacts(messageId), params); + return request.post(withToolCallPreviews(endpoints.messagesArtifacts(messageId)), params); }; export const branchMessage = async ( payload: m.TBranchMessageRequest, ): Promise => { - return request.post(endpoints.messagesBranch(), payload); + return request.post(withToolCallPreviews(endpoints.messagesBranch()), payload); }; export interface OwnerMessageText { @@ -1162,14 +1169,26 @@ export function getOwnerMessageTexts( return request.post(`${endpoints.messages({ conversationId })}/owner-text`, { messageIds }); } -export function getMessagesByConvoId(conversationId: string): Promise { +/** + * Loads a conversation's messages. `toolPreviews` asks for bounded previews of settled tool + * calls; a caller that needs every byte (export, share, trace) leaves it off. + */ +export function getMessagesByConvoId( + conversationId: string, + options?: { toolPreviews?: boolean }, +): Promise { if ( conversationId === config.Constants.NEW_CONVO || conversationId === config.Constants.PENDING_CONVO ) { return Promise.resolve([]); } - return request.get(endpoints.messages({ conversationId })); + const url = endpoints.messages({ conversationId }); + return request.get(options?.toolPreviews === true ? withToolCallPreviews(url) : url); +} + +export function getToolCallPart(params: q.ToolCallPartParams): Promise { + return request.get(endpoints.messageToolCallPart(params)); } export function getMessageById(conversationId: string, messageId: string): Promise { diff --git a/packages/data-provider/src/index.ts b/packages/data-provider/src/index.ts index df59360348..2be4fec215 100644 --- a/packages/data-provider/src/index.ts +++ b/packages/data-provider/src/index.ts @@ -11,6 +11,7 @@ export * from './file-config'; export * from './resolve-llm-delivery-path'; /* messages */ export * from './messages'; +export * from './previews'; export * from './errors'; /* run steps */ export * from './runSteps'; diff --git a/packages/data-provider/src/keys.ts b/packages/data-provider/src/keys.ts index c7aa940a43..d5704fef52 100644 --- a/packages/data-provider/src/keys.ts +++ b/packages/data-provider/src/keys.ts @@ -1,5 +1,7 @@ export enum QueryKeys { messages = 'messages', + /* Full content of one tool-call part, fetched when its preview is not enough */ + toolCallPart = 'toolCallPart', ownerMessageTexts = 'ownerMessageTexts', sharedMessages = 'sharedMessages', sharedStartupConfig = 'sharedStartupConfig', diff --git a/packages/data-provider/src/previews.spec.ts b/packages/data-provider/src/previews.spec.ts new file mode 100644 index 0000000000..111db865df --- /dev/null +++ b/packages/data-provider/src/previews.spec.ts @@ -0,0 +1,47 @@ +import type { FullToolCall } from './previews'; +import { getToolCallPreviewRevision, hasToolCallPreview } from './previews'; + +const preview = (overrides: Partial = {}): FullToolCall => ({ + id: 'call_1', + name: 'bash_tool', + args: '{"command":"make"}', + output: 'stdout:\nfirst\n…\n[exit code: 0]', + outputTruncated: true, + outputLength: 40_000, + previewRevision: '1700000000000', + ...overrides, +}); + +describe('getToolCallPreviewRevision', () => { + it('is stable for the same preview', () => { + expect(getToolCallPreviewRevision(preview())).toBe(getToolCallPreviewRevision(preview())); + }); + + it('changes when the head changes even with the same length and tail', () => { + const before = getToolCallPreviewRevision(preview()); + const after = getToolCallPreviewRevision( + preview({ output: 'stdout:\nfirsT\n…\n[exit code: 0]' }), + ); + expect(after).not.toBe(before); + }); + + it('changes when the stored message changes, even if the preview looks identical', () => { + expect(getToolCallPreviewRevision(preview({ previewRevision: '1700000000001' }))).not.toBe( + getToolCallPreviewRevision(preview()), + ); + }); + + it('changes with the arguments and with object arguments', () => { + expect(getToolCallPreviewRevision(preview({ args: { command: 'make test' } }))).not.toBe( + getToolCallPreviewRevision(preview({ args: { command: 'make' } })), + ); + }); +}); + +describe('hasToolCallPreview', () => { + it('reads only the shortening markers', () => { + expect(hasToolCallPreview(preview())).toBe(true); + expect(hasToolCallPreview({ previewRevision: '1' })).toBe(false); + expect(hasToolCallPreview(undefined)).toBe(false); + }); +}); diff --git a/packages/data-provider/src/previews.ts b/packages/data-provider/src/previews.ts new file mode 100644 index 0000000000..42d77f2969 --- /dev/null +++ b/packages/data-provider/src/previews.ts @@ -0,0 +1,75 @@ +import type { TMessageContentParts } from './types/content'; +import type { Agents } from './types/agents'; + +/** + * Query parameter a client adds to a conversation's message load to receive bounded previews of + * settled tool calls instead of their full output, arguments and subagent transcripts. A server + * that predates previews ignores it, and a client that omits it receives the full payload, so + * either side can upgrade first. + */ +export const TOOL_CALL_PREVIEWS_PARAM = 'toolPreviews'; + +/** The preview format this client understands; a later format bumps it. */ +export const TOOL_CALL_PREVIEWS_VERSION = '1'; + +/** Markers the server sets on a tool call whose content it shortened or left out. */ +export type ToolCallPreviewMarkers = Pick< + Agents.ToolCall, + | 'outputTruncated' + | 'outputLength' + | 'argsTruncated' + | 'argsLength' + | 'subagentContentOmitted' + | 'subagentContentParts' + | 'previewRevision' +>; + +/** A tool call as the full-part endpoint returns it, transcript included. */ +export type FullToolCall = Agents.ToolCall & { subagent_content?: TMessageContentParts[] }; + +/** 32-bit FNV-1a over UTF-16 code units; a fingerprint for cache keys, not a security hash. */ +function fingerprint(text: string): string { + let hash = 0x811c9dc5; + for (let i = 0; i < text.length; i++) { + hash ^= text.charCodeAt(i); + hash = Math.imul(hash, 0x01000193); + } + return (hash >>> 0).toString(36); +} + +/** + * Identifies the stored version a preview stands for: the server's `previewRevision` (when the + * message last changed) plus a fingerprint of everything the preview itself carries. Either + * changes when the stored call does, so a client cache of the full part keyed on it never + * outlives the content it was fetched for. + */ +export function getToolCallPreviewRevision(toolCall: FullToolCall): string { + const output = typeof toolCall.output === 'string' ? toolCall.output : ''; + const args = + typeof toolCall.args === 'string' ? toolCall.args : JSON.stringify(toolCall.args ?? null); + const content = [ + output, + args, + toolCall.outputLength ?? '', + toolCall.argsLength ?? '', + toolCall.subagentContentParts ?? '', + ].join('\u0000'); + return `${toolCall.previewRevision ?? ''}:${fingerprint(content)}`; +} + +/** True when any of the tool call's content is a preview rather than the stored value. */ +export function hasToolCallPreview(toolCall: ToolCallPreviewMarkers | null | undefined): boolean { + return ( + toolCall?.outputTruncated === true || + toolCall?.argsTruncated === true || + toolCall?.subagentContentOmitted === true + ); +} + +/** + * The exit-status trailer the attached-workspace `bash_tool` appends to its output + * (`formatCommandResult` in `packages/api/src/code/command.ts`), anchored to the end. Shared so + * the card that reads the verdict and the server that bounds previews agree on its extent. + */ +export const COMMAND_RESULT_TRAILER: RegExp = + /\n((?:\[(?:exit code: -?\d+|terminated by [\w+-]+|timed out|output truncated)\])+)((?:\nCommand reached timeoutMs: [^\n]*)?(?:\n\[directory hint: [^\n]*\])?)$/; diff --git a/packages/data-provider/src/types/agents.ts b/packages/data-provider/src/types/agents.ts index 6b6ecf69c6..f6ab352536 100644 --- a/packages/data-provider/src/types/agents.ts +++ b/packages/data-provider/src/types/agents.ts @@ -121,6 +121,21 @@ export namespace Agents { generationId?: string; }; }; + /** The server sent a bounded preview of `output`; the full value loads on demand. */ + outputTruncated?: true; + /** Length of the full `output` in UTF-16 code units, present with `outputTruncated`. */ + outputLength?: number; + /** The server sent a bounded preview of `args`; the full value loads on demand. */ + argsTruncated?: true; + /** Length of the full serialized `args`, present with `argsTruncated`. */ + argsLength?: number; + /** The server left out `subagent_content`; the full value loads on demand. */ + subagentContentOmitted?: true; + /** Number of top-level parts in the omitted `subagent_content`. */ + subagentContentParts?: number; + /** When the stored message last changed, set on previews so a client cache of the full part + * is keyed to the stored version it came from. */ + previewRevision?: string; /** The tool call was rejected before execution because its input failed schema validation. */ inputValidationError?: true; /** Server-stamped provenance; see `PartMetadata.executor`. */ diff --git a/packages/data-provider/src/types/queries.ts b/packages/data-provider/src/types/queries.ts index 4552b5fff5..c46750374a 100644 --- a/packages/data-provider/src/types/queries.ts +++ b/packages/data-provider/src/types/queries.ts @@ -1,5 +1,6 @@ import type { InfiniteData } from '@tanstack/react-query'; import type { RerankerTypes, SearchProviders, ScraperProviders } from '../config'; +import type { FullToolCall } from '../previews'; import type * as p from '../accessPermissions'; import type * as a from '../types/agents'; import type * as s from '../schemas'; @@ -110,6 +111,26 @@ export type MessagesListResponse = { nextCursor: string | null; }; +/** Locates one tool-call part. `toolCallId` disambiguates when the part moved. */ +export type ToolCallPartParams = { + conversationId: string; + messageId: string; + partIndex: number; + toolCallId?: string; + /** Host run-step id; provider tool-call ids can repeat within one response. */ + stepId?: string; + /** Agent that produced the part, when parallel agents share a response. */ + agentId?: string; +}; + +export type ToolCallPartResponse = { + conversationId: string; + messageId: string; + /** Index of the part in the stored message, which may differ from the requested one. */ + partIndex: number; + tool_call: FullToolCall; +}; + /* Shared Links */ export type SharedMessagesResponse = Omit & { messages: s.TMessage[]; diff --git a/packages/data-schemas/src/app/service.ts b/packages/data-schemas/src/app/service.ts index 3f1f01039b..924b674d2d 100644 --- a/packages/data-schemas/src/app/service.ts +++ b/packages/data-schemas/src/app/service.ts @@ -4,6 +4,7 @@ import { chatProjectsConfigSchema, filtersConfigSchema, conversationListConfigSchema, + toolCallPreviewsConfigSchema, hasActiveFiltersConfig, getConfigDefaults, langfuseConfigSchema, @@ -106,6 +107,18 @@ export function loadConversationListConfig( return conversationListConfigSchema.parse({}); } +/** Resolves the tool-call preview bounds; an invalid block keeps the defaults. */ +export function loadToolCallPreviewsConfig( + config: DeepPartial, +): NonNullable { + const parsed = toolCallPreviewsConfigSchema.safeParse(config.toolCallPreviews ?? {}); + if (parsed.success) { + return parsed.data; + } + logger.warn('[AppService] Invalid toolCallPreviews config', parsed.error.flatten()); + return toolCallPreviewsConfigSchema.parse({}); +} + export function loadFiltersConfig(config: DeepPartial): AppConfig['filters'] { const raw = config.filters; if (raw === undefined) { @@ -221,6 +234,7 @@ export const AppService = async (params?: { cloudfront: config.cloudfront as AppConfig['cloudfront'], secureImageLinks: config.secureImageLinks !== false, conversationList: loadConversationListConfig(config), + toolCallPreviews: loadToolCallPreviewsConfig(config), }; const agentsDefaults = agentsConfigSetup(config); diff --git a/packages/data-schemas/src/types/app.ts b/packages/data-schemas/src/types/app.ts index 32196ab952..eb4782f98a 100644 --- a/packages/data-schemas/src/types/app.ts +++ b/packages/data-schemas/src/types/app.ts @@ -15,6 +15,7 @@ import type { TAnthropicEndpoint, SummarizationConfig, TConversationListConfig, + TToolCallPreviewsConfig, SkillSyncConfig, FiltersConfig, } from 'librechat-data-provider'; @@ -118,6 +119,8 @@ export interface AppConfig { secureImageLinks?: TCustomConfig['secureImageLinks']; /** Validation limits for the conversation list's filter facets */ conversationList?: TConversationListConfig; + /** Bounds for the tool-call previews sent on conversation loads */ + toolCallPreviews?: TToolCallPreviewsConfig; /** Processed model specifications */ modelSpecs?: TCustomConfig['modelSpecs']; /** Available tools */