From b2bcbf9380987fec753701442151e6996c2b94a0 Mon Sep 17 00:00:00 2001 From: Danny Avila Date: Sat, 10 Oct 2026 08:43:39 -0400 Subject: [PATCH] =?UTF-8?q?=F0=9F=A7=AF=20feat:=20Recover=20Conversations?= =?UTF-8?q?=20Whose=20Attachments=20Keep=20Failing=20(#16965)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * 🧯 feat: Recover Conversations Whose Attachments Keep Failing Once a branch carries a problem attachment, every turn resends it and fails the same way. A failed response whose branch carries files now offers "Retry without files" (re-asks the failed turn with no files as a new version) and, when earlier turns carry files, "Branch without files" (a copy up to the last good answer with every upload and code output left out, with the failed message restored to the composer). File-shaped errors say so in the card. The edit view shows a user message's files as removable chips; "Update & rerun" sends only the kept files and Save is disabled after a removal. Fork is offered on a failed turn and gains a "Leave out files" option, backed by a new `excludeFiles` fork option that drops `files`, `attachments` and the conversation's `files` from the copy. Bedrock rejects a request whose documents share a name, and history resends every earlier document, so repeated names are now numbered before formatting. * 🧯 fix: Cover Steered Files, Hand Off the Whole Failed Turn, Name Replayed Documents Once - Excluding files from a fork also strips attachments steered into a response (`content[].files` on steer parts), which replay like uploads, and the decision lives in `forkFileScope` in packages/api. - The recovery card counts steered attachments as files, and reads the specific code inside a generic `error` envelope, so a rate limit is not offered files as the fix. - "Branch without files" hands the failed turn to the copy's composer the way a message is put back for editing: text through the in-memory handoff, quotes, manual skills and the reasoning override through their compose-time atoms. - Document names are deduplicated over the formatted payload after the steer stamp, so replayed steer documents are numbered too. - Fork stays withheld on a failed response, whose copy would carry the error as an ordinary answer; the card's branch action copies up to the turn before. - Remove-file buttons are natively disabled while the editor is locked. * 🧯 fix: Keep Search Sources, Scope Root Retries, Skip Legacy Rate Limits - A fork without files keeps metadata-only attachments (search sources, file search results) and drops only file-backed ones; the recovery card likewise counts only file-backed attachments as files. - A turn sent at the root takes none of the conversation's file refs: its siblings' uploads (an edited first message, or one retried without its files) no longer come back through the unanchored-continuation fallback. - Rows that carry a legacy LangChain model, key or rate-limit code are not offered files as the fix. * 🧯 fix: Align File Exclusion With Replay's File Shapes, Skip Assistants, Verify the Copy - Shared `messageCarriesFiles` (with `isFileContentPart`, `hasContentPartFiles`, `isFileAttachment`) in data-provider names every shape historical replay reads a file by: uploads, file-backed attachments, attached `files` on any content part, and `image_file`, `file` or direct `file_id` locators. The fork drops all of them and the recovery card detects all of them. - Assistants conversations get no recovery actions: their upstream thread keeps every attachment, and those endpoints offer no fork either. - "Branch without files" checks the returned copy; a server that ignored `excludeFiles` gets an error toast instead of a false recovery. * 🧯 fix: Type Replay's Provider-Native File Locators for the Part Predicates --- api/server/controllers/agents/client.js | 3 + api/server/controllers/agents/client.test.js | 43 +++ api/server/routes/__tests__/convos.spec.js | 24 ++ api/server/routes/convos.js | 4 +- api/server/utils/import/fork.database.spec.js | 206 +++++++++++ api/server/utils/import/fork.js | 13 +- .../Chat/Input/Files/FileContainer.tsx | 7 +- .../src/components/Chat/Input/Files/Image.tsx | 6 +- .../Chat/Input/Files/RemoveFile.tsx | 17 +- .../Chat/Messages/Content/Container.tsx | 13 +- .../Chat/Messages/Content/EditFiles.tsx | 94 +++++ .../Chat/Messages/Content/EditMessage.tsx | 52 ++- .../Chat/Messages/Content/MessageContent.tsx | 2 + .../Content/__tests__/EditMessage.spec.tsx | 99 ++++++ client/src/components/Chat/Messages/Fork.tsx | 10 + .../components/Chat/Messages/HoverButtons.tsx | 3 +- .../Chat/Messages/__tests__/Fork.spec.tsx | 33 ++ .../Messages/__tests__/HoverButtons.spec.tsx | 24 ++ .../Messages/Content/Error/attachments.ts | 149 ++++++++ .../Messages/Content/Error/parts.tsx | 14 +- .../Messages/Content/Error/recovery.tsx | 249 +++++++++++++ .../Content/__tests__/attachments.spec.ts | 233 +++++++++++++ .../Content/__tests__/recovery.spec.tsx | 330 ++++++++++++++++++ client/src/locales/en/translation.json | 13 + .../src/agents/__tests__/initialize.test.ts | 34 ++ packages/api/src/agents/initialize.ts | 9 +- packages/api/src/conversations/files.spec.ts | 134 +++++++ packages/api/src/conversations/files.ts | 89 +++++ packages/api/src/conversations/index.ts | 1 + .../api/src/files/encode/document.spec.ts | 97 ++++- packages/api/src/files/encode/document.ts | 86 ++++- packages/data-provider/src/messages.spec.ts | 67 +++- packages/data-provider/src/messages.ts | 56 +++ packages/data-provider/src/types.ts | 2 + 34 files changed, 2191 insertions(+), 25 deletions(-) create mode 100644 api/server/utils/import/fork.database.spec.js create mode 100644 client/src/components/Chat/Messages/Content/EditFiles.tsx create mode 100644 client/src/components/Messages/Content/Error/attachments.ts create mode 100644 client/src/components/Messages/Content/Error/recovery.tsx create mode 100644 client/src/components/Messages/Content/__tests__/attachments.spec.ts create mode 100644 client/src/components/Messages/Content/__tests__/recovery.spec.tsx create mode 100644 packages/api/src/conversations/files.spec.ts create mode 100644 packages/api/src/conversations/files.ts diff --git a/api/server/controllers/agents/client.js b/api/server/controllers/agents/client.js index 746cb5b7e4..aab16bed67 100644 --- a/api/server/controllers/agents/client.js +++ b/api/server/controllers/agents/client.js @@ -145,6 +145,7 @@ const { collectFreshSkillPrimeNames, isSkillPrimeMessage, collectFileIds, + dedupeDocumentNames, processTextWithTokenLimit, logAgentMemorySnapshot, createAgentMemoryCallback, @@ -2901,6 +2902,8 @@ class AgentClient extends BaseClient { } } } + /** After the steer stamp, which adds replayed steers' documents of its own. */ + dedupeDocumentNames(payload); if (hasFileContext) { for (let i = 0; i < orderedMessages.length; i++) { memoryPayload.push( diff --git a/api/server/controllers/agents/client.test.js b/api/server/controllers/agents/client.test.js index 3d30076aef..e253ba1b7f 100644 --- a/api/server/controllers/agents/client.test.js +++ b/api/server/controllers/agents/client.test.js @@ -6295,6 +6295,49 @@ describe('AgentClient - titleConvo', () => { expect(client.publishRunContextMeta).toHaveBeenCalledTimes(expectSeed ? 1 : 0); }); + it('gives every resent document a unique name before formatting', async () => { + const bedrockDocument = (name) => ({ + type: 'document', + document: { name, format: 'pdf', source: { bytes: Buffer.from(name) } }, + }); + const firstQuestion = { + messageId: 'user-first', + parentMessageId: null, + sender: 'User', + isCreatedByUser: true, + text: 'Summarize the fees.', + documents: [bedrockDocument('Stripe fees')], + }; + const answer = { + messageId: 'assistant-first', + parentMessageId: 'user-first', + sender: 'Assistant', + isCreatedByUser: false, + text: 'Here is the summary.', + }; + const secondQuestion = { + messageId: 'user-second', + parentMessageId: 'assistant-first', + sender: 'User', + isCreatedByUser: true, + text: 'Compare with this one.', + documents: [bedrockDocument('Stripe fees')], + }; + const messages = [firstQuestion, answer, secondQuestion]; + client.setModelBoundStoredMessages(messages); + + const result = await client.buildMessages(messages, 'user-second', {}, {}); + + const names = result.prompt.flatMap((message) => + Array.isArray(message.content) + ? message.content + .filter((part) => part?.type === 'document') + .map((part) => part.document.name) + : [], + ); + expect(names).toEqual(['Stripe fees', 'Stripe fees (2)']); + }); + it('preserves persisted source identity through both agent formatting passes', async () => { mockReq.config.filters = { messages: { diff --git a/api/server/routes/__tests__/convos.spec.js b/api/server/routes/__tests__/convos.spec.js index 0c448a67ab..5798439d9b 100644 --- a/api/server/routes/__tests__/convos.spec.js +++ b/api/server/routes/__tests__/convos.spec.js @@ -367,6 +367,30 @@ describe('Convos Routes', () => { ); }); + it('forwards excludeFiles only when it is exactly true', async () => { + forkConversation.mockResolvedValue({ conversation: { conversationId: 'forked-convo' } }); + + await request(app).post('/api/convos/fork').send({ + conversationId: 'source-convo', + messageId: 'source-message', + excludeFiles: true, + }); + await request(app).post('/api/convos/fork').send({ + conversationId: 'source-convo', + messageId: 'source-message', + excludeFiles: 'true', + }); + + expect(forkConversation).toHaveBeenNthCalledWith( + 1, + expect.objectContaining({ excludeFiles: true }), + ); + expect(forkConversation).toHaveBeenNthCalledWith( + 2, + expect.objectContaining({ excludeFiles: false }), + ); + }); + it('returns a raw-free 400 when cloned content is blocked', async () => { const error = Object.assign(new Error('PRIVATE-SENTINEL'), { code: 'content_filter_block', diff --git a/api/server/routes/convos.js b/api/server/routes/convos.js index ef6b220150..95ab08f8a1 100644 --- a/api/server/routes/convos.js +++ b/api/server/routes/convos.js @@ -808,7 +808,8 @@ router.post( router.post('/fork', forkIpLimiter, forkUserLimiter, configMiddleware, async (req, res) => { try { /** @type {TForkConvoRequest} */ - const { conversationId, messageId, option, splitAtTarget, latestMessageId } = req.body; + const { conversationId, messageId, option, splitAtTarget, latestMessageId, excludeFiles } = + req.body; const result = await forkConversation({ requestUserId: req.user.id, originalConvoId: conversationId, @@ -817,6 +818,7 @@ router.post('/fork', forkIpLimiter, forkUserLimiter, configMiddleware, async (re records: true, splitAtTarget, option, + excludeFiles: excludeFiles === true, interfaceConfig: req.config?.interfaceConfig, filters: req.config?.filters, ...(req.config?.messageFilter?.pii == null diff --git a/api/server/utils/import/fork.database.spec.js b/api/server/utils/import/fork.database.spec.js new file mode 100644 index 0000000000..f819a76a4f --- /dev/null +++ b/api/server/utils/import/fork.database.spec.js @@ -0,0 +1,206 @@ +const mongoose = require('mongoose'); +const { MongoMemoryServer } = require('mongodb-memory-server'); +const { Constants, EModelEndpoint, ForkOptions } = require('librechat-data-provider'); +const { createModels, runAsSystem, tenantStorage } = require('@librechat/data-schemas'); + +jest.mock('~/server/services/Config', () => ({ + getAppConfig: jest.fn().mockResolvedValue({ interfaceConfig: {} }), + getEndpointsConfig: jest.fn().mockResolvedValue({ openAI: { userProvide: false } }), +})); + +jest.mock('~/server/controllers/ModelController', () => ({ + getModelsConfig: jest.fn().mockResolvedValue({ openAI: ['gpt-4o'] }), +})); + +createModels(mongoose); +const { forkConversation } = require('./fork'); + +const owner = 'fork-owner'; +const tenantId = 'fork-tenant'; +const conversationId = 'source-conversation'; + +const screenshot = { + file_id: 'screenshot-1', + filename: 'screenshot.png', + filepath: '/images/screenshot.png', + type: 'image/png', +}; +const chart = { + file_id: 'chart-1', + filename: 'chart.png', + filepath: '/images/chart.png', + type: 'image/png', + toolCallId: 'call-1', +}; + +/** One branch: a question with a screenshot, an answer with a code output, a follow-up, and a + * failed answer; plus a sibling branch off the first question that the direct path skips. */ +const sourceMessages = [ + { + messageId: 'user-1', + parentMessageId: Constants.NO_PARENT, + text: 'What does this screenshot show?', + isCreatedByUser: true, + files: [screenshot], + tokenCount: 1500, + }, + { + messageId: 'assistant-1', + parentMessageId: 'user-1', + text: 'A chart of weekly sales.', + isCreatedByUser: false, + content: [ + { type: 'text', text: 'A chart of weekly sales.' }, + { + type: 'steer', + steer: 'Use this screenshot instead', + steerId: 'steer-1', + files: [screenshot], + }, + ], + attachments: [chart], + tokenCount: 12, + }, + { + messageId: 'user-2', + parentMessageId: 'assistant-1', + text: 'Plot it again in blue.', + isCreatedByUser: true, + tokenCount: 8, + }, + { + messageId: 'assistant-2', + parentMessageId: 'user-2', + text: 'Error uploading code environment file: 429', + isCreatedByUser: false, + error: true, + }, + { + messageId: 'assistant-sibling', + parentMessageId: 'user-1', + text: 'An alternate answer.', + isCreatedByUser: false, + }, +]; + +async function seedSourceConversation() { + await runAsSystem(async () => { + await mongoose.models.Conversation.create({ + conversationId, + user: owner, + tenantId, + title: 'Screenshots', + endpoint: EModelEndpoint.openAI, + model: 'gpt-4o', + files: [screenshot.file_id, chart.file_id], + }); + await mongoose.models.Message.insertMany( + sourceMessages.map((message, index) => ({ + ...message, + conversationId, + user: owner, + tenantId, + sender: message.isCreatedByUser ? 'User' : 'GPT-4o', + endpoint: EModelEndpoint.openAI, + model: 'gpt-4o', + createdAt: new Date(Date.UTC(2026, 0, 1, 0, index)), + })), + ); + }); +} + +function fork(options) { + return tenantStorage.run({ tenantId, userId: owner }, () => + forkConversation({ + requestUserId: owner, + originalConvoId: conversationId, + targetMessageId: 'assistant-2', + option: ForkOptions.DIRECT_PATH, + records: true, + ...options, + }), + ); +} + +function readPersisted(forkedConversationId) { + return runAsSystem(async () => ({ + conversation: await mongoose.models.Conversation.findOne({ + conversationId: forkedConversationId, + }).lean(), + messages: await mongoose.models.Message.find({ conversationId: forkedConversationId }) + .sort({ createdAt: 1 }) + .lean(), + })); +} + +describe('forkConversation with excludeFiles (database)', () => { + let mongoServer; + + beforeAll(async () => { + mongoServer = await MongoMemoryServer.create(); + await mongoose.connect(mongoServer.getUri()); + }); + + beforeEach(async () => { + await mongoose.connection.dropDatabase(); + await seedSourceConversation(); + }); + + afterAll(async () => { + await mongoose.disconnect(); + await mongoServer?.stop(); + }); + + it('copies the visible branch with no uploads, code outputs or conversation file list', async () => { + const result = await fork({ excludeFiles: true }); + const persisted = await readPersisted(result.conversation.conversationId); + + expect(persisted.conversation.conversationId).not.toBe(conversationId); + expect(persisted.conversation.files ?? []).toEqual([]); + expect(persisted.messages.map((message) => message.text)).toEqual([ + 'What does this screenshot show?', + 'A chart of weekly sales.', + 'Plot it again in blue.', + 'Error uploading code environment file: 429', + ]); + for (const message of persisted.messages) { + expect(message.files ?? []).toEqual([]); + expect(message.attachments ?? []).toEqual([]); + } + expect(persisted.messages[1].content).toEqual([ + { type: 'text', text: 'A chart of weekly sales.' }, + { type: 'steer', steer: 'Use this screenshot instead', steerId: 'steer-1' }, + ]); + + const [question, answer, followUp, failure] = persisted.messages; + expect(question.parentMessageId).toBe(Constants.NO_PARENT); + expect(answer.parentMessageId).toBe(question.messageId); + expect(followUp.parentMessageId).toBe(answer.messageId); + expect(failure.parentMessageId).toBe(followUp.messageId); + /** A count that covered dropped uploads or steered files goes with them; text-only turns + * keep theirs. */ + expect(question.tokenCount).toBeUndefined(); + expect(answer.tokenCount).toBeUndefined(); + expect(followUp.tokenCount).toBe(8); + }); + + it('keeps the original conversation and its files intact', async () => { + await fork({ excludeFiles: true }); + const original = await readPersisted(conversationId); + + expect(original.conversation.files).toEqual([screenshot.file_id, chart.file_id]); + expect(original.messages).toHaveLength(sourceMessages.length); + const byId = new Map(original.messages.map((message) => [message.messageId, message])); + expect(byId.get('user-1').files).toEqual([screenshot]); + expect(byId.get('assistant-1').attachments).toEqual([chart]); + }); + + it('still copies files when the option is off', async () => { + const result = await fork({}); + const persisted = await readPersisted(result.conversation.conversationId); + + expect(persisted.conversation.files).toEqual([screenshot.file_id, chart.file_id]); + expect(persisted.messages[0].files).toEqual([screenshot]); + expect(persisted.messages[1].attachments).toEqual([chart]); + }); +}); diff --git a/api/server/utils/import/fork.js b/api/server/utils/import/fork.js index dc141a6319..f39ed02ccb 100644 --- a/api/server/utils/import/fork.js +++ b/api/server/utils/import/fork.js @@ -5,6 +5,7 @@ const { createNativeCopyPreflight, withoutTraceRefs, isTemporaryRecord, + forkFileScope, getAllMessagesUpToParent, transferNativeCopyProvenance, } = require('@librechat/api'); @@ -22,17 +23,19 @@ const BaseClient = require('~/app/clients/BaseClient'); * @param {ImportBatchBuilder} importBatchBuilder - Instance of ImportBatchBuilder * @param {object} [options] - Clone behavior for the source conversation. * @param {boolean} [options.detachSubagentRuntime=false] - Remove durable child execution metadata. + * @param {boolean} [options.excludeFiles=false] - Drop uploads and code outputs from every clone. * @returns {Map} Map of original messageIds to new messageIds */ function cloneMessagesWithTimestamps( messagesToClone, importBatchBuilder, - { detachSubagentRuntime = false, nativeCopy = false } = {}, + { detachSubagentRuntime = false, nativeCopy = false, excludeFiles = false } = {}, ) { const { entries, idMapping } = cloneLineage(messagesToClone, uuidv4); + const fileScope = forkFileScope(excludeFiles); for (const { source, messageId, parentMessageId, createdAt } of entries) { const clonedMessage = { - ...withoutTraceRefs(source), + ...fileScope.message(withoutTraceRefs(source)), messageId, parentMessageId, createdAt, @@ -62,6 +65,8 @@ function cloneMessagesWithTimestamps( * @param {boolean} [params.records=false] - Optional flag for returning actual database records or resulting conversation and messages. * @param {boolean} [params.splitAtTarget=false] - Optional flag for splitting the messages at the target message level. * @param {string} [params.latestMessageId] - latestMessageId - Required if splitAtTarget is true. + * @param {boolean} [params.excludeFiles=false] - Copy without uploads or code outputs, so a branch whose + * attachments keep failing can continue without any of them. * @param {object} [params.interfaceConfig] - Runtime interface config used to apply retention to cloned records. * @param {object} [params.filters] - Source-aware content filters applied before cloned records are persisted. * @param {object} [params.legacyPii] - Legacy messageFilter.pii applied before cloned records are persisted. @@ -77,6 +82,7 @@ async function forkConversation({ records = false, splitAtTarget = false, latestMessageId, + excludeFiles = false, filters, legacyPii, builderFactory = createImportBatchBuilder, @@ -129,12 +135,13 @@ async function forkConversation({ * task protocol and private serialized model transcript. */ detachSubagentRuntime: originalConvo.subagentThread != null, nativeCopy: true, + excludeFiles, }); const result = importBatchBuilder.finishConversation( newTitle || originalConvo.title, new Date(), - originalConvo, + forkFileScope(excludeFiles).conversation(originalConvo), ); await importBatchBuilder.saveBatch(); logger.debug( diff --git a/client/src/components/Chat/Input/Files/FileContainer.tsx b/client/src/components/Chat/Input/Files/FileContainer.tsx index e63a6b50c6..f146277e05 100644 --- a/client/src/components/Chat/Input/Files/FileContainer.tsx +++ b/client/src/components/Chat/Input/Files/FileContainer.tsx @@ -21,6 +21,8 @@ const FileContainer = ({ buttonClassName, containerClassName, onDelete, + removeLabel, + removeDisabled, onClick, }: { file: Partial; @@ -56,6 +58,9 @@ const FileContainer = ({ buttonClassName?: string; containerClassName?: string; onDelete?: () => void; + /** Accessible name for the remove control; defaults to a generic "Remove file". */ + removeLabel?: string; + removeDisabled?: boolean; onClick?: React.MouseEventHandler; }) => { const fileType = getFileType(overrideType ?? file.type); @@ -144,7 +149,7 @@ const FileContainer = ({ {details} )} - {onDelete && } + {onDelete && } ); }; diff --git a/client/src/components/Chat/Input/Files/Image.tsx b/client/src/components/Chat/Input/Files/Image.tsx index f13e6e744d..443a9df219 100644 --- a/client/src/components/Chat/Input/Files/Image.tsx +++ b/client/src/components/Chat/Input/Files/Image.tsx @@ -8,12 +8,16 @@ const Image = ({ onDelete, progress = 1, source = FileSources.local, + removeLabel, + removeDisabled, }: { imageBase64?: string; url?: string; onDelete: () => void; progress: number; // between 0 and 1 source?: FileSources; + removeLabel?: string; + removeDisabled?: boolean; }) => { return (
@@ -23,7 +27,7 @@ const Image = ({
- +
); }; diff --git a/client/src/components/Chat/Input/Files/RemoveFile.tsx b/client/src/components/Chat/Input/Files/RemoveFile.tsx index bb83b6a135..a61c58a4aa 100644 --- a/client/src/components/Chat/Input/Files/RemoveFile.tsx +++ b/client/src/components/Chat/Input/Files/RemoveFile.tsx @@ -1,13 +1,24 @@ import { useLocalize } from '~/hooks'; -export default function RemoveFile({ onRemove }: { onRemove: () => void }) { +export default function RemoveFile({ + onRemove, + label, + disabled = false, +}: { + onRemove: () => void; + disabled?: boolean; + /** Names the file, where several chips sit side by side and "Remove file" would not say which. */ + label?: string; +}) { const localize = useLocalize(); return ( )} diff --git a/client/src/components/Chat/Messages/Content/MessageContent.tsx b/client/src/components/Chat/Messages/Content/MessageContent.tsx index 3ad7414eb7..c6deffbadb 100644 --- a/client/src/components/Chat/Messages/Content/MessageContent.tsx +++ b/client/src/components/Chat/Messages/Content/MessageContent.tsx @@ -4,6 +4,7 @@ import { Alert, DelayedRender } from '@librechat/client'; import { Constants, parseThinkingContent } from 'librechat-data-provider'; import type { TMessage } from 'librechat-data-provider'; import type { TMessageContentProps, TDisplayProps } from '~/common'; +import AttachmentRecoveryActions from '~/components/Messages/Content/Error/recovery'; import useSmoothStreaming from '~/hooks/Messages/useSmoothStreaming'; import Error from '~/components/Messages/Content/Error'; import ToolCallLimitNotice from './ToolCallLimitNotice'; @@ -85,6 +86,7 @@ export const ErrorMessage = ({ + ); diff --git a/client/src/components/Chat/Messages/Content/__tests__/EditMessage.spec.tsx b/client/src/components/Chat/Messages/Content/__tests__/EditMessage.spec.tsx index 7a89390656..5bb6761a25 100644 --- a/client/src/components/Chat/Messages/Content/__tests__/EditMessage.spec.tsx +++ b/client/src/components/Chat/Messages/Content/__tests__/EditMessage.spec.tsx @@ -432,3 +432,102 @@ describe('EditMessage', () => { expect(enterEdit).toHaveBeenCalledWith(true); }); }); + +describe('EditMessage file removal', () => { + const report = { file_id: 'report', filename: 'report.pdf', type: 'application/pdf' }; + const notes = { file_id: 'notes', filename: 'notes.txt', type: 'text/plain' }; + const withFiles = { ...message, files: [report, notes] } as TMessage; + + beforeEach(() => { + jest.clearAllMocks(); + mockGetMessages.mockReturnValue([withFiles]); + }); + + it('reruns with only the files that were kept, as a new version of the turn', async () => { + const user = userEvent.setup(); + const ask = jest.fn(); + const { enterEdit, setSiblingIdx } = renderEditor({ ask, editedMessage: withFiles }); + + const removeButtons = screen.getAllByRole('button', { name: 'com_ui_remove_file_named' }); + expect(removeButtons).toHaveLength(2); + await user.click(removeButtons[0]); + + expect(screen.getAllByRole('button', { name: 'com_ui_remove_file_named' })).toHaveLength(1); + expect(screen.getByText('com_ui_edit_files_removed')).toBeInTheDocument(); + /** Focus stays in the chip row instead of falling to the page. */ + expect(screen.getByRole('button', { name: 'com_ui_remove_file_named' })).toHaveFocus(); + + await user.click(screen.getByRole('button', { name: 'com_ui_update_rerun' })); + + expect(ask).toHaveBeenCalledWith( + { text: 'Original message', parentMessageId: 'root', conversationId: 'conversation-1' }, + expect.objectContaining({ overrideFiles: [notes] }), + ); + expect(withFiles.files).toEqual([report, notes]); + expect(setSiblingIdx).toHaveBeenCalledWith(-1); + expect(enterEdit).toHaveBeenCalledWith(true); + }); + + it('does not let Save pretend to remove files', async () => { + const user = userEvent.setup(); + renderEditor({ editedMessage: withFiles }); + + await user.type(screen.getByTestId('message-text-editor'), ' and more'); + expect(screen.getByRole('button', { name: 'com_ui_save' })).toBeEnabled(); + + await user.click(screen.getAllByRole('button', { name: 'com_ui_remove_file_named' })[1]); + expect(screen.getByRole('button', { name: 'com_ui_save' })).toBeDisabled(); + }); + + it('sends no files once every file is removed, and moves focus to the text', async () => { + const user = userEvent.setup(); + const ask = jest.fn(); + renderEditor({ ask, editedMessage: withFiles }); + + await user.click(screen.getAllByRole('button', { name: 'com_ui_remove_file_named' })[0]); + await user.click(screen.getByRole('button', { name: 'com_ui_remove_file_named' })); + + expect(screen.queryByRole('button', { name: 'com_ui_remove_file_named' })).toBeNull(); + expect(screen.getByTestId('message-text-editor')).toHaveFocus(); + await user.click(screen.getByRole('button', { name: 'com_ui_update_rerun' })); + + expect(ask).toHaveBeenCalledWith( + expect.objectContaining({ text: 'Original message' }), + expect.objectContaining({ overrideFiles: [] }), + ); + }); + + it('cannot rerun a files-only message with every file removed', async () => { + const user = userEvent.setup(); + const ask = jest.fn(); + renderEditor({ ask, editedMessage: { ...withFiles, text: '', files: [report] } as TMessage }); + + await user.click(screen.getByRole('button', { name: 'com_ui_remove_file_named' })); + + expect(screen.getByRole('button', { name: 'com_ui_update_rerun' })).toBeDisabled(); + }); + + it('disables file removal while the editor is locked', () => { + render( + , + ); + + for (const button of screen.getAllByRole('button', { name: 'com_ui_remove_file_named' })) { + expect(button).toBeDisabled(); + } + }); + + it('shows no file chips on an answer', () => { + renderEditor({ editedMessage: { ...assistantMessage, files: [report] } as TMessage }); + + expect(screen.queryByRole('button', { name: 'com_ui_remove_file_named' })).toBeNull(); + }); +}); diff --git a/client/src/components/Chat/Messages/Fork.tsx b/client/src/components/Chat/Messages/Fork.tsx index eee419843c..6ae32b5f04 100644 --- a/client/src/components/Chat/Messages/Fork.tsx +++ b/client/src/components/Chat/Messages/Fork.tsx @@ -217,6 +217,8 @@ function Fork({ const localize = useLocalize(); const { showToast } = useToastContext(); const [remember, setRemember] = useState(false); + /** Per fork, never remembered: a copy without files is a recovery step, not a preference. */ + const [excludeFiles, setExcludeFiles] = useState(false); const { navigateToConvo } = useNavigateToConvo(); const timeoutRef = useRef(null); const [forkSetting, setForkSetting] = useRecoilState(store.forkSetting); @@ -280,6 +282,7 @@ function Fork({ option, splitAtTarget, latestMessageId: getLatestMessageId?.(), + ...(excludeFiles ? { excludeFiles } : {}), }); }; @@ -422,6 +425,13 @@ function Fork({ labelKey="com_ui_fork_split_target" infoKey="com_ui_fork_info_start" /> + )} - {/* Fork Button */} + {/* Fork Button: withheld on a failed turn, whose copy would carry the provider's error as an + ordinary answer. The error card's "Branch without files" copies up to the turn before. */} {!error && !isActiveStreamingMessage && ( { }), ); }); + + it('copies without files only when that box is checked for this fork', async () => { + render( + + + , + ); + + fireEvent.click(screen.getByRole('button', { name: 'com_ui_fork_open_menu' })); + fireEvent.click(await screen.findByRole('checkbox', { name: 'com_ui_fork_exclude_files' })); + fireEvent.click(screen.getByRole('button', { name: 'com_ui_fork_visible' })); + + expect(mockMutate).toHaveBeenLastCalledWith( + expect.objectContaining({ + messageId: 'assistant-1', + option: ForkOptions.DIRECT_PATH, + excludeFiles: true, + }), + ); + }); + + it('keeps files by default', async () => { + render( + + + , + ); + + fireEvent.click(screen.getByRole('button', { name: 'com_ui_fork_open_menu' })); + fireEvent.click(await screen.findByRole('button', { name: 'com_ui_fork_visible' })); + + expect(mockMutate.mock.calls.at(-1)?.[0]).not.toHaveProperty('excludeFiles'); + }); }); diff --git a/client/src/components/Chat/Messages/__tests__/HoverButtons.spec.tsx b/client/src/components/Chat/Messages/__tests__/HoverButtons.spec.tsx index 4dfdc6a49a..bbb0a51055 100644 --- a/client/src/components/Chat/Messages/__tests__/HoverButtons.spec.tsx +++ b/client/src/components/Chat/Messages/__tests__/HoverButtons.spec.tsx @@ -162,6 +162,30 @@ describe('HoverButtons edit affordance', () => { expect(screen.getByTestId('copy-response-button')).toBeEnabled(); }); + /** A fork of the failed row would copy the provider's error as an ordinary answer, and its text + * is the persisted error, so neither is offered; the error card carries the way out. */ + it('withholds fork and edit on an error response', () => { + const errorMessage = { + ...userMessage, + messageId: 'assistant-error', + parentMessageId: 'user-1', + isCreatedByUser: false, + error: true, + text: 'Error uploading code environment file: 429', + } as TMessage; + + const container = renderHoverButtons({ + isSubmitting: false, + message: errorMessage, + isLast: true, + latestMessageId: errorMessage.messageId, + thread: [userMessage, errorMessage], + }); + + expect(screen.queryByRole('button', { name: 'Open Fork Menu' })).toBeNull(); + expect(container.querySelector(`#edit-${errorMessage.messageId}`)).toBeNull(); + }); + it('disables copy when the response serializes to nothing', () => { const errorPartMessage = { ...userMessage, diff --git a/client/src/components/Messages/Content/Error/attachments.ts b/client/src/components/Messages/Content/Error/attachments.ts new file mode 100644 index 0000000000..50eb3cb230 --- /dev/null +++ b/client/src/components/Messages/Content/Error/attachments.ts @@ -0,0 +1,149 @@ +import { + Constants, + ErrorTypes, + findMessageById, + messageCarriesFiles, + parseLangChainErrorCode, +} from 'librechat-data-provider'; +import type { TMessage } from 'librechat-data-provider'; +import { isSubmittableMessage } from '~/utils/messages'; +import { extractJson, isJson } from '~/utils/json'; + +/** + * Failures that come from what a conversation carries rather than from the model or the network: + * a file the code environment refused or rate-limited, a file that could not be restored or is + * gone, a turn over the attachment limits, documents the provider rejected, and a request too long + * for the context window, which a conversation full of screenshots or documents reaches. Every + * later turn resends the same files, so each of these recurs until the files leave the branch. + */ +const attachmentErrorPatterns: RegExp[] = [ + /code environment file/i, + new RegExp(ErrorTypes.RESOURCE_RECOVERY_REQUIRED, 'i'), + new RegExp(`"(${ErrorTypes.INPUT_LENGTH}|${ErrorTypes.FINAL_CONTEXT_OVERFLOW})"`, 'i'), + /prompt is too long|input is too long|context_length_exceeded|maximum context length/i, + /request_too_large|request entity too large|payload too large/i, + /\battach(ed|ments?)\b/i, + /duplicate document names?/i, + /\b(image|document|pdf|file)s?\b[^.\n]{0,60}\b(too large|too big|exceeds?|too many|could not be (processed|read)|unsupported|not supported)\b/i, +]; + +/** Whether a failed turn's text reads as a failure caused by the conversation's files. */ +export function isAttachmentError(text: string): boolean { + return attachmentErrorPatterns.some((pattern) => pattern.test(text)); +} + +/** + * Typed failures whose cause the payload already states, where files may still be to blame: the + * provider refused or could not complete the request without saying it was the files. + */ +const fileAgnosticErrorCodes = new Set([ + /** Anthropic's own error bodies are typed just `error`. */ + 'error', + ErrorTypes.UPSTREAM_MODEL_ERROR, + ErrorTypes.INVALID_REQUEST, + ErrorTypes.GOOGLE_ERROR, + ErrorTypes.EMPTY_MESSAGES, +]); + +/** LangChain codes with a remedy of their own: the model, its key, or its rate limit. */ +const remedialLangChainCodes = new Set([ + 'MODEL_NOT_FOUND', + 'MODEL_RATE_LIMIT', + 'MODEL_AUTHENTICATION', +]); + +/** The code a typed payload names, top level first, then an OpenAI-style `error` envelope. */ +function readErrorCode(text: string): string | undefined { + const json = extractJson(text); + if (json === '' || !isJson(json)) { + return undefined; + } + const payload = JSON.parse(json) as { + code?: unknown; + type?: unknown; + error?: { code?: unknown; type?: unknown } | null; + }; + const outer = [payload.code, payload.type]; + const nested = [payload.error?.code, payload.error?.type]; + /** A generic `error` type is only the envelope; what failed is named inside it. */ + const candidates = payload.type === 'error' ? [...nested, ...outer] : [...outer, ...nested]; + return candidates.find((value): value is string => typeof value === 'string' && value !== ''); +} + +/** + * Whether dropping files is a plausible way out of this failure. Anything that reads as a file + * problem is; so is a provider failure that says nothing about its cause. A failure with its own + * remedy (a missing key, a balance, a rate or message limit, a moderation block) is not, and + * offering files as the fix there would point the reader the wrong way. + */ +export function mayBeAttachmentError(text: string): boolean { + if (isAttachmentError(text)) { + return true; + } + /** Rows persisted before LangChain failures were typed carry the code in the message text. */ + const langChainCode = parseLangChainErrorCode(text); + if (langChainCode != null && remedialLangChainCodes.has(langChainCode)) { + return false; + } + const code = readErrorCode(text); + return code == null || fileAgnosticErrorCodes.has(code); +} + +export type AttachmentRecovery = { + /** The user turn the failed response answered, which a retry sends again. */ + parent: TMessage; + /** The turn's own text can be sent again without the files it carried. */ + canRetry: boolean; + /** + * Earlier turns carry files too, which a retry would still resend, so the branch can only be + * cleared by copying it without any. Names the message the copy ends at: the response before + * the failed turn, so the copy has no failure in it and the failed message can be sent again. + */ + branchTargetId?: string; +}; + +/** + * Which ways out a failed response offers when its branch carries files. Walks only the failed + * turn's ancestors, through the messages array's memoized id index, so the cost is the branch + * depth rather than the conversation size. + */ +export function findAttachmentRecovery( + messages: TMessage[] | null | undefined, + failed: Pick, +): AttachmentRecovery | null { + if (failed.isCreatedByUser === true) { + return null; + } + const parent = findMessageById(messages, failed.parentMessageId); + if (parent == null || parent.isCreatedByUser !== true) { + return null; + } + + const visited = new Set([parent.messageId]); + let earlierCarriesFiles = false; + let currentId = parent.parentMessageId; + while (currentId != null && currentId !== Constants.NO_PARENT && !visited.has(currentId)) { + visited.add(currentId); + const ancestor = findMessageById(messages, currentId); + if (ancestor == null) { + break; + } + if (messageCarriesFiles(ancestor)) { + earlierCarriesFiles = true; + break; + } + currentId = ancestor.parentMessageId; + } + + const canRetry = messageCarriesFiles(parent) && isSubmittableMessage(parent.text, 0); + const previousId = parent.parentMessageId; + const branchTargetId = + earlierCarriesFiles && previousId != null && previousId !== Constants.NO_PARENT + ? previousId + : undefined; + + if (!canRetry && branchTargetId == null) { + return null; + } + return { parent, canRetry, branchTargetId }; +} diff --git a/client/src/components/Messages/Content/Error/parts.tsx b/client/src/components/Messages/Content/Error/parts.tsx index f28e23cb75..9c58841f8d 100644 --- a/client/src/components/Messages/Content/Error/parts.tsx +++ b/client/src/components/Messages/Content/Error/parts.tsx @@ -340,14 +340,26 @@ export function ErrorActions({ children }: { children: React.ReactNode }) { export function ErrorAction({ onClick, children, + variant, + disabled = false, }: { onClick: () => void; children: React.ReactNode; + /** A second action next to the primary one reads as the alternative, not a peer. */ + variant?: 'outline'; + disabled?: boolean; }) { return ( /** One step tighter than the button default (8px) so the control reads as nested inside the * error card's 12px corner rather than echoing it. */ - ); diff --git a/client/src/components/Messages/Content/Error/recovery.tsx b/client/src/components/Messages/Content/Error/recovery.tsx new file mode 100644 index 0000000000..dc2683c8df --- /dev/null +++ b/client/src/components/Messages/Content/Error/recovery.tsx @@ -0,0 +1,249 @@ +import { useContext, useRef } from 'react'; +import { useStore } from 'jotai'; +import { useRecoilCallback } from 'recoil'; +import { useToastContext } from '@librechat/client'; +import { + ForkOptions, + ContentTypes, + findMessageById, + messageCarriesFiles, + isAssistantsEndpoint, +} from 'librechat-data-provider'; +import type { TMessage } from 'librechat-data-provider'; +import type { AttachmentRecovery } from './attachments'; +import type { TranslationKeys } from '~/hooks'; +import { + useMessagesIsSubmitting, + useIsMessagesViewReadOnly, + useOptionalMessagesOperations, + useOptionalMessagesConversation, +} from '~/Providers/MessagesViewContext'; +import { + getReasoningStateKey, + pendingReasoningOverrideFamily, +} from '~/components/Chat/Input/Composer/state'; +import { findAttachmentRecovery, isAttachmentError, mayBeAttachmentError } from './attachments'; +import { useMessageContext } from '~/Providers/MessageContext'; +import { useLocalize, useNavigateToConvo } from '~/hooks'; +import { useForkConvoMutation } from '~/data-provider'; +import { ChatContext } from '~/Providers/ChatContext'; +import { ErrorAction, ErrorActions } from './parts'; +import { useGetAddedConvo } from '~/hooks/Chat'; +import store from '~/store'; + +/** What the card says the actions do, matched to the ones it offers. */ +function getExplanationKey({ canRetry, branchTargetId }: AttachmentRecovery): TranslationKeys { + if (!canRetry) { + return 'com_error_attachments_branch_info'; + } + return branchTargetId == null + ? 'com_error_attachments_retry_info' + : 'com_error_attachments_both_info'; +} + +/** An error row renders one card per error part; the recovery belongs under the last one. */ +function isLastErrorPart(message: TMessage, partIndex: number | undefined): boolean { + if (partIndex == null || !Array.isArray(message.content)) { + return true; + } + for (let i = message.content.length - 1; i >= 0; i--) { + if (message.content[i]?.type === ContentTypes.ERROR) { + return i === partIndex; + } + } + return true; +} + +const getStatus = (error: unknown): number | undefined => { + const candidate = error as { + status?: number; + statusCode?: number; + response?: { status?: number }; + }; + return candidate?.response?.status ?? candidate?.status ?? candidate?.statusCode; +}; + +/** + * The way out of a turn that fails because of the files its branch carries. Every later turn + * resends those files, so a plain retry fails the same way; this offers to send the failed message + * again without its own files, and, when earlier turns carry files too, to continue in a copy of + * the conversation that has none. Both leave the original untouched: the retry is a new version of + * the message beside the old one, and the copy is a new conversation. + */ +function RecoveryActions({ + text, + recovery, + conversationId, +}: { + text: string; + recovery: AttachmentRecovery; + conversationId: string; +}) { + const localize = useLocalize(); + const isSubmitting = useMessagesIsSubmitting(); + const { ask } = useOptionalMessagesOperations(); + const getAddedConvo = useGetAddedConvo(); + const { navigateToConvo } = useNavigateToConvo(); + const { showToast } = useToastContext(); + const reasoningStore = useStore(); + /** The failed turn, staged in the copy's composer so it can be sent again as it was. */ + const failedTurnRef = useRef(undefined); + + /** + * Hands the failed turn to the copy's composer the way a message is put back for editing: its + * text through the in-memory handoff the composer drains on mount (never the draft store, which + * the Save Drafts preference may switch off), and its quotes, manual skills and reasoning + * override through their compose-time atoms. Only its files stay behind. + */ + const stageFailedTurn = useRecoilCallback( + ({ set }) => + (forkedId: string, turn: TMessage): boolean => { + const hasText = turn.text.trim() !== ''; + if (hasText) { + set(store.pendingComposerTextByConvoId(forkedId), turn.text); + } + if ((turn.quotes?.length ?? 0) > 0) { + set(store.pendingQuotesByConvoId(forkedId), turn.quotes ?? []); + } + if ((turn.manualSkills?.length ?? 0) > 0) { + set(store.pendingManualSkillsByConvoId(forkedId), turn.manualSkills ?? []); + } + if (turn.reasoningOverride != null) { + reasoningStore.set( + pendingReasoningOverrideFamily(getReasoningStateKey(forkedId, 0)), + turn.reasoningOverride, + ); + } + return hasText; + }, + [reasoningStore], + ); + + const forkConvo = useForkConvoMutation({ + onSuccess: (data) => { + /** A server that predates `excludeFiles` ignores it and returns an ordinary copy, which + * would fail the same way; say so rather than announce a recovery that did not happen. */ + if (data.messages.some((message) => messageCarriesFiles(message))) { + showToast({ + message: localize('com_ui_branch_without_files_unsupported'), + status: 'error', + }); + return; + } + const turn = failedTurnRef.current; + const forkedId = data.conversation.conversationId; + const staged = turn != null && forkedId != null && stageFailedTurn(forkedId, turn); + navigateToConvo(data.conversation); + showToast({ + message: localize( + staged ? 'com_ui_branch_without_files_draft' : 'com_ui_branch_without_files_done', + ), + status: 'success', + }); + }, + onError: (error) => { + showToast({ + message: localize( + getStatus(error) === 429 ? 'com_ui_fork_error_rate_limit' : 'com_ui_fork_error', + ), + status: 'error', + }); + }, + }); + + const { parent, canRetry, branchTargetId } = recovery; + const busy = isSubmitting || forkConvo.isLoading; + + const retryWithoutFiles = () => { + ask( + { text: parent.text, parentMessageId: parent.parentMessageId, conversationId }, + { + overrideFiles: [], + overrideManualSkills: parent.manualSkills, + overrideQuotes: parent.quotes, + overrideReasoning: parent.reasoningOverride ?? null, + addedConvo: getAddedConvo() || undefined, + }, + ); + }; + + const branchWithoutFiles = () => { + if (branchTargetId == null) { + return; + } + failedTurnRef.current = parent; + forkConvo.mutate({ + conversationId, + messageId: branchTargetId, + option: ForkOptions.DIRECT_PATH, + excludeFiles: true, + }); + }; + + return ( +
+

+ {isAttachmentError(text) && <>{localize('com_error_attachments_cause')} } + {localize(getExplanationKey(recovery))} +

+ + {canRetry && ( + + {localize('com_ui_retry_without_files')} + + )} + {branchTargetId != null && ( + + {localize('com_ui_branch_without_files')} + + )} + +
+ ); +} + +/** + * Offers `RecoveryActions` under a failed response whose branch carries files. Only a live, + * writable chat can act on them, so a search result, a shared link or a read-only subagent thread + * shows the error alone, and none of them mounts the hooks the actions need. + */ +export default function AttachmentRecoveryActions({ + text, + messageId: rowMessageId, +}: { + text: string; + messageId?: string; +}) { + const chat = useContext(ChatContext); + const readOnly = useIsMessagesViewReadOnly(); + const { conversation } = useOptionalMessagesConversation(); + const { getMessages } = useOptionalMessagesOperations(); + const { messageId: contextMessageId, partIndex } = useMessageContext(); + + /** An assistants thread keeps its own copy of every attachment upstream, so neither a retry + * nor a local copy leaves them behind; those endpoints offer no fork for the same reason. */ + if ( + chat == null || + readOnly || + conversation?.subagentThread != null || + isAssistantsEndpoint(conversation?.endpoint) || + !mayBeAttachmentError(text) + ) { + return null; + } + const messages = getMessages(); + const message = findMessageById(messages, rowMessageId ?? contextMessageId); + if (message == null || !isLastErrorPart(message, rowMessageId ? undefined : partIndex)) { + return null; + } + const recovery = findAttachmentRecovery(messages, message); + const conversationId = recovery?.parent.conversationId ?? conversation?.conversationId; + if (recovery == null || !conversationId) { + return null; + } + return ; +} diff --git a/client/src/components/Messages/Content/__tests__/attachments.spec.ts b/client/src/components/Messages/Content/__tests__/attachments.spec.ts new file mode 100644 index 0000000000..628bb2460f --- /dev/null +++ b/client/src/components/Messages/Content/__tests__/attachments.spec.ts @@ -0,0 +1,233 @@ +import { Constants, ErrorTypes, ContentTypes } from 'librechat-data-provider'; +import type { TMessage } from 'librechat-data-provider'; +import { + isAttachmentError, + mayBeAttachmentError, + findAttachmentRecovery, +} from '../Error/attachments'; + +describe('isAttachmentError', () => { + it.each([ + ['a code environment upload rate limit', 'Error uploading code environment file: 429'], + [ + 'the agent attachment limit counting resent files', + 'An error occurred while processing the request: This turn exceeds the configured attachment count limit (12 > 10). Remove some attachments or use smaller files and try again.', + ], + [ + 'duplicate document names on Bedrock', + JSON.stringify({ + type: ErrorTypes.UPSTREAM_MODEL_ERROR, + message: + 'Messages can’t contain duplicate document names. Rename the document and retry your request.', + }), + ], + ['an unrestorable file', JSON.stringify({ type: ErrorTypes.RESOURCE_RECOVERY_REQUIRED })], + ['an over-long prompt', 'An error occurred while processing the request: prompt is too long'], + ['a typed input length error', JSON.stringify({ type: ErrorTypes.INPUT_LENGTH, info: '9/5' })], + [ + 'a typed context overflow', + JSON.stringify({ type: ErrorTypes.FINAL_CONTEXT_OVERFLOW, projectedMessageTokens: 9 }), + ], + ['a missing attachment', 'An attached file is no longer available. Remove it and retry.'], + ['an oversized image', 'messages.3.content.1.image.source.base64: image exceeds 5 MB maximum'], + ])('recognizes %s', (_label, text) => { + expect(isAttachmentError(text)).toBe(true); + }); + + it.each([ + ['a model rate limit', JSON.stringify({ type: ErrorTypes.MODEL_RATE_LIMIT })], + ['a dropped stream', 'Error connecting to server, try refreshing the page.'], + ['an invalid key', JSON.stringify({ type: ErrorTypes.INVALID_USER_KEY })], + ['provider prose', 'An error occurred while processing the request: 500 Internal Server Error'], + ])('does not blame files for %s', (_label, text) => { + expect(isAttachmentError(text)).toBe(false); + }); +}); + +describe('mayBeAttachmentError', () => { + it.each([ + ['a file failure', 'Error uploading code environment file: 429'], + ['an unexplained provider failure', JSON.stringify({ type: ErrorTypes.UPSTREAM_MODEL_ERROR })], + [ + 'an Anthropic error body', + JSON.stringify({ type: 'error', error: { type: 'invalid_request_error', message: 'bad' } }), + ], + ['provider prose', 'An error occurred while processing the request: 400 Bad Request'], + ])('offers the files as a way out of %s', (_label, text) => { + expect(mayBeAttachmentError(text)).toBe(true); + }); + + it.each([ + ['a missing key', JSON.stringify({ type: ErrorTypes.NO_USER_KEY })], + ['a model rate limit', JSON.stringify({ type: ErrorTypes.MODEL_RATE_LIMIT })], + ['a token balance', JSON.stringify({ type: 'token_balance', balance: 0 })], + [ + 'a legacy LangChain rate limit', + 'An error occurred while processing the request: 429 Too many requests\n\nTroubleshooting URL: https://docs.langchain.com/oss/javascript/langchain/errors/MODEL_RATE_LIMIT/', + ], + ['a nested quota code', JSON.stringify({ error: { code: 'insufficient_quota' } })], + [ + 'an Anthropic rate limit inside its generic envelope', + JSON.stringify({ type: 'error', error: { type: 'rate_limit_error', message: 'slow down' } }), + ], + ])('does not point at files for %s', (_label, text) => { + expect(mayBeAttachmentError(text)).toBe(false); + }); +}); + +const screenshot = { file_id: 'screenshot', filename: 'screenshot.png', type: 'image/png' }; + +const user = ( + messageId: string, + parentMessageId: string, + fields: Partial = {}, +): TMessage => + ({ + messageId, + parentMessageId, + conversationId: 'convo', + isCreatedByUser: true, + text: `question ${messageId}`, + ...fields, + }) as TMessage; + +const answer = ( + messageId: string, + parentMessageId: string, + fields: Partial = {}, +): TMessage => + ({ + messageId, + parentMessageId, + conversationId: 'convo', + isCreatedByUser: false, + text: `answer ${messageId}`, + ...fields, + }) as TMessage; + +describe('findAttachmentRecovery', () => { + it('offers only a retry when the failed turn alone carries files', () => { + const messages = [ + user('u1', Constants.NO_PARENT), + answer('a1', 'u1'), + user('u2', 'a1', { files: [screenshot] }), + answer('a2', 'u2', { error: true }), + ]; + + const recovery = findAttachmentRecovery(messages, messages[3]); + + expect(recovery).toEqual({ parent: messages[2], canRetry: true, branchTargetId: undefined }); + }); + + it('also offers a branch ending before the failed turn when earlier turns carry files', () => { + const messages = [ + user('u1', Constants.NO_PARENT, { files: [screenshot] }), + answer('a1', 'u1'), + user('u2', 'a1', { files: [screenshot] }), + answer('a2', 'u2', { error: true }), + ]; + + expect(findAttachmentRecovery(messages, messages[3])).toEqual({ + parent: messages[2], + canRetry: true, + branchTargetId: 'a1', + }); + }); + + it('counts code outputs on earlier answers as files', () => { + const messages = [ + user('u1', Constants.NO_PARENT), + answer('a1', 'u1', { + attachments: [{ file_id: 'chart', toolCallId: 't' }] as TMessage['attachments'], + }), + user('u2', 'a1'), + answer('a2', 'u2', { error: true }), + ]; + + expect(findAttachmentRecovery(messages, messages[3])).toEqual({ + parent: messages[2], + canRetry: false, + branchTargetId: 'a1', + }); + }); + + it('counts attachments steered into an earlier response as files', () => { + const steered = [ + { type: ContentTypes.TEXT, text: 'Working on it.' }, + { type: ContentTypes.STEER, steer: 'use this', steerId: 's1', files: [screenshot] }, + ] as TMessage['content']; + const messages = [ + user('u1', Constants.NO_PARENT), + answer('a1', 'u1', { content: steered }), + user('u2', 'a1'), + answer('a2', 'u2', { error: true }), + ]; + + expect(findAttachmentRecovery(messages, messages[3])).toEqual({ + parent: messages[2], + canRetry: false, + branchTargetId: 'a1', + }); + }); + + it('counts a generated image in an earlier response as a file', () => { + const messages = [ + user('u1', Constants.NO_PARENT), + answer('a1', 'u1', { + content: [ + { type: ContentTypes.IMAGE_FILE, image_file: { file_id: 'generated' } }, + ] as TMessage['content'], + }), + user('u2', 'a1'), + answer('a2', 'u2', { error: true }), + ]; + + expect(findAttachmentRecovery(messages, messages[3])?.branchTargetId).toBe('a1'); + }); + + it('does not count search sources as files', () => { + const messages = [ + user('u1', Constants.NO_PARENT), + answer('a1', 'u1', { + attachments: [{ type: 'web_search', toolCallId: 't' }] as TMessage['attachments'], + }), + user('u2', 'a1'), + answer('a2', 'u2', { error: true }), + ]; + + expect(findAttachmentRecovery(messages, messages[3])).toBeNull(); + }); + + it('ignores files on a sibling branch', () => { + const messages = [ + user('u1', Constants.NO_PARENT), + answer('a1', 'u1'), + user('sibling', 'a1', { files: [screenshot] }), + user('u2', 'a1'), + answer('a2', 'u2', { error: true }), + ]; + + expect(findAttachmentRecovery(messages, messages[4])).toBeNull(); + }); + + it('offers nothing to retry for a files-only message, since there is no text to resend', () => { + const messages = [ + user('u1', Constants.NO_PARENT, { text: '', files: [screenshot] }), + answer('a1', 'u1', { error: true }), + ]; + + expect(findAttachmentRecovery(messages, messages[1])).toBeNull(); + }); + + it('offers nothing on a user turn or a response with no user turn behind it', () => { + const messages = [ + user('u1', Constants.NO_PARENT, { files: [screenshot] }), + answer('a1', 'u1'), + answer('a2', 'a1', { error: true }), + ]; + + expect(findAttachmentRecovery(messages, messages[0])).toBeNull(); + expect(findAttachmentRecovery(messages, messages[2])).toBeNull(); + expect(findAttachmentRecovery(undefined, messages[1])).toBeNull(); + }); +}); diff --git a/client/src/components/Messages/Content/__tests__/recovery.spec.tsx b/client/src/components/Messages/Content/__tests__/recovery.spec.tsx new file mode 100644 index 0000000000..b5ee825142 --- /dev/null +++ b/client/src/components/Messages/Content/__tests__/recovery.spec.tsx @@ -0,0 +1,330 @@ +import React from 'react'; +import { getDefaultStore } from 'jotai'; +import userEvent from '@testing-library/user-event'; +import { RecoilRoot, useRecoilValue } from 'recoil'; +import { act, render, screen } from '@testing-library/react'; +import { Constants, ContentTypes, ForkOptions } from 'librechat-data-provider'; +import type { TConversation, TMessage } from 'librechat-data-provider'; +import type { MessagesViewContextValue } from '~/Providers/MessagesViewContext'; +import type { ChatContract } from '~/hooks/Chat/contract'; +import { + MessagesViewContext, + MessagesSubmittingContext, + MessagesOperationsContext, +} from '~/Providers/MessagesViewContext'; +import { + getReasoningStateKey, + pendingReasoningOverrideFamily, +} from '~/components/Chat/Input/Composer/state'; +import { MessageContext } from '~/Providers/MessageContext'; +import AttachmentRecoveryActions from '../Error/recovery'; +import translation from '~/locales/en/translation.json'; +import { ChatContext } from '~/Providers/ChatContext'; +import store from '~/store'; + +const catalog = translation as Record; + +const mockNavigateToConvo = jest.fn(); +const mockShowToast = jest.fn(); +const mockForkMutate = jest.fn(); +let mockForkOptions: { + onSuccess?: (data: { conversation: TConversation; messages: TMessage[] }) => void; +} = {}; + +jest.mock('~/hooks', () => ({ + useLocalize: () => (key: string) => + (jest.requireActual('~/locales/en/translation.json') as Record)[key] ?? key, + useNavigateToConvo: () => ({ navigateToConvo: mockNavigateToConvo }), +})); + +jest.mock('~/hooks/Chat', () => ({ + useGetAddedConvo: () => () => null, +})); + +jest.mock('~/data-provider', () => ({ + useForkConvoMutation: (options: typeof mockForkOptions) => { + mockForkOptions = options; + return { mutate: mockForkMutate, isLoading: false }; + }, +})); + +jest.mock('@librechat/client', () => ({ + ...jest.requireActual('@librechat/client'), + useToastContext: () => ({ showToast: mockShowToast }), +})); + +const screenshot = { file_id: 'screenshot', filename: 'screenshot.png', type: 'image/png' }; + +const user = (messageId: string, parentMessageId: string, fields: Partial = {}) => + ({ + messageId, + parentMessageId, + conversationId: 'convo', + isCreatedByUser: true, + text: `question ${messageId}`, + ...fields, + }) as TMessage; + +const answer = (messageId: string, parentMessageId: string, fields: Partial = {}) => + ({ + messageId, + parentMessageId, + conversationId: 'convo', + isCreatedByUser: false, + text: `answer ${messageId}`, + ...fields, + }) as TMessage; + +/** Reads what the forked conversation's composer was handed. */ +let composerHandoff: { text?: string; quotes: string[]; skills: string[] } | undefined; +function ComposerProbe() { + composerHandoff = { + text: useRecoilValue(store.pendingComposerTextByConvoId('forked')), + quotes: useRecoilValue(store.pendingQuotesByConvoId('forked')), + skills: useRecoilValue(store.pendingManualSkillsByConvoId('forked')), + }; + return null; +} + +const uploadErrorText = 'Error uploading code environment file: 429'; + +function renderRecovery({ + messages, + messageId = 'a2', + partIndex, + text = uploadErrorText, + inChat = true, + readOnly = false, + isSubmitting = false, + ask = jest.fn(() => true), + endpoint = 'agents', +}: { + messages: TMessage[]; + messageId?: string; + partIndex?: number; + text?: string; + inChat?: boolean; + readOnly?: boolean; + isSubmitting?: boolean; + ask?: jest.Mock; + endpoint?: string; +}) { + const conversation = { conversationId: 'convo', endpoint } as TConversation; + const operations = { + ask, + regenerate: jest.fn(), + handleContinue: jest.fn(), + getMessages: () => messages, + setMessages: jest.fn(), + } as unknown as MessagesViewContextValue; + const view = { + ...operations, + conversation, + conversationId: 'convo', + readOnly, + isSubmitting, + } as MessagesViewContextValue; + const content = ( + + + + + + + + + + ); + render( + + + {inChat ? ( + + {content} + + ) : ( + content + )} + , + ); + return { ask }; +} + +const retryFixture = () => [ + user('u1', Constants.NO_PARENT), + answer('a1', 'u1'), + user('u2', 'a1', { + files: [screenshot], + manualSkills: ['charts'], + quotes: ['quoted line'], + }), + answer('a2', 'u2', { error: true }), +]; + +const branchFixture = () => [ + user('u1', Constants.NO_PARENT, { files: [screenshot] }), + answer('a1', 'u1'), + user('u2', 'a1', { + files: [screenshot], + quotes: ['quoted line'], + manualSkills: ['charts'], + reasoningOverride: { key: 'reasoning_effort', value: 'high' } as TMessage['reasoningOverride'], + }), + answer('a2', 'u2', { error: true }), +]; + +describe('AttachmentRecoveryActions', () => { + beforeEach(() => { + jest.clearAllMocks(); + composerHandoff = undefined; + }); + + it('retries the failed turn as a new version without any files', async () => { + const messages = retryFixture(); + const { ask } = renderRecovery({ messages }); + + expect(screen.getByText(catalog.com_error_attachments_cause, { exact: false })).toBeTruthy(); + expect(screen.queryByRole('button', { name: catalog.com_ui_branch_without_files })).toBeNull(); + await userEvent.click(screen.getByRole('button', { name: catalog.com_ui_retry_without_files })); + + expect(ask).toHaveBeenCalledTimes(1); + expect(ask).toHaveBeenCalledWith( + { text: 'question u2', parentMessageId: 'a1', conversationId: 'convo' }, + expect.objectContaining({ + overrideFiles: [], + overrideManualSkills: ['charts'], + overrideQuotes: ['quoted line'], + overrideReasoning: null, + }), + ); + expect(ask.mock.calls[0][1]).not.toHaveProperty('isRegenerate'); + }); + + it('branches into a copy without files that ends before the failed turn', async () => { + renderRecovery({ messages: branchFixture() }); + + expect( + screen.getByText(catalog.com_error_attachments_both_info, { exact: false }), + ).toBeTruthy(); + await userEvent.click( + screen.getByRole('button', { name: catalog.com_ui_branch_without_files }), + ); + + expect(mockForkMutate).toHaveBeenCalledWith({ + conversationId: 'convo', + messageId: 'a1', + option: ForkOptions.DIRECT_PATH, + excludeFiles: true, + }); + + const forked = { conversationId: 'forked' } as TConversation; + const copied = [user('c1', Constants.NO_PARENT), answer('c2', 'c1')]; + act(() => mockForkOptions.onSuccess?.({ conversation: forked, messages: copied })); + + /** Everything the failed turn carried except its files reaches the copy's composer. */ + expect(composerHandoff).toEqual({ + text: 'question u2', + quotes: ['quoted line'], + skills: ['charts'], + }); + expect( + getDefaultStore().get(pendingReasoningOverrideFamily(getReasoningStateKey('forked', 0))), + ).toEqual({ key: 'reasoning_effort', value: 'high' }); + expect(mockNavigateToConvo).toHaveBeenCalledWith(forked); + expect(mockShowToast).toHaveBeenCalledWith( + expect.objectContaining({ message: catalog.com_ui_branch_without_files_draft }), + ); + }); + + it('reports a copy that still has its files instead of announcing a recovery', async () => { + renderRecovery({ messages: branchFixture() }); + await userEvent.click( + screen.getByRole('button', { name: catalog.com_ui_branch_without_files }), + ); + + const forked = { conversationId: 'forked' } as TConversation; + const copied = [user('c1', Constants.NO_PARENT, { files: [screenshot] }), answer('c2', 'c1')]; + act(() => mockForkOptions.onSuccess?.({ conversation: forked, messages: copied })); + + expect(mockNavigateToConvo).not.toHaveBeenCalled(); + expect(composerHandoff?.text).toBeUndefined(); + expect(mockShowToast).toHaveBeenCalledWith( + expect.objectContaining({ + message: catalog.com_ui_branch_without_files_unsupported, + status: 'error', + }), + ); + }); + + it('offers nothing in an assistants conversation, whose thread keeps its own files', () => { + renderRecovery({ messages: branchFixture(), endpoint: 'assistants' }); + + expect(screen.queryByRole('button')).toBeNull(); + }); + + it('offers the actions without blaming files for an unrelated failure', () => { + renderRecovery({ messages: retryFixture(), text: 'Model returned 500' }); + + expect(screen.queryByText(catalog.com_error_attachments_cause, { exact: false })).toBeNull(); + expect(screen.getByText(catalog.com_error_attachments_retry_info)).toBeTruthy(); + }); + + it('stays out of a failure with its own remedy', () => { + renderRecovery({ messages: retryFixture(), text: JSON.stringify({ type: 'no_user_key' }) }); + + expect(screen.queryByRole('button')).toBeNull(); + }); + + it('renders nothing when the branch carries no files', () => { + const messages = [user('u1', Constants.NO_PARENT), answer('a2', 'u1', { error: true })]; + renderRecovery({ messages }); + + expect(screen.queryByRole('button')).toBeNull(); + }); + + it.each([ + ['outside a chat', { inChat: false }], + ['in a read-only view', { readOnly: true }], + ])('renders nothing %s', (_label, options) => { + renderRecovery({ messages: retryFixture(), ...options }); + + expect(screen.queryByRole('button')).toBeNull(); + }); + + it('disables the actions while a response is generating', () => { + renderRecovery({ messages: branchFixture(), isSubmitting: true }); + + for (const button of screen.getAllByRole('button')) { + expect((button as HTMLButtonElement).disabled).toBe(true); + } + }); + + it('shows the actions once, under the last error part of a row', () => { + const messages = retryFixture(); + messages[3] = answer('a2', 'u2', { + content: [ + { type: ContentTypes.ERROR, [ContentTypes.ERROR]: 'first' }, + { type: ContentTypes.TEXT, text: 'partial' }, + { type: ContentTypes.ERROR, [ContentTypes.ERROR]: uploadErrorText }, + ] as TMessage['content'], + }); + + renderRecovery({ messages, partIndex: 0 }); + expect(screen.queryByRole('button')).toBeNull(); + }); + + it('shows the actions under the last error part', () => { + const messages = retryFixture(); + messages[3] = answer('a2', 'u2', { + content: [ + { type: ContentTypes.ERROR, [ContentTypes.ERROR]: 'first' }, + { type: ContentTypes.ERROR, [ContentTypes.ERROR]: uploadErrorText }, + ] as TMessage['content'], + }); + + renderRecovery({ messages, partIndex: 1 }); + expect(screen.getByRole('button', { name: catalog.com_ui_retry_without_files })).toBeTruthy(); + }); +}); diff --git a/client/src/locales/en/translation.json b/client/src/locales/en/translation.json index 0b5d0574de..406ab4e7dd 100644 --- a/client/src/locales/en/translation.json +++ b/client/src/locales/en/translation.json @@ -693,6 +693,10 @@ "com_error_no_user_key_generic": "An API key is required to send messages.", "com_error_non_browser": "This request was blocked because it did not come from a browser.", "com_error_attachment_limit": "LibreChat's attachment limit was exceeded. Remove some attachments or use smaller files and try again.", + "com_error_attachments_both_info": "Retry without files resends only your last message without its files. Branch without files copies the conversation into a new chat with no files at all. Your original conversation is kept.", + "com_error_attachments_branch_info": "Branch without files copies this conversation into a new chat with no files. Your original conversation is kept.", + "com_error_attachments_cause": "The files in this conversation are likely causing this, and every new message sends them again.", + "com_error_attachments_retry_info": "Retry without files sends your message again without its files. Your original message is kept.", "com_error_details_attachment": "Attachment limit details", "com_error_provider_failed": "{{0}} could not complete this request.", "com_error_refusal": "{{0}} declined to answer this request. Rewrite your message and try again.", @@ -1353,6 +1357,10 @@ "com_ui_branch_created": "Branch created successfully", "com_ui_branch_error": "Failed to create branch", "com_ui_branch_message": "Create branch from this response", + "com_ui_branch_without_files": "Branch without files", + "com_ui_branch_without_files_done": "Copied this conversation without files.", + "com_ui_branch_without_files_draft": "Copied without files. Your message is ready to send.", + "com_ui_branch_without_files_unsupported": "The copy still has its files, because this server can't leave them out yet. Your original conversation is unchanged.", "com_ui_bulk_delete_error": "Some items could not be deleted", "com_ui_by_author": "by {{0}}", "com_ui_cache_read": "Cache read", @@ -1662,6 +1670,7 @@ "com_ui_during_run_actions": "More send options", "com_ui_edit": "Edit", "com_ui_edit_editing_image": "Editing image", + "com_ui_edit_files_removed": "Rerun to send without the removed files. Your original message stays available as an earlier version.", "com_ui_edit_mcp_server": "Edit MCP Server", "com_ui_edit_mcp_server_dialog_description": "Unique Server Identifier: {{serverName}}", "com_ui_edit_memory": "Edit Memory", @@ -1774,12 +1783,14 @@ "com_ui_fork_default": "Use default fork option", "com_ui_fork_error": "There was an error forking the conversation", "com_ui_fork_error_rate_limit": "Too many fork requests. Please try again later", + "com_ui_fork_exclude_files": "Leave out files", "com_ui_fork_from_message": "Select a fork option", "com_ui_fork_info_1": "Use this setting to fork messages with the desired behavior.", "com_ui_fork_info_2": "\"Forking\" refers to creating a new conversation that start/end from specific messages in the current conversation, creating a copy according to the options selected.", "com_ui_fork_info_3": "The \"target message\" refers to either the message this popup was opened from, or, if you check \"{{0}}\", the latest message in the conversation.", "com_ui_fork_info_branches": "This option forks the visible messages, along with related branches; in other words, the direct path to the target message, including branches along the path.", "com_ui_fork_info_button_label": "View information about forking conversations", + "com_ui_fork_info_exclude_files": "If checked, the new conversation copies the messages without their attached or generated files, so later messages don't send them again.", "com_ui_fork_info_remember": "Check this to remember the options you select for future usage, making it quicker to fork conversations as preferred.", "com_ui_fork_info_start": "If checked, forking will commence from this message to the latest message in the conversation, according to the behavior selected above.", "com_ui_fork_info_target": "This option forks all messages leading up to the target message, including its neighbors; in other words, all message branches, whether or not they are visible or along the same path, are included.", @@ -2603,6 +2614,7 @@ "com_ui_remote_agents_allow_share": "Allow users to grant API access to agents to others", "com_ui_remote_agents_allow_share_public": "Allow users to grant API access to agents to all users", "com_ui_remote_agents_allow_use": "Allow users to create API keys and query agents remotely", + "com_ui_remove_file_named": "Remove {{0}}", "com_ui_remove_from_project": "Remove from project", "com_ui_remove_queued": "Remove message", "com_ui_dismiss_unconfirmed_delivery": "Dismiss unconfirmed delivery", @@ -2633,6 +2645,7 @@ "com_ui_results_found_more": "{{count}}+ results found", "com_ui_retrieved_files": "Searched your files", "com_ui_retry": "Retry", + "com_ui_retry_without_files": "Retry without files", "com_ui_revoke": "Revoke", "com_ui_revoke_info": "Revoke all provider API keys", "com_ui_revoke_key_confirm": "Are you sure you want to revoke this key?", diff --git a/packages/api/src/agents/__tests__/initialize.test.ts b/packages/api/src/agents/__tests__/initialize.test.ts index 9ac8f6bcf5..143e4c95d8 100644 --- a/packages/api/src/agents/__tests__/initialize.test.ts +++ b/packages/api/src/agents/__tests__/initialize.test.ts @@ -4617,6 +4617,40 @@ describe('initializeAgent — code-generated file thread filter (regression)', ( ); }); + it('takes no conversation files for a turn sent at the root', async () => { + /* A root turn has no history: the conversation's file refs belong to sibling roots, such + * as the first message edited or retried without its files, and must not come back. */ + const { agent, req, res, loadTools, db } = setupExecuteCodeAgent(); + + const getMessages = jest.fn().mockResolvedValue([]); + const getConvoFiles = jest.fn().mockResolvedValue(['sibling-root-file']); + const getDeferredProvisionFiles = jest.fn().mockResolvedValue([]); + const getToolFilesByIds = jest.fn().mockResolvedValue([]); + + await initializeAgent( + { + req, + res, + agent, + loadTools, + endpointOption: { endpoint: EModelEndpoint.agents }, + conversationId: 'conv-1', + parentMessageId: Constants.NO_PARENT as string, + allowedProviders: new Set([Providers.OPENAI]), + isInitialAgent: true, + codeEnvAvailable: true, + }, + { ...db, getMessages, getConvoFiles, getDeferredProvisionFiles, getToolFilesByIds }, + ); + + expect(getConvoFiles).not.toHaveBeenCalled(); + expect(getMessages).not.toHaveBeenCalled(); + expect(getDeferredProvisionFiles).not.toHaveBeenCalled(); + for (const [ids] of getToolFilesByIds.mock.calls) { + expect(ids).toEqual([]); + } + }); + it('keeps an anchored branch scoped to itself when it references no files', async () => { /* Widening an empty branch to the conversation would provision a sibling branch's * attachments, sending files this branch never mentioned to the Code API or RAG. */ diff --git a/packages/api/src/agents/initialize.ts b/packages/api/src/agents/initialize.ts index 4eb34948a1..c37937cb15 100644 --- a/packages/api/src/agents/initialize.ts +++ b/packages/api/src/agents/initialize.ts @@ -1682,6 +1682,10 @@ export async function initializeAgent( /** Falsy anchors cannot match a parent chain, so they get no walk. */ const threadAnchor = parentMessageId && parentMessageId !== Constants.NO_PARENT ? parentMessageId : null; + /* A turn sent at the root has no history at all. The conversation's file refs then + * belong to sibling roots (an edited first message, or one retried without its files), + * so a root turn takes none of them, unlike a continuation that names no anchor. */ + const isRootTurn = parentMessageId === Constants.NO_PARENT; /* Either provisioning resource needs the anchor: deferred attachments for * file_search are found by thread file ids just as code files are. */ const needsThreadWalk = wantsProvisioning && threadAnchor != null && getThreadMessages != null; @@ -1697,7 +1701,10 @@ export async function initializeAgent( * every code-output ref. */ const [convoFileIds, threadMessages] = await Promise.all([ - readResolvedConversationFiles(runtime, conversationId) ?? db.getConvoFiles(conversationId), + isRootTurn + ? [] + : (readResolvedConversationFiles(runtime, conversationId) ?? + db.getConvoFiles(conversationId)), needsThreadWalk && getThreadMessages ? getThreadMessages({ conversationId }, 'messageId parentMessageId files attachments') : null, diff --git a/packages/api/src/conversations/files.spec.ts b/packages/api/src/conversations/files.spec.ts new file mode 100644 index 0000000000..67ca5f5368 --- /dev/null +++ b/packages/api/src/conversations/files.spec.ts @@ -0,0 +1,134 @@ +import { ContentTypes } from 'librechat-data-provider'; +import type { TAttachment, TConversation, TMessage } from 'librechat-data-provider'; +import { forkFileScope, withoutMessageFiles, withoutConversationFiles } from './files'; + +const upload = { file_id: 'upload-1', filename: 'screenshot.png', type: 'image/png' }; +const output = { file_id: 'output-1', filename: 'chart.png', toolCallId: 'call-1' } as TAttachment; + +describe('withoutMessageFiles', () => { + it('drops uploads and the token count that covered them, keeping the text', () => { + const message: Partial = { + messageId: 'user-1', + text: 'What is in this screenshot?', + isCreatedByUser: true, + files: [upload], + tokenCount: 1600, + }; + + expect(withoutMessageFiles(message)).toEqual({ + messageId: 'user-1', + text: 'What is in this screenshot?', + isCreatedByUser: true, + }); + }); + + it('drops code outputs but keeps the token count of a turn that had no uploads', () => { + const content = [{ type: 'text', text: 'Here is the chart.' }] as TMessage['content']; + const message: Partial = { + messageId: 'assistant-1', + text: '', + content, + attachments: [output], + tokenCount: 42, + }; + + expect(withoutMessageFiles(message)).toEqual({ + messageId: 'assistant-1', + text: '', + content, + tokenCount: 42, + }); + }); + + it('drops attachments steered into a response, and the count that covered them', () => { + const content = [ + { type: ContentTypes.TEXT, text: 'Working on it.' }, + { type: ContentTypes.STEER, steer: 'Use this one instead', steerId: 's1', files: [upload] }, + { type: ContentTypes.STEER, steer: 'And be brief', steerId: 's2' }, + ] as TMessage['content']; + const message: Partial = { messageId: 'assistant-2', content, tokenCount: 900 }; + + expect(withoutMessageFiles(message)).toEqual({ + messageId: 'assistant-2', + content: [ + { type: ContentTypes.TEXT, text: 'Working on it.' }, + { type: ContentTypes.STEER, steer: 'Use this one instead', steerId: 's1' }, + { type: ContentTypes.STEER, steer: 'And be brief', steerId: 's2' }, + ], + }); + expect(content?.[1]).toHaveProperty('files', [upload]); + }); + + it('drops content parts that are themselves stored files', () => { + const content = [ + { type: ContentTypes.TEXT, text: 'Here is the image.' }, + { type: ContentTypes.IMAGE_FILE, image_file: { file_id: 'generated-1' } }, + { type: 'file', file: { file_id: 'provider-file-1' } }, + { type: 'input_file', file_id: 'direct-1' }, + ] as unknown as TMessage['content']; + const message: Partial = { messageId: 'assistant-4', content, tokenCount: 30 }; + + expect(withoutMessageFiles(message)).toEqual({ + messageId: 'assistant-4', + content: [{ type: ContentTypes.TEXT, text: 'Here is the image.' }], + }); + }); + + it('keeps metadata-only attachments such as search sources', () => { + const sources = { type: 'web_search', toolCallId: 'call-2', messageId: 'a' } as TAttachment; + const message: Partial = { messageId: 'assistant-3', attachments: [output, sources] }; + + expect(withoutMessageFiles(message)).toEqual({ + messageId: 'assistant-3', + attachments: [sources], + }); + }); + + it('treats an empty upload list as no uploads', () => { + expect(withoutMessageFiles({ messageId: 'user-2', files: [], tokenCount: 7 })).toEqual({ + messageId: 'user-2', + tokenCount: 7, + }); + }); + + it('leaves the source message untouched', () => { + const message: Partial = { messageId: 'user-3', files: [upload], tokenCount: 9 }; + withoutMessageFiles(message); + expect(message).toEqual({ messageId: 'user-3', files: [upload], tokenCount: 9 }); + }); +}); + +describe('withoutConversationFiles', () => { + it('drops conversation file ids and keeps the settings', () => { + const conversation: Partial & { files?: string[] } = { + conversationId: 'convo-1', + title: 'Screenshots', + model: 'gpt-4o', + files: ['upload-1'], + file_ids: ['legacy-1'], + }; + + expect(withoutConversationFiles(conversation)).toEqual({ + conversationId: 'convo-1', + title: 'Screenshots', + model: 'gpt-4o', + }); + }); +}); + +describe('forkFileScope', () => { + const message: Partial = { messageId: 'user-1', files: [upload], tokenCount: 3 }; + const conversation = { conversationId: 'convo-1', files: ['upload-1'] }; + + it('copies files unless exclusion is asked for', () => { + const scope = forkFileScope(); + expect(scope.message(message)).toBe(message); + expect(scope.conversation(conversation)).toBe(conversation); + }); + + it('leaves every file reference out when excluding', () => { + const scope = forkFileScope(true); + expect(scope.message(message)).toEqual({ messageId: 'user-1' }); + expect(scope.conversation(conversation)).toEqual({ conversationId: 'convo-1' }); + }); +}); diff --git a/packages/api/src/conversations/files.ts b/packages/api/src/conversations/files.ts new file mode 100644 index 0000000000..f615bf4e7b --- /dev/null +++ b/packages/api/src/conversations/files.ts @@ -0,0 +1,89 @@ +import { isFileAttachment, isFileContentPart, hasContentPartFiles } from 'librechat-data-provider'; +import type { TConversation, TMessage } from 'librechat-data-provider'; + +type MessageFileRefs = Partial>; + +type ConversationFileRefs = Partial> & { + files?: string[] | null; +}; + +type ContentPart = NonNullable[number]; + +export type MessageWithoutFiles = Omit< + T, + 'files' | 'attachments' | 'tokenCount' +> & + Pick; + +/** A part's own attached files (a steer's) go; a part that is itself a file goes entirely. */ +function withoutContentFiles(content: ContentPart[]): ContentPart[] { + let changed = false; + const kept: ContentPart[] = []; + for (const part of content) { + if (isFileContentPart(part)) { + changed = true; + continue; + } + if (!hasContentPartFiles(part)) { + kept.push(part); + continue; + } + changed = true; + const { files: _files, ...rest } = part as ContentPart & { files?: readonly object[] }; + kept.push(rest as ContentPart); + } + return changed ? kept : content; +} + +/** + * Drops every file reference a later turn collects from history and sends again, in the shapes + * historical replay reads (`messageCarriesFiles`): uploads on a user turn (`files`), file-backed + * attachments such as code outputs (metadata-only ones like search sources stay), and file + * references in the content (a steered attachment, a generated image, a provider file block). A + * stored token count that covered a dropped file goes too, so the next turn recounts what is left + * instead of budgeting context for files it no longer sends. + */ +export function withoutMessageFiles(message: T): MessageWithoutFiles { + const { files, attachments, tokenCount, ...rest } = message; + let droppedFiles = (files?.length ?? 0) > 0; + const keptAttachments = attachments?.filter((attachment) => !isFileAttachment(attachment)); + if (keptAttachments != null && keptAttachments.length > 0) { + (rest as MessageFileRefs).attachments = keptAttachments; + } + if (Array.isArray(rest.content)) { + const content = withoutContentFiles(rest.content); + if (content !== rest.content) { + droppedFiles = true; + rest.content = content; + } + } + if (droppedFiles || tokenCount == null) { + return rest; + } + return { ...rest, tokenCount }; +} + +/** Drops the conversation-level file ids that turns without a branch anchor fall back to. */ +export function withoutConversationFiles( + conversation: T, +): Omit { + const { files: _files, file_ids: _fileIds, ...rest } = conversation; + return rest; +} + +/** + * How a copied conversation treats files: as they are, or with every reference a later turn would + * send again left out (`excludeFiles`). Native copies (forks) apply it to each cloned message and + * to the conversation, so the decision lives here rather than in each copy path. + */ +export function forkFileScope(excludeFiles = false): { + message: (message: T) => T | MessageWithoutFiles; + conversation: ( + conversation: T, + ) => T | Omit; +} { + if (excludeFiles !== true) { + return { message: (message) => message, conversation: (conversation) => conversation }; + } + return { message: withoutMessageFiles, conversation: withoutConversationFiles }; +} diff --git a/packages/api/src/conversations/index.ts b/packages/api/src/conversations/index.ts index 30688b865c..4b56f82e8d 100644 --- a/packages/api/src/conversations/index.ts +++ b/packages/api/src/conversations/index.ts @@ -1,6 +1,7 @@ export * from './announce'; export * from './archive'; export * from './chatgpt'; +export * from './files'; export * from './filters'; export * from './import'; export * from './lineage'; diff --git a/packages/api/src/files/encode/document.spec.ts b/packages/api/src/files/encode/document.spec.ts index f58d591ed6..c73423a3ae 100644 --- a/packages/api/src/files/encode/document.spec.ts +++ b/packages/api/src/files/encode/document.spec.ts @@ -1,8 +1,9 @@ import { Providers } from '@librechat/agents'; import { mbToBytes } from 'librechat-data-provider'; import type { AppConfig, IMongoFile } from '@librechat/data-schemas'; -import type { ServerRequest } from '~/types'; -import { encodeAndFormatDocuments } from './document'; +import type { BedrockDocumentBlock, ServerRequest } from '~/types'; +import type { PayloadContentPart } from './document'; +import { encodeAndFormatDocuments, dedupeDocumentNames } from './document'; /** Mock the validation module */ jest.mock('~/files/validation', () => ({ @@ -1203,3 +1204,95 @@ describe('encodeAndFormatDocuments - fileConfig integration', () => { }); }); }); + +describe('dedupeDocumentNames', () => { + const bedrock = (name: string): BedrockDocumentBlock => ({ + type: 'document', + document: { name, format: 'pdf', source: { bytes: Buffer.from(name) } }, + }); + type Message = { content?: string | PayloadContentPart[] | null }; + const names = (parts: PayloadContentPart[] = []): string[] => + parts.flatMap((part) => + part.document?.name != null ? [part.document.name] : names(part.media ?? []), + ); + const contentOf = (message: Message) => (Array.isArray(message.content) ? message.content : []); + + it('renames repeated Bedrock document names across the payload in order', () => { + const first = bedrock('Stripe fees'); + const messages: Message[] = [ + { content: [{ type: 'text' }, first, bedrock('invoice')] }, + { content: 'A plain answer' }, + { content: [bedrock('Stripe fees'), bedrock('STRIPE FEES')] }, + { content: [{ type: 'text' }, bedrock('Stripe fees')] }, + ]; + + dedupeDocumentNames(messages); + + expect(messages.map((message) => names(contentOf(message)))).toEqual([ + ['Stripe fees', 'invoice'], + [], + ['Stripe fees (2)', 'STRIPE FEES (3)'], + ['Stripe fees (4)'], + ]); + /** The first occurrence keeps its own block object and its message keeps its array. */ + expect(contentOf(messages[0])[1]).toBe(first); + }); + + it('includes documents inside replayed steer media', () => { + const steer: PayloadContentPart = { + type: 'steer', + media: [{ type: 'text' }, bedrock('notes')], + }; + const messages: Message[] = [ + { content: [bedrock('notes')] }, + { content: [{ type: 'text' }, steer] }, + ]; + + dedupeDocumentNames(messages); + + expect(names(contentOf(messages[1]))).toEqual(['notes (2)']); + expect(names(steer.media)).toEqual(['notes']); + }); + + it('does not mutate the renamed source blocks or arrays', () => { + const repeated = bedrock('report'); + const secondContent: PayloadContentPart[] = [repeated]; + const messages: Message[] = [{ content: [bedrock('report')] }, { content: secondContent }]; + + dedupeDocumentNames(messages); + + expect(repeated.document.name).toBe('report'); + expect(secondContent[0]).toBe(repeated); + expect(names(contentOf(messages[1]))).toEqual(['report (2)']); + }); + + it('skips past a name a later upload already uses and keeps names within the limit', () => { + const long = 'a'.repeat(199) + ' '; + const messages: Message[] = [ + { content: [bedrock('notes'), bedrock('notes (2)')] }, + { content: [bedrock('notes'), bedrock(long), bedrock(long)] }, + ]; + + dedupeDocumentNames(messages); + + const second = names(contentOf(messages[1])); + expect(names(contentOf(messages[0]))).toEqual(['notes', 'notes (2)']); + expect(second[0]).toBe('notes (3)'); + expect(second[2]).toBe(`${'a'.repeat(196)} (2)`); + expect(second[2].length).toBeLessThanOrEqual(200); + }); + + it('leaves blocks without a Bedrock document name alone', () => { + const anthropic = { + type: 'document', + title: 'report.pdf', + source: { type: 'base64', media_type: 'application/pdf', data: 'x' }, + } as PayloadContentPart; + const content = [anthropic, { ...anthropic }]; + const messages: Message[] = [{ content }]; + + dedupeDocumentNames(messages); + + expect(messages[0].content).toBe(content); + }); +}); diff --git a/packages/api/src/files/encode/document.ts b/packages/api/src/files/encode/document.ts index 57ef72db9a..7fa7323499 100644 --- a/packages/api/src/files/encode/document.ts +++ b/packages/api/src/files/encode/document.ts @@ -23,6 +23,90 @@ import { import { validatePdf, validateBedrockDocument } from '~/files/validation'; import { runGuardedEncode } from './memoryGuard'; +/** The longest document name LibreChat sends to Bedrock. */ +const BEDROCK_DOCUMENT_NAME_LIMIT = 200; + +/** A content part of a formatted message, as far as document naming needs to read it. */ +export interface PayloadContentPart { + type?: string; + document?: { name?: string }; + /** A replayed steer's own content, stamped for the formatter (`stampSteerPartMedia`). */ + media?: PayloadContentPart[]; +} + +/** `name (n)`, trimmed so the suffix fits the limit and never follows a space. */ +function numberedDocumentName(name: string, index: number): string { + const suffix = ` (${index})`; + return `${name.slice(0, BEDROCK_DOCUMENT_NAME_LIMIT - suffix.length).trimEnd()}${suffix}`; +} + +/** Hands out each name once, numbering repeats in the order they are asked for. */ +function createDocumentNamer(): (name: string) => string { + const used = new Set(); + const nextIndex = new Map(); + return (original) => { + const baseKey = original.toLowerCase(); + let name = original; + let index = nextIndex.get(baseKey) ?? 2; + while (used.has(name.toLowerCase())) { + name = numberedDocumentName(original, index); + index++; + } + nextIndex.set(baseKey, index); + used.add(name.toLowerCase()); + return name; + }; +} + +/** The parts with every Bedrock document renamed by `nameOf`; the same array when none changed. */ +function renameDocumentParts( + parts: PayloadContentPart[], + nameOf: (name: string) => string, +): PayloadContentPart[] { + let renamed: PayloadContentPart[] | undefined; + for (let i = 0; i < parts.length; i++) { + const part = parts[i]; + let next = part; + const original = part?.type === 'document' ? part.document?.name : undefined; + if (typeof original === 'string') { + const name = nameOf(original); + next = name === original ? part : { ...part, document: { ...part.document, name } }; + } else if (Array.isArray(part?.media)) { + const media = renameDocumentParts(part.media, nameOf); + next = media === part.media ? part : { ...part, media }; + } + if (next !== part) { + renamed ??= [...parts]; + renamed[i] = next; + } + } + return renamed ?? parts; +} + +/** + * Bedrock rejects a request in which two documents share a name, and every turn resends the + * documents of earlier turns (and of steers replayed with their attachments), so two uploads with + * the same filename would fail every later turn of the conversation. Renames each repeat across the + * formatted payload in conversation order (`report`, `report (2)`), leaving the first occurrence, + * every other kind of part, and the parts themselves untouched: a message whose documents change + * gets a new content array. Names compare case-insensitively, as the restriction is not + * documented to be case-sensitive. + */ +export function dedupeDocumentNames( + messages: Array<{ content?: string | PayloadContentPart[] | null }>, +): void { + const nameOf = createDocumentNamer(); + for (const message of messages) { + if (!Array.isArray(message?.content)) { + continue; + } + const content = renameDocumentParts(message.content, nameOf); + if (content !== message.content) { + message.content = content; + } + } +} + /** Anthropic only accepts PDFs as base64 documents; textual types must use a text source */ function getAnthropicDocumentSource( mimeType: string, @@ -271,7 +355,7 @@ export async function encodeAndFormatDocuments( const sanitizedName = (file.filename || 'document') .replace(/[^a-zA-Z0-9\s\-()[\]]/g, '_') - .slice(0, 200); + .slice(0, BEDROCK_DOCUMENT_NAME_LIMIT); result.documents.push({ type: 'document', document: { diff --git a/packages/data-provider/src/messages.spec.ts b/packages/data-provider/src/messages.spec.ts index d453f6b856..96916c1759 100644 --- a/packages/data-provider/src/messages.spec.ts +++ b/packages/data-provider/src/messages.spec.ts @@ -2,7 +2,16 @@ import type { SummaryContentPart } from './types/content'; import type { ParentMessage } from './messages'; import type { TFile } from './types/files'; import type { TMessage } from './types'; -import { buildTree, findMessageById, isCompactedLeaf, isUserInitiatedCompaction } from './messages'; +import { + buildTree, + findMessageById, + isCompactedLeaf, + isFileAttachment, + isFileContentPart, + messageCarriesFiles, + hasContentPartFiles, + isUserInitiatedCompaction, +} from './messages'; import { ContentTypes } from './types/runs'; const msg = (messageId: string, parentMessageId: string, over: Partial = {}): TMessage => @@ -287,3 +296,59 @@ describe('isUserInitiatedCompaction', () => { expect(isUserInitiatedCompaction({ content } as TMessage)).toBe(false); }); }); + +type Content = NonNullable; + +describe('messageCarriesFiles', () => { + it('finds uploads, file-backed attachments and file references in content', () => { + expect(messageCarriesFiles({ files: [{ file_id: 'upload' }] })).toBe(true); + expect( + messageCarriesFiles({ + attachments: [{ file_id: 'chart', toolCallId: 't' }] as TMessage['attachments'], + }), + ).toBe(true); + expect( + messageCarriesFiles({ + content: [ + { type: ContentTypes.STEER, steer: 'use this', files: [{ file_id: 'steered' }] }, + ] as Content, + }), + ).toBe(true); + expect( + messageCarriesFiles({ + content: [{ type: ContentTypes.IMAGE_FILE, image_file: { file_id: 'img' } }] as Content, + }), + ).toBe(true); + }); + + it('ignores empty lists, search sources and plain content', () => { + expect(messageCarriesFiles({ files: [] })).toBe(false); + expect( + messageCarriesFiles({ + attachments: [{ type: 'web_search', toolCallId: 't' }] as TMessage['attachments'], + }), + ).toBe(false); + expect( + messageCarriesFiles({ + content: [ + { type: ContentTypes.TEXT, text: 'hello' }, + { type: ContentTypes.STEER, steer: 'and this', files: [] }, + ] as Content, + }), + ).toBe(false); + }); +}); + +describe('file reference predicates', () => { + it('recognizes each locator shape replay reads', () => { + /** Provider-native locators that persisted rows carry outside the typed content union. */ + expect(isFileContentPart({ file: { file_id: 'f' } })).toBe(true); + expect(isFileContentPart({ file_id: 'f' })).toBe(true); + expect(isFileContentPart({ type: ContentTypes.TEXT, text: 'x' } as Content[number])).toBe( + false, + ); + expect(isFileContentPart(undefined)).toBe(false); + expect(hasContentPartFiles(null)).toBe(false); + expect(isFileAttachment(undefined)).toBe(false); + }); +}); diff --git a/packages/data-provider/src/messages.ts b/packages/data-provider/src/messages.ts index b424f64890..2f6f5ff03d 100644 --- a/packages/data-provider/src/messages.ts +++ b/packages/data-provider/src/messages.ts @@ -240,3 +240,59 @@ export function isCompactedLeaf(message?: Pick | null): boo } return usable; } + +/** The shapes a persisted content part references a stored file by, as historical replay reads + * them: attached `files`, an `image_file` or `file` locator, or a direct `file_id`. */ +export type FileLocatorPart = { + files?: readonly object[] | null; + image_file?: { file_id?: unknown } | null; + file?: { file_id?: unknown } | null; + file_id?: unknown; +}; + +const isFileId = (value: unknown): boolean => typeof value === 'string' && value !== ''; + +/** Whether a content part is itself a stored file (a generated image, a provider file block). */ +export function isFileContentPart( + part: TMessageContentParts | FileLocatorPart | null | undefined, +): boolean { + if (part == null) { + return false; + } + const locator = part as FileLocatorPart; + return ( + isFileId(locator.image_file?.file_id) || + isFileId(locator.file?.file_id) || + isFileId(locator.file_id) + ); +} + +/** Whether a content part carries attached files of its own, as a steer part can. */ +export function hasContentPartFiles( + part: TMessageContentParts | FileLocatorPart | null | undefined, +): boolean { + const files = (part as FileLocatorPart | null | undefined)?.files; + return Array.isArray(files) && files.length > 0; +} + +/** Whether an attachment is a stored file (a code output) rather than tool metadata such as + * search sources, which carry no `file_id`. */ +export function isFileAttachment( + attachment: NonNullable[number] | null | undefined, +): boolean { + return attachment != null && 'file_id' in attachment && isFileId(attachment.file_id); +} + +/** + * Whether a message references any stored file a later turn would collect from history and send + * again: uploads, file-backed attachments, and file references inside its content. + */ +export function messageCarriesFiles( + message: Pick, +): boolean { + return ( + (message.files?.length ?? 0) > 0 || + (message.attachments?.some(isFileAttachment) ?? false) || + (message.content?.some((part) => isFileContentPart(part) || hasContentPartFiles(part)) ?? false) + ); +} diff --git a/packages/data-provider/src/types.ts b/packages/data-provider/src/types.ts index c967d54ac2..c2fc8391ca 100644 --- a/packages/data-provider/src/types.ts +++ b/packages/data-provider/src/types.ts @@ -577,6 +577,8 @@ export type TForkConvoRequest = { option?: string; splitAtTarget?: boolean; latestMessageId?: string; + /** Copy without uploads or code outputs, and without the conversation's file list. */ + excludeFiles?: boolean; }; export type TForkConvoResponse = {