mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-08-27 04:07:05 +00:00
🤐 feat: Allow Promptless Sends When Files Are Attached (#13717)
* ✨ feat: Allow sending file attachments without a text message When an agent asks the user to upload a document, the user could attach the file but still had to type a placeholder message ("OK", "Here is the file") before the send button enabled and the submit guard let the message through. Attachments now count as submittable content: - New isSubmittableMessage(text, fileCount) util: non-whitespace text OR at least one attached file. - ask() in useChatFunctions uses it instead of bailing on empty text, so an empty draft with attached files submits. - SendButton receives the attached file count and enables accordingly. - ChatForm only marks the text field as required when no files are attached, so react-hook-form validation no longer blocks handleSubmit. Submitting an empty draft with no attachments is still rejected at all three layers. Fixes #13646 * Address review: support replayed file-only turns + drop empty vision text - ask(): count replayed attachments (overrideFiles) in the submittable check and skip it entirely for regenerate, so a file-only message can be regenerated or saved-and-resubmitted instead of being rejected as empty. - formatVisionMessage(): omit the text content part when the message text is empty. Anthropic rejects empty text content blocks with HTTP 400, and an empty block adds nothing for other providers; image-only sends now format cleanly. Added formatMessages tests for with-text and image-only (Anthropic + other) cases. * Address review: keep attachment-only turns valid for providers, answer mode, and titles - formatMessage: substitute minimal text when a user turn carries files but no inline content, so Anthropic does not reject an empty user message for RAG or code-environment attachments. - assistants chatV1: send the same stand-in for attachment-only Threads messages, which reject an empty body. The persisted message keeps empty text. - ChatForm: attachments no longer make an empty draft submittable in answer mode, where submitText consumes the click without answering or sending. - agents request: seed title generation from attachment filenames when the turn has no text, so immediate-mode titles are not invented from an empty string. - useChatFunctions.regenerate.spec: mock the utils barrel over the real module so new exports resolve. * Cover the agents path for attachment-only turns AgentClient formats its payload with the SDK's formatMessage, not the local one, so the earlier guard missed the endpoint the feature actually targets: an attachment-only turn still reached Anthropic as an empty user message. Apply the same stand-in after the file-context and quote merges, so a turn that already gained inline content is untouched. * Carry filenames on freshly attached files The fresh-file submission mapping copied only file_id, filepath, type, and dimensions, so the attachment-only title fallback read an undefined filename and produced nothing. Include filename, and cover it with a test that submits an empty draft with one attachment. * Address review: cover assistants v2, fresh agent attachments, editor, and title fallback - agents client: the current turn has no files during buildMessages, so read the resolved attachments from message_file_map instead. The previous guard only ever fired for persisted historical turns. - assistants chatV2: the default assistants endpoint routes here, so it needs the same stand-in body chatV1 got. - assistants title: fall back to filenames, then the response, and keep the default title rather than saving an empty one. - EditMessage: retained attachments make an empty edit submittable, matching the composer, so the overrideFiles replay path is reachable from the UI. --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>
This commit is contained in:
Anubhav Anand
GitHub
parent
e1ac7d2bda
commit
a2ad0aa0c8
18 changed files with 375 additions and 16 deletions
|
|
@ -1,3 +1,4 @@
|
|||
const { ATTACHMENT_ONLY_TEXT } = require('@librechat/api');
|
||||
const { EModelEndpoint, ContentTypes } = require('librechat-data-provider');
|
||||
const {
|
||||
AIMessage,
|
||||
|
|
@ -18,12 +19,18 @@ const {
|
|||
* @returns {(Object)} - The formatted message.
|
||||
*/
|
||||
const formatVisionMessage = ({ message, image_urls, endpoint }) => {
|
||||
// Omit an empty text part for image-only messages. Anthropic rejects empty
|
||||
// text content blocks with HTTP 400, and an empty block adds nothing for
|
||||
// other providers either.
|
||||
const hasText = typeof message.content === 'string' && message.content.trim() !== '';
|
||||
const textPart = hasText ? [{ type: ContentTypes.TEXT, text: message.content }] : [];
|
||||
|
||||
if (endpoint === EModelEndpoint.anthropic) {
|
||||
message.content = [...image_urls, { type: ContentTypes.TEXT, text: message.content }];
|
||||
message.content = [...image_urls, ...textPart];
|
||||
return message;
|
||||
}
|
||||
|
||||
message.content = [{ type: ContentTypes.TEXT, text: message.content }, ...image_urls];
|
||||
message.content = [...textPart, ...image_urls];
|
||||
|
||||
return message;
|
||||
};
|
||||
|
|
@ -71,6 +78,15 @@ const formatMessage = ({ message, userName, assistantName, endpoint, langChain =
|
|||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* An attachment-only turn whose files reach the model out-of-band (RAG,
|
||||
* code environment) leaves nothing in the content itself, and providers
|
||||
* such as Anthropic reject an empty user message outright.
|
||||
*/
|
||||
if (role === 'user' && content === '' && message.files?.length > 0) {
|
||||
formattedMessage.content = ATTACHMENT_ONLY_TEXT;
|
||||
}
|
||||
|
||||
if (_name) {
|
||||
formattedMessage.name = _name;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
const { ATTACHMENT_ONLY_TEXT } = require('@librechat/api');
|
||||
const { Constants } = require('librechat-data-provider');
|
||||
const { HumanMessage, AIMessage, SystemMessage } = require('@librechat/agents/langchain/messages');
|
||||
const { formatMessage, formatLangChainMessages, formatFromLangChain } = require('./formatMessages');
|
||||
|
|
@ -184,6 +185,63 @@ describe('formatMessage', () => {
|
|||
content: 'Hello',
|
||||
});
|
||||
});
|
||||
|
||||
it('includes the text part for vision messages that have text', () => {
|
||||
const image = { type: 'image_url', image_url: { url: 'data:image/png;base64,abc' } };
|
||||
const result = formatMessage({
|
||||
message: { role: 'user', text: 'Describe this', image_urls: [image] },
|
||||
endpoint: 'anthropic',
|
||||
});
|
||||
expect(result.content).toEqual([image, { type: 'text', text: 'Describe this' }]);
|
||||
});
|
||||
|
||||
it('omits the empty text part for image-only Anthropic messages', () => {
|
||||
const image = { type: 'image_url', image_url: { url: 'data:image/png;base64,abc' } };
|
||||
const result = formatMessage({
|
||||
message: { role: 'user', text: '', image_urls: [image] },
|
||||
endpoint: 'anthropic',
|
||||
});
|
||||
// No empty { type: 'text', text: '' } block; Anthropic rejects those with HTTP 400.
|
||||
expect(result.content).toEqual([image]);
|
||||
});
|
||||
|
||||
it('omits the empty text part for image-only messages on other endpoints', () => {
|
||||
const image = { type: 'image_url', image_url: { url: 'data:image/png;base64,abc' } };
|
||||
const result = formatMessage({
|
||||
message: { role: 'user', text: ' ', image_urls: [image] },
|
||||
endpoint: 'openAI',
|
||||
});
|
||||
expect(result.content).toEqual([image]);
|
||||
});
|
||||
|
||||
it('substitutes text for an attachment-only turn with no inline content', () => {
|
||||
const result = formatMessage({
|
||||
message: { role: 'user', text: '', files: [{ file_id: 'f1', embedded: true }] },
|
||||
endpoint: 'anthropic',
|
||||
});
|
||||
expect(result.content).toBe(ATTACHMENT_ONLY_TEXT);
|
||||
});
|
||||
|
||||
it('keeps the user text when an attachment-only turn also has text', () => {
|
||||
const result = formatMessage({
|
||||
message: { role: 'user', text: 'Summarize it', files: [{ file_id: 'f1', embedded: true }] },
|
||||
endpoint: 'anthropic',
|
||||
});
|
||||
expect(result.content).toBe('Summarize it');
|
||||
});
|
||||
|
||||
it('leaves empty content alone when the turn carries no files', () => {
|
||||
const result = formatMessage({ message: { role: 'user', text: '' }, endpoint: 'anthropic' });
|
||||
expect(result.content).toBe('');
|
||||
});
|
||||
|
||||
it('does not substitute text for an assistant turn', () => {
|
||||
const result = formatMessage({
|
||||
message: { role: 'assistant', text: '', files: [{ file_id: 'f1' }] },
|
||||
endpoint: 'anthropic',
|
||||
});
|
||||
expect(result.content).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('formatLangChainMessages', () => {
|
||||
|
|
|
|||
|
|
@ -78,6 +78,7 @@ const {
|
|||
countFormattedMessageTokens,
|
||||
prependFileContext,
|
||||
prependQuotes,
|
||||
applyAttachmentOnlyText,
|
||||
hydrateMissingIndexTokenCounts,
|
||||
injectSkillPrimes,
|
||||
collectFreshSkillPrimeNames,
|
||||
|
|
@ -1368,6 +1369,19 @@ class AgentClient extends BaseClient {
|
|||
prependQuotes(memoryFormattedMessage, message.quotes);
|
||||
}
|
||||
|
||||
/**
|
||||
* An attachment-only turn whose files reach the model out-of-band (file
|
||||
* search, code environment) leaves nothing in the content itself, and
|
||||
* providers such as Anthropic reject an empty user message outright.
|
||||
* Applied after the context and quote merges so a turn that already
|
||||
* gained inline content keeps it. The current turn is not carrying
|
||||
* `files` yet (BaseClient assigns them after this returns), so the
|
||||
* resolved attachments come from `message_file_map`.
|
||||
*/
|
||||
const turnFiles = this.message_file_map?.[message.messageId] ?? message.files;
|
||||
applyAttachmentOnlyText(formattedMessage, turnFiles);
|
||||
applyAttachmentOnlyText(memoryFormattedMessage, turnFiles);
|
||||
|
||||
memoryPayload.push(memoryFormattedMessage);
|
||||
|
||||
const dbTokenCount = Number(orderedMessages[i].tokenCount);
|
||||
|
|
|
|||
|
|
@ -25,6 +25,7 @@ const {
|
|||
isSteerPreemptSupported,
|
||||
buildRecoveredSteerPayload,
|
||||
deleteAgentCheckpoint,
|
||||
getAttachmentTitleText,
|
||||
} = require('@librechat/api');
|
||||
const { disposeClient } = require('~/server/cleanup');
|
||||
const {
|
||||
|
|
@ -1325,7 +1326,7 @@ const ResumableAgentController = async (req, res, next, initializeClient, addTit
|
|||
|
||||
if (titleEligible && titleTiming === 'immediate') {
|
||||
immediateTitlePromise = addTitle(req, {
|
||||
text,
|
||||
text: text || getAttachmentTitleText(req.body.files),
|
||||
conversationId,
|
||||
client,
|
||||
immediate: true,
|
||||
|
|
@ -1712,7 +1713,7 @@ const ResumableAgentController = async (req, res, next, initializeClient, addTit
|
|||
}
|
||||
} else if (shouldGenerateTitle) {
|
||||
addTitle(req, {
|
||||
text,
|
||||
text: text || getAttachmentTitleText(req.body.files),
|
||||
response: { ...response },
|
||||
client,
|
||||
})
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ const {
|
|||
checkBalance,
|
||||
getBalanceConfig,
|
||||
getModelMaxTokens,
|
||||
ATTACHMENT_ONLY_TEXT,
|
||||
} = require('@librechat/api');
|
||||
const {
|
||||
Time,
|
||||
|
|
@ -321,9 +322,14 @@ const chatV1 = async (req, res) => {
|
|||
parentMessageId = previousMessages[previousMessages.length - 1].messageId;
|
||||
}
|
||||
|
||||
/**
|
||||
* Threads rejects an empty message body, so an attachment-only turn sends
|
||||
* a minimal note instead. The persisted message keeps its empty text.
|
||||
*/
|
||||
const isAttachmentOnly = !text?.trim() && files.length > 0;
|
||||
let userMessage = {
|
||||
role: 'user',
|
||||
content: text,
|
||||
content: isAttachmentOnly ? ATTACHMENT_ONLY_TEXT : text,
|
||||
metadata: {
|
||||
messageId: userMessageId,
|
||||
},
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ const {
|
|||
checkBalance,
|
||||
getBalanceConfig,
|
||||
getModelMaxTokens,
|
||||
ATTACHMENT_ONLY_TEXT,
|
||||
} = require('@librechat/api');
|
||||
const {
|
||||
Time,
|
||||
|
|
@ -194,12 +195,17 @@ const chatV2 = async (req, res) => {
|
|||
parentMessageId = previousMessages[previousMessages.length - 1].messageId;
|
||||
}
|
||||
|
||||
/**
|
||||
* Threads rejects an empty message body, so an attachment-only turn sends
|
||||
* a minimal note instead. The persisted message keeps its empty text.
|
||||
*/
|
||||
const isAttachmentOnly = !text?.trim() && files.length > 0;
|
||||
let userMessage = {
|
||||
role: 'user',
|
||||
content: [
|
||||
{
|
||||
type: ContentTypes.TEXT,
|
||||
text,
|
||||
text: isAttachmentOnly ? ATTACHMENT_ONLY_TEXT : text,
|
||||
},
|
||||
],
|
||||
metadata: {
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
const { isEnabled, sanitizeTitle } = require('@librechat/api');
|
||||
const { isEnabled, sanitizeTitle, getAttachmentTitleText } = require('@librechat/api');
|
||||
const { logger } = require('@librechat/data-schemas');
|
||||
const { CacheKeys } = require('librechat-data-provider');
|
||||
const getLogStores = require('~/cache/getLogStores');
|
||||
|
|
@ -78,7 +78,18 @@ const addTitle = async (req, { text, responseText, conversationId }) => {
|
|||
);
|
||||
} catch (error) {
|
||||
logger.error('[addTitle] Error generating title:', error);
|
||||
const fallbackTitle = text.length > 40 ? text.substring(0, 37) + '...' : text;
|
||||
/**
|
||||
* An attachment-only turn has no text to fall back on, and saving the
|
||||
* empty string would replace the conversation's default title with a
|
||||
* blank sidebar entry. Use the filenames, then the response, and leave
|
||||
* the default in place when neither says anything.
|
||||
*/
|
||||
const fallbackSource = text || getAttachmentTitleText(req?.body?.files) || responseText || '';
|
||||
if (!fallbackSource) {
|
||||
return;
|
||||
}
|
||||
const fallbackTitle =
|
||||
fallbackSource.length > 40 ? fallbackSource.substring(0, 37) + '...' : fallbackSource;
|
||||
await titleCache.set(key, fallbackTitle, 120000);
|
||||
await saveConvo(
|
||||
{
|
||||
|
|
|
|||
|
|
@ -397,8 +397,14 @@ const ChatForm = memo(function ChatForm({
|
|||
|
||||
useQueryParams({ textAreaRef });
|
||||
|
||||
/** Attachments stand in for text only on the normal send path. Answer mode
|
||||
* hands the composer text straight to the paused run, which answers with
|
||||
* values and cannot consume files, so an empty draft must stay unsubmittable
|
||||
* there rather than enabling a button whose submit is silently dropped. */
|
||||
const submittableFileCount = answerMode.active ? 0 : files.size;
|
||||
|
||||
const { ref, ...registerProps } = methods.register('text', {
|
||||
required: true,
|
||||
required: submittableFileCount === 0,
|
||||
onChange: useCallback(
|
||||
(e: React.ChangeEvent<HTMLTextAreaElement>) =>
|
||||
methods.setValue('text', e.target.value, { shouldValidate: true }),
|
||||
|
|
@ -699,6 +705,7 @@ const ChatForm = memo(function ChatForm({
|
|||
<SendButton
|
||||
ref={submitButtonRef}
|
||||
control={methods.control}
|
||||
fileCount={submittableFileCount}
|
||||
disabled={
|
||||
filesLoading ||
|
||||
disableInputs ||
|
||||
|
|
|
|||
|
|
@ -2,12 +2,14 @@ import React, { forwardRef } from 'react';
|
|||
import { useWatch } from 'react-hook-form';
|
||||
import { SendIcon, TooltipAnchor } from '@librechat/client';
|
||||
import type { Control } from 'react-hook-form';
|
||||
import { cn, isSubmittableMessage } from '~/utils';
|
||||
import { useLocalize } from '~/hooks';
|
||||
import { cn } from '~/utils';
|
||||
|
||||
type SendButtonProps = {
|
||||
disabled: boolean;
|
||||
control: Control<{ text: string }>;
|
||||
/** Number of attached files; attachments allow sending without text */
|
||||
fileCount?: number;
|
||||
};
|
||||
|
||||
const SubmitButton = React.memo(
|
||||
|
|
@ -41,8 +43,8 @@ const SubmitButton = React.memo(
|
|||
const SendButton = React.memo(
|
||||
forwardRef((props: SendButtonProps, ref: React.ForwardedRef<HTMLButtonElement>) => {
|
||||
const data = useWatch({ control: props.control });
|
||||
const content = data?.text?.trim();
|
||||
return <SubmitButton ref={ref} disabled={props.disabled || !content} />;
|
||||
const canSubmit = isSubmittableMessage(data?.text, props.fileCount);
|
||||
return <SubmitButton ref={ref} disabled={props.disabled || !canSubmit} />;
|
||||
}),
|
||||
);
|
||||
|
||||
|
|
|
|||
|
|
@ -185,7 +185,9 @@ const EditMessage = ({
|
|||
);
|
||||
|
||||
const { ref, ...registerProps } = register('text', {
|
||||
required: true,
|
||||
/** Retained attachments make an otherwise empty edit submittable, matching
|
||||
* the composer; `ask` replays them through `overrideFiles`. */
|
||||
required: (message.files?.length ?? 0) === 0,
|
||||
onChange: (e) => {
|
||||
setValue('text', e.target.value, { shouldDirty: true, shouldValidate: true });
|
||||
},
|
||||
|
|
|
|||
|
|
@ -57,6 +57,7 @@ jest.mock('~/store', () => ({
|
|||
useGetEphemeralAgent: () => mockGetEphemeralAgent,
|
||||
}));
|
||||
jest.mock('~/utils', () => ({
|
||||
...jest.requireActual('~/utils'),
|
||||
logger: {
|
||||
log: jest.fn(),
|
||||
dir: jest.fn(),
|
||||
|
|
@ -292,3 +293,52 @@ describe('useChatFunctions regenerate', () => {
|
|||
expect(messages.at(-1)?.messageId).toBe('assistant-1_');
|
||||
});
|
||||
});
|
||||
|
||||
describe('useChatFunctions ask attachments', () => {
|
||||
beforeEach(() => {
|
||||
jest.clearAllMocks();
|
||||
mockGetQueryData.mockReturnValue({});
|
||||
});
|
||||
|
||||
/** The server titles an attachment-only turn from the submitted filenames
|
||||
* (getAttachmentTitleText), so the fresh-file mapping must carry them. */
|
||||
it('carries the filename on freshly attached files', () => {
|
||||
const setMessages = jest.fn();
|
||||
const setSubmission = jest.fn();
|
||||
const setFiles = jest.fn();
|
||||
const files = new Map([
|
||||
[
|
||||
'file-1',
|
||||
{
|
||||
file_id: 'file-1',
|
||||
filepath: '/uploads/file-1',
|
||||
filename: 'quarterly-report.pdf',
|
||||
type: 'application/pdf',
|
||||
},
|
||||
],
|
||||
]) as unknown as Parameters<typeof useChatFunctions>[0]['files'];
|
||||
|
||||
const { result } = renderHook(() =>
|
||||
useChatFunctions({
|
||||
isSubmitting: false,
|
||||
latestMessage: null,
|
||||
conversation: conversation(Constants.NEW_CONVO as string),
|
||||
getMessages: () => [],
|
||||
setMessages,
|
||||
setSubmission,
|
||||
files,
|
||||
setFiles,
|
||||
}),
|
||||
);
|
||||
|
||||
act(() => {
|
||||
result.current.ask({ text: '' });
|
||||
});
|
||||
|
||||
const submission = setSubmission.mock.calls.at(-1)?.[0] as TSubmission;
|
||||
expect(submission.userMessage.files?.[0]).toMatchObject({
|
||||
file_id: 'file-1',
|
||||
filename: 'quarterly-report.pdf',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -30,6 +30,7 @@ import {
|
|||
logger,
|
||||
requestChatFocus,
|
||||
hasStreamStartFailed,
|
||||
isSubmittableMessage,
|
||||
createDualMessageContent,
|
||||
getRouteChatProjectId,
|
||||
} from '~/utils';
|
||||
|
|
@ -292,7 +293,18 @@ export default function useChatFunctions({
|
|||
} = {},
|
||||
) => {
|
||||
text = text.trim();
|
||||
if (!!isSubmitting || text === '') {
|
||||
/**
|
||||
* Attached files make an otherwise empty draft submittable, e.g. replying
|
||||
* to an agent that asked for a document upload. Replayed turns (regenerate,
|
||||
* or save-and-submit carrying `overrideFiles`) reuse stored attachments that
|
||||
* aren't in the compose `files` map, so count those too and never re-block a
|
||||
* regenerate of an already-validated file-only turn.
|
||||
*/
|
||||
const replayFileCount = overrideFiles?.length ?? 0;
|
||||
if (
|
||||
!!isSubmitting ||
|
||||
(!isRegenerate && !isSubmittableMessage(text, (files?.size ?? 0) + replayFileCount))
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
|
||||
|
|
@ -548,6 +560,7 @@ export default function useChatFunctions({
|
|||
currentMsg.files = Array.from(files.values()).map((file) => ({
|
||||
file_id: file.file_id,
|
||||
filepath: file.filepath,
|
||||
filename: file.filename,
|
||||
type: file.type ?? '', // Ensure type is not undefined
|
||||
height: file.height,
|
||||
width: file.width,
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ import {
|
|||
getHeaderPrefixForScreenReader,
|
||||
areMessageFieldsEqual,
|
||||
areMessageRowPropsEqual,
|
||||
isSubmittableMessage,
|
||||
} from '../messages';
|
||||
|
||||
const translations: Record<string, string> = {
|
||||
|
|
@ -344,3 +345,27 @@ describe('areMessageRowPropsEqual', () => {
|
|||
).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('isSubmittableMessage', () => {
|
||||
it('accepts non-whitespace text without files', () => {
|
||||
expect(isSubmittableMessage('Hello')).toBe(true);
|
||||
expect(isSubmittableMessage(' Hello ', 0)).toBe(true);
|
||||
});
|
||||
|
||||
it('rejects an empty draft with no files', () => {
|
||||
expect(isSubmittableMessage('')).toBe(false);
|
||||
expect(isSubmittableMessage(' ')).toBe(false);
|
||||
expect(isSubmittableMessage(undefined)).toBe(false);
|
||||
expect(isSubmittableMessage(null)).toBe(false);
|
||||
});
|
||||
|
||||
it('accepts an empty draft when files are attached', () => {
|
||||
expect(isSubmittableMessage('', 1)).toBe(true);
|
||||
expect(isSubmittableMessage(' ', 2)).toBe(true);
|
||||
expect(isSubmittableMessage(undefined, 1)).toBe(true);
|
||||
});
|
||||
|
||||
it('accepts text alongside files', () => {
|
||||
expect(isSubmittableMessage('Translate this', 1)).toBe(true);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -191,6 +191,14 @@ export const getAllContentText = (message?: TMessage | null): string => {
|
|||
return '';
|
||||
};
|
||||
|
||||
/**
|
||||
* Whether a draft message has enough content to submit: non-whitespace
|
||||
* text, or at least one attached file. Lets users send a file without
|
||||
* having to type a placeholder message alongside it.
|
||||
*/
|
||||
export const isSubmittableMessage = (text?: string | null, fileCount = 0): boolean =>
|
||||
(text ?? '').trim() !== '' || fileCount > 0;
|
||||
|
||||
export const hasStreamStartFailed = (message?: Pick<TMessage, 'metadata'> | null): boolean =>
|
||||
message?.metadata?.[STREAM_START_FAILED_METADATA_KEY] === true;
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,12 @@
|
|||
import { ContentTypes } from 'librechat-data-provider';
|
||||
import { prependFileContext, prependQuotes, type FormattedMessageWithContent } from './client';
|
||||
import type { TMessage } from 'librechat-data-provider';
|
||||
import {
|
||||
prependQuotes,
|
||||
prependFileContext,
|
||||
applyAttachmentOnlyText,
|
||||
type FormattedMessageWithContent,
|
||||
} from './client';
|
||||
import { ATTACHMENT_ONLY_TEXT } from '~/files/context';
|
||||
|
||||
describe('prependFileContext', () => {
|
||||
it('prepends file context to string content', () => {
|
||||
|
|
@ -113,3 +120,52 @@ describe('prependQuotes', () => {
|
|||
expect(message.content).toBe('Explain this.');
|
||||
});
|
||||
});
|
||||
|
||||
describe('applyAttachmentOnlyText', () => {
|
||||
const withFiles = [{ file_id: 'f1' }] as TMessage['files'];
|
||||
|
||||
it('substitutes text for an empty user turn that carries files', () => {
|
||||
const message: FormattedMessageWithContent = { role: 'user', content: '' };
|
||||
|
||||
applyAttachmentOnlyText(message, withFiles);
|
||||
|
||||
expect(message.content).toBe(ATTACHMENT_ONLY_TEXT);
|
||||
});
|
||||
|
||||
it('leaves a user turn that already has text alone', () => {
|
||||
const message: FormattedMessageWithContent = { role: 'user', content: 'Summarize it' };
|
||||
|
||||
applyAttachmentOnlyText(message, withFiles);
|
||||
|
||||
expect(message.content).toBe('Summarize it');
|
||||
});
|
||||
|
||||
it('leaves content that quotes or file context already filled alone', () => {
|
||||
const message: FormattedMessageWithContent = {
|
||||
role: 'user',
|
||||
content: [{ type: ContentTypes.TEXT, text: 'Attached file text' }],
|
||||
};
|
||||
|
||||
applyAttachmentOnlyText(message, withFiles);
|
||||
|
||||
expect(message.content).toEqual([{ type: ContentTypes.TEXT, text: 'Attached file text' }]);
|
||||
});
|
||||
|
||||
it('ignores turns without files', () => {
|
||||
const message: FormattedMessageWithContent = { role: 'user', content: '' };
|
||||
|
||||
applyAttachmentOnlyText(message, []);
|
||||
expect(message.content).toBe('');
|
||||
|
||||
applyAttachmentOnlyText(message, null);
|
||||
expect(message.content).toBe('');
|
||||
});
|
||||
|
||||
it('ignores non-user turns', () => {
|
||||
const message: FormattedMessageWithContent = { role: 'assistant', content: '' };
|
||||
|
||||
applyAttachmentOnlyText(message, withFiles);
|
||||
|
||||
expect(message.content).toBe('');
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ import type { MessageContentComplex } from '@librechat/agents';
|
|||
import type { Agent, TMessage } from 'librechat-data-provider';
|
||||
import type { ServerRequest } from '~/types';
|
||||
import { logAxiosError, mergeQuotedText, formatQuotesAsMarkdown } from '~/utils';
|
||||
import { ATTACHMENT_ONLY_TEXT } from '~/files/context';
|
||||
import Tokenizer from '~/utils/tokenizer';
|
||||
|
||||
export const omitTitleOptions: Set<string> = new Set([
|
||||
|
|
@ -69,9 +70,37 @@ export type FormattedMessageContentPart = {
|
|||
};
|
||||
|
||||
export type FormattedMessageWithContent = {
|
||||
role?: string;
|
||||
content?: string | FormattedMessageContentPart[];
|
||||
};
|
||||
|
||||
/**
|
||||
* Substitutes stand-in text for a user turn that carries attachments but has
|
||||
* nothing the provider can see: file search and code environment files reach
|
||||
* the model out-of-band, so the content stays empty and Anthropic rejects the
|
||||
* message outright. Apply after the file-context and quote merges so a turn
|
||||
* that already gained inline content is left alone. The stored `message.text`
|
||||
* keeps its empty value, so the UI still renders the attachment on its own.
|
||||
*
|
||||
* Takes the turn's files rather than the message because the current turn does
|
||||
* not carry them yet: `BaseClient` assigns `userMessage.files` only after
|
||||
* `buildMessages` returns, so callers pass the resolved attachments instead.
|
||||
*/
|
||||
export function applyAttachmentOnlyText(
|
||||
formattedMessage: FormattedMessageWithContent,
|
||||
files?: TMessage['files'] | null,
|
||||
): void {
|
||||
if (formattedMessage.role !== 'user' || !files?.length) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (formattedMessage.content !== '') {
|
||||
return;
|
||||
}
|
||||
|
||||
formattedMessage.content = ATTACHMENT_ONLY_TEXT;
|
||||
}
|
||||
|
||||
export function prependFileContext(
|
||||
formattedMessage: FormattedMessageWithContent,
|
||||
fileContext?: string | null,
|
||||
|
|
|
|||
30
packages/api/src/files/context.spec.ts
Normal file
30
packages/api/src/files/context.spec.ts
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
import type { TFile } from 'librechat-data-provider';
|
||||
import { getAttachmentTitleText } from './context';
|
||||
|
||||
const file = (filename?: string): TFile => ({ filename }) as TFile;
|
||||
|
||||
describe('getAttachmentTitleText', () => {
|
||||
it('returns an empty string when there are no files', () => {
|
||||
expect(getAttachmentTitleText()).toBe('');
|
||||
expect(getAttachmentTitleText(null)).toBe('');
|
||||
expect(getAttachmentTitleText([])).toBe('');
|
||||
});
|
||||
|
||||
it('lists a single filename', () => {
|
||||
expect(getAttachmentTitleText([file('report.pdf')])).toBe('Attached file(s): report.pdf');
|
||||
});
|
||||
|
||||
it('lists every filename', () => {
|
||||
expect(getAttachmentTitleText([file('a.pdf'), file('b.csv')])).toBe(
|
||||
'Attached file(s): a.pdf, b.csv',
|
||||
);
|
||||
});
|
||||
|
||||
it('skips files that carry no filename', () => {
|
||||
expect(getAttachmentTitleText([file(), file('kept.txt')])).toBe('Attached file(s): kept.txt');
|
||||
});
|
||||
|
||||
it('returns an empty string when no file has a filename', () => {
|
||||
expect(getAttachmentTitleText([file(), file()])).toBe('');
|
||||
});
|
||||
});
|
||||
|
|
@ -1,9 +1,34 @@
|
|||
import { logger } from '@librechat/data-schemas';
|
||||
import { FileSources, mergeFileConfig } from 'librechat-data-provider';
|
||||
import type { IMongoFile } from '@librechat/data-schemas';
|
||||
import type { TFile } from 'librechat-data-provider';
|
||||
import type { TokenCountFn } from '~/utils/text';
|
||||
import type { ServerRequest } from '~/types';
|
||||
import { processTextWithTokenLimit } from '~/utils/text';
|
||||
import type { TokenCountFn } from '~/utils/text';
|
||||
|
||||
/**
|
||||
* Stand-in text for a user turn that carries attachments but no typed message.
|
||||
* Anthropic and the Assistants API both reject empty user content, and files
|
||||
* that reach the model out-of-band (RAG, code environment) leave nothing else
|
||||
* in the turn, so the payload needs this minimal note. The stored message keeps
|
||||
* its empty text so the UI still renders the attachment on its own.
|
||||
*/
|
||||
export const ATTACHMENT_ONLY_TEXT = 'Please refer to the attached file(s).';
|
||||
|
||||
/**
|
||||
* Title-generation input for a turn the user sent without typing anything.
|
||||
* Immediate title timing runs before any response exists, so the attachment
|
||||
* filenames are the only conversation-specific signal available; without them
|
||||
* the title model is prompted with an empty string and invents a topic.
|
||||
*/
|
||||
export function getAttachmentTitleText(files?: TFile[] | null): string {
|
||||
if (!files?.length) {
|
||||
return '';
|
||||
}
|
||||
|
||||
const filenames = files.map((file) => file.filename).filter(Boolean);
|
||||
return filenames.length > 0 ? `Attached file(s): ${filenames.join(', ')}` : '';
|
||||
}
|
||||
|
||||
/**
|
||||
* Extracts text context from attachments and returns formatted text.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue