mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-17 21:43:06 +00:00
* feat: add unseen reply indicators, reply notifications, and mark as unread A reply that lands while the user is elsewhere now leaves a trace. The conversation carries a durable reply stamp written with the assistant message that backs it, the sidebar row shows an unread dot until that message is actually rendered, and an optional tab badge, desktop notification and chime announce arrivals in a tab that is away. Rows can also be marked unread by hand from the row menu, in the archived view as well as the main list. The stamp is written by a compare-and-swap so concurrent replies order on the database rather than on the application hosts' clocks, and only once output that a reader can actually see is persisted: a stopped turn with nothing to read, an empty synchronized row, or a failed message write never lights a dot that cannot be cleared. Read intent survives archives, metadata edits and preset applies, and a later read always outranks a delayed unread. * fix: gate reply alerts on config and announce only readable replies Three review rounds kept finding the same two shapes, so both are answered once rather than patched again. The alert capabilities are now operator-configurable through `interface.replyNotifications` in `librechat.yaml`: the tab badge, desktop notifications and the chime each have a gate, and the away poll's page size is a field rather than a constant, with defaults that reproduce the shipped behavior. A capability the operator turns off reads as off whatever the device stored, and its settings toggle is not offered. Deciding whether a persisted turn may raise an indicator now lives in one place in `packages/api`, with the Responses API controller, the abort route, the Assistants thread sync and the client's override path all calling it instead of carrying their own copy. That closes the last case where a reply nobody can read lit a dot nothing could clear: a `/v1/responses` completion whose output held only reasoning or tool calls persists an empty row, and stamping it left an unread conversation that opening could not settle. Also: the retention backfill no longer advances `updatedAt` past a reply stamp, which an away tab reads as a metadata-only promotion and withholds the alert for; arrival evidence is swept of conversations no cached list still holds, so a long-lived tab stops growing with its lifetime number of replies; and generated Lighthouse profile directories are ignored where the runner leaves them. * test: open the sidebar by what the drawer paints, not by its test ids The drawer became a mounted, sliding panel on canary, so the controls this helper read no longer say what it assumed. `close-sidebar-button` answers from the off-canvas copy before the expanded state commits, which returned the helper with the conversation list still hidden behind the pane, and the panel's own `new-chat-button` has a rail copy that a plain locator reaches first. Both readings passed on the previous drawer, which unmounted when closed. Settling on the close control's accessible name, and on the first candidate that is actually painted, is what a reader perceives and survives either drawer. One click is one toggle and the opener sits in the pane the drawer marks inert for the whole of its travel, so each attempt is given that travel to itself rather than reissued into it. Also serves `interface.replyNotifications` in the operator-gate scenario with a fresh body instead of the upstream response, since reusing its encoding headers for a re-serialized payload leaves the client unable to read the config, which looks exactly like an operator who configured nothing. * fix: gate the assistant save on renderable output and stop overselling the poll limit Two gaps this round's own work left behind. `saveAssistantMessage` still decided to stamp from the message id alone, so an Assistants run that persisted an empty row raised a dot that opening the conversation could never clear. It now asks `isAnnounceableReply`, the same predicate every other persistence path asks, which is the point of having moved that decision into one place. `pollLimit` advertised a range up to 1000 while `getConvosByCursor` clamps every conversation read to a hundred-row page, so an operator who set 500 would have been told it applied and silently served a hundred. The schema now stops at the page the server will actually return; covering a deployment whose replies outpace one page needs the server-side unseen query, not a wider page. * test: let the mobile scenarios use the surface they are actually asserting on Three scenarios opened the drawer on a page they only ever send a reply from. On the mobile project that drawer sits over the composer and marks the pane inert, so there was nothing left to type in and the run waited out its whole budget. They ask for the composer now, which is what they were always about. The two that read a row after leaving the list open it again first. A reply sent from another tab closes this one's drawer, because `sidebarExpanded` is stored per browser rather than per tab, and opening a chat closes it the way it does for a reader. Both are what someone would do to look at the row, and neither changes what is being asserted. Mark as unread is dispatched rather than clicked, like the row controls around it: the menu is portaled, and behind the drawer's scrim it inherits `pointer-events: none` and goes click-dead. * test: dispatch the row menu's unread item like the controls around it The portaled menu sits under the mobile drawer's scrim, which intercepts the pointer, so a real click retried until the budget ran out. The sibling scenarios and the row's own controls already dispatch; these two were the last real clicks. * fix: hold reply alerts until the deployment answers, and read the resolved temporary state The capability gate treated a startup config that had not loaded yet the same as one that set nothing, so a device that stored "on" could badge, poll or notify in the window before an operator's `false` arrived. Nothing is permitted now until the config is in hand; only a loaded config that lacks the field reads as the shipped default, which is the case a backend predating the setting produces. The Responses API announcement read the temporary flag from the request body alone, while the conversation save beside it resolves it from the stored conversation first. A temporary chat restored or resolved on the server without the flag in the body would have been announced through the unread indicators; both completion branches now resolve it the same way the save does. * fix: route the last reply stamps through the shared owner and hand back failed claims Moving the announce decision into packages/api left three callers still deciding for themselves. The agents controller re-stamped an unfinished turn whose terminal persistence was skipped, and the resumed legacy turn stamped its directly saved row, both from the message id alone, so a preempted or stopped turn that persisted nothing readable lit a dot that could never be acknowledged. Both ask isAnnounceableReply now. The non-agents abort path carried an inline copy of the same predicate; it calls announceReply instead. Error turns keep their direct stamp: the error card is what renders them. A notification claim was written to shared storage before the notification was constructed, so a constructor that throws left the reply claimed in every tab with nothing ever shown. The claim is handed back when construction fails, removed only while it still names that exact stamp. * fix: judge the normal reply save by the shared predicate and hand back failed chimes The regular BaseClient save still stamped any assistant reply that persisted, readable or not, because the conversation write took a bare reply id. It takes the persisted reply now and asks isAnnounceableReply before it stamps, which leaves the error turns as the only direct stamps, and those are rendered by the error card. saveTurnConversation is exercised against a real store in both directions. announce.ts derives its write-context type locally so the two modules do not import each other. The chime claimed its replies before building the tones, so an output that failed while they were being scheduled left them claimed in every tab with nothing played. Those claims are handed back on failure, the same way a notification that could not be constructed now releases its own. * feat: let operators set the reply polling cadence The away poll and the focused refresh were fixed at 30 seconds and five minutes, which are request-rate levers on the conversation list and nothing an operator could tune. Both are fields on `interface.replyNotifications` now, bounded and defaulting to the values they had, and the watcher reads them through the same capability hook as the rest of the block. * fix: keep reply identity through list merges and drop rows the server no longer has The list merge carried three of the four read-state fields by hand and left out `lastResponseMessageId`, so an SSE update on an unseen conversation erased which branch the reply landed on, and a missing identity reads as visible. It uses the same `preserveReadState` helper as the other two merge paths now, and the test that covered the other three fields covers this one too. A local cache merge was only inspected on a list's first page for new arrivals. A title or oldest-first sort leaves a replied row where it is, so its first reply on a later page went into the alerts baseline without an arrival and its chime or notification was lost. Local merges scan every loaded page; server responses keep the first-page rule, and only rows already known are considered. An unread request that matched nothing means the conversation is gone: the server's owner-scoped update returns the row even for a no-op. The mutation rolled back the read fields and kept the row; it removes it from the caches now, as a delete does. * fix: discard a job completion that resolves after the watcher has gone The fetch that settles a finished job had no teardown guard, unlike the away poll. Signing out or switching accounts unmounts the watcher, and a response still in flight for the previous session would then merge into the caches the next one reads. The result is dropped once the watcher has unmounted, tracked on unmount rather than in the jobs effect's cleanup, which re-runs on every change to the running set while its earlier completions are still wanted. * fix: move error-turn stamps into packages/api and settle overlapping unread no-matches Error turns were stamped in CJS: the error middleware checked eligibility, called the stamp and shaped the event's conversation snapshot itself, and the agents controller stamped its own. Both call announceErrorTurn now, and the skipped-persistence restamp goes through announceReply, so no stamp is issued directly from /api any more. The existing sendError tests pass against the real helper unchanged. An unread call that no longer owned the row dropped a server no-match with the rest of its stale answers, so a deleted conversation could be restored by a newer call's rollback. The no-match now evicts before ownership is consulted. It stays after the superseded check: a superseded call never reached the server, and its synthetic `modified: false` is not a no-match. Also: Settings offers the reply toggles only once the startup config has loaded, matching the capability hook; a cancelled reply-discovery snapshot is restarted even though nothing observes it; and the focused refresh is capped at the five minutes the discovery snapshot stays authoritative, since that refresh is what renews it. * fix: settle the assistant final in packages/api, announce attachment-only replies Both assistant controllers carried the same persistence barrier: write the response, check the row and the conversation both persisted, and unwrap the settled conversation for the final event. That is behaviour, and it was duplicated in CJS; settleAssistantFinal owns it now and the controllers call it. Their existing tests pass unchanged. The announce predicate ignored attachments, so a reply made only of files, such as a resumed workspace artifact with empty content and text, raised no dot, badge or alert. The message body renders attachments and acknowledgement checks the body, so such a reply can be cleared as well as announced; every announce path now passes them through. The stopped-reply announcement took the storage engine's own id type in its exported signature. It takes plain string ids now, and saveConvo accepts them alongside its own, since the update casts them against the schema; the id type stays inside data-schemas. * fix: normalize the stopped reply's written ids in packages/api The abort route filtered and stringified the ids of the rows it had written before handing them to announceStoppedReply. That is data shaping, and it sat in CJS; the announcement takes the ids as the caller holds them and drops the unwritten ones itself. * style: order the conversation row classes for Tailwind v4 The v4 upgrade on canary changed the class order Prettier enforces; these three rows carried the v3 order. * fix: replay replies a dead tab's focus lease held back, keep the focused refresh at a full page A tab killed without firing blur or pagehide leaves its focus lease for up to a minute. A reply arriving under it was baselined and then suppressed, and nothing reran when the lease lapsed, so its chime and notification were lost. Arrivals held back only by another tab's lease are kept and rechecked when that lease expires; a lease the focused tab keeps refreshing still holds them. The focused refresh reused the away-poll limit, so an operator who lowered it also shrank the discovery page that keeps the unseen count complete behind a filtered list. The limit now bounds only the away poll. * fix: announce a held reply as soon as the focused tab releases its lease A tab that blurs normally clears its lease long before the lapse, and a reply held back for it is due as soon as nobody is looking. Lease changes from other tabs now trigger the recheck, not only the expiry timer.
763 lines
26 KiB
JavaScript
763 lines
26 KiB
JavaScript
const path = require('path');
|
|
const { v4 } = require('uuid');
|
|
const { countTokens, announceReply, isAnnounceableReply } = require('@librechat/api');
|
|
const { escapeRegExp } = require('@librechat/data-schemas');
|
|
const {
|
|
Constants,
|
|
ContentTypes,
|
|
AnnotationTypes,
|
|
defaultOrderQuery,
|
|
} = require('librechat-data-provider');
|
|
const {
|
|
saveConvo,
|
|
getMessages,
|
|
spendTokens,
|
|
saveMessage,
|
|
stampConvoLastResponse,
|
|
} = require('~/models');
|
|
const { retrieveAndProcessFile } = require('~/server/services/Files/process');
|
|
|
|
/**
|
|
* Initializes a new thread or adds messages to an existing thread.
|
|
*
|
|
* @param {Object} params - The parameters for initializing a thread.
|
|
* @param {OpenAIClient} params.openai - The OpenAI client instance.
|
|
* @param {Object} params.body - The body of the request.
|
|
* @param {ThreadMessage[]} params.body.messages - A list of messages to start the thread with.
|
|
* @param {Object} [params.body.metadata] - Optional metadata for the thread.
|
|
* @param {string} [params.thread_id] - Optional existing thread ID. If provided, a message will be added to this thread.
|
|
* @return {Promise<Thread>} A promise that resolves to the newly created thread object or the updated thread object.
|
|
*/
|
|
async function initThread({ openai, body, thread_id: _thread_id }) {
|
|
let thread = {};
|
|
const messages = [];
|
|
if (_thread_id) {
|
|
const message = await openai.beta.threads.messages.create(_thread_id, body.messages[0]);
|
|
messages.push(message);
|
|
} else {
|
|
thread = await openai.beta.threads.create(body);
|
|
}
|
|
|
|
const thread_id = _thread_id || thread.id;
|
|
return { messages, thread_id, ...thread };
|
|
}
|
|
|
|
/**
|
|
* Saves a user message to the DB in the Assistants endpoint format.
|
|
*
|
|
* @param {Object} req - The request object.
|
|
* @param {Object} params - The parameters of the user message
|
|
* @param {string} params.user - The user's ID.
|
|
* @param {string} params.text - The user's prompt.
|
|
* @param {string} params.messageId - The user message Id.
|
|
* @param {string} params.model - The model used by the assistant.
|
|
* @param {string} params.assistant_id - The current assistant Id.
|
|
* @param {string} params.thread_id - The thread Id.
|
|
* @param {string} params.conversationId - The message's conversationId
|
|
* @param {string} params.endpoint - The conversation endpoint
|
|
* @param {string} [params.parentMessageId] - Optional if initial message.
|
|
* Defaults to Constants.NO_PARENT.
|
|
* @param {string} [params.instructions] - Optional: from preset for `instructions` field.
|
|
* Overrides the instructions of the assistant.
|
|
* @param {string} [params.promptPrefix] - Optional: from preset for `additional_instructions` field.
|
|
* @param {import('librechat-data-provider').TFile[]} [params.files] - Optional. List of Attached File Objects.
|
|
* @param {string[]} [params.file_ids] - Optional. List of File IDs attached to the userMessage.
|
|
* @return {Promise<Run>} A promise that resolves to the created run object.
|
|
*/
|
|
async function saveUserMessage(req, params) {
|
|
const tokenCount = await countTokens(params.text);
|
|
|
|
const userMessage = {
|
|
user: params.user,
|
|
endpoint: params.endpoint,
|
|
messageId: params.messageId,
|
|
conversationId: params.conversationId,
|
|
parentMessageId: params.parentMessageId ?? Constants.NO_PARENT,
|
|
/* For messages, use the assistant_id instead of model */
|
|
model: params.assistant_id,
|
|
thread_id: params.thread_id,
|
|
sender: 'User',
|
|
text: params.text,
|
|
isCreatedByUser: true,
|
|
tokenCount,
|
|
};
|
|
|
|
const convo = {
|
|
endpoint: params.endpoint,
|
|
conversationId: params.conversationId,
|
|
promptPrefix: params.promptPrefix,
|
|
instructions: params.instructions,
|
|
assistant_id: params.assistant_id,
|
|
model: params.model,
|
|
};
|
|
|
|
if (params.files?.length) {
|
|
userMessage.files = params.files.map(({ file_id }) => ({ file_id }));
|
|
convo.file_ids = params.file_ids;
|
|
}
|
|
|
|
const ctx = {
|
|
userId: req?.user?.id,
|
|
isTemporary: req?.resolvedConversation?.isTemporary ?? req?.body?.isTemporary,
|
|
expiredAt: req?.resolvedConversation?.expiredAt,
|
|
interfaceConfig: req?.config?.interfaceConfig,
|
|
};
|
|
const message = await saveMessage(ctx, userMessage);
|
|
const savedConvo = await saveConvo(
|
|
{ ...ctx, expiredAt: message?.expiredAt ?? ctx.expiredAt },
|
|
convo,
|
|
{
|
|
context: 'api/server/services/Threads/manage.js #saveUserMessage',
|
|
...(message?._id != null ? { appendMessageIds: [message._id] } : {}),
|
|
},
|
|
);
|
|
if (savedConvo != null) {
|
|
req.resolvedConversation = savedConvo;
|
|
}
|
|
return message;
|
|
}
|
|
|
|
/**
|
|
* Saves an Assistant message to the DB in the Assistants endpoint format.
|
|
*
|
|
* @param {Object} req - The request object.
|
|
* @param {Object} params - The parameters of the Assistant message
|
|
* @param {string} params.user - The user's ID.
|
|
* @param {string} params.messageId - The message Id.
|
|
* @param {string} params.text - The concatenated text of the message.
|
|
* @param {string} params.assistant_id - The assistant Id.
|
|
* @param {string} params.thread_id - The thread Id.
|
|
* @param {string} params.model - The model used by the assistant.
|
|
* @param {ContentPart[]} params.content - The message content parts.
|
|
* @param {string} params.conversationId - The message's conversationId
|
|
* @param {string} params.endpoint - The conversation endpoint
|
|
* @param {string} params.parentMessageId - The latest user message that triggered this response.
|
|
* @param {string} [params.instructions] - Optional: from preset for `instructions` field.
|
|
* @param {string} [params.spec] - Optional: Model spec identifier.
|
|
* @param {string} [params.iconURL]
|
|
* Overrides the instructions of the assistant.
|
|
* @param {string} [params.promptPrefix] - Optional: from preset for `additional_instructions` field.
|
|
* @return {Promise<{message: Object|null, conversation: Object|null}>} The persisted assistant
|
|
* message and the conversation snapshot settled by the same write.
|
|
*/
|
|
async function saveAssistantMessage(req, params) {
|
|
// const tokenCount = // TODO: need to count each content part
|
|
|
|
const ctx = {
|
|
userId: req?.user?.id,
|
|
isTemporary: req?.resolvedConversation?.isTemporary ?? req?.body?.isTemporary,
|
|
expiredAt: req?.resolvedConversation?.expiredAt,
|
|
interfaceConfig: req?.config?.interfaceConfig,
|
|
};
|
|
const message = await saveMessage(ctx, {
|
|
user: params.user,
|
|
endpoint: params.endpoint,
|
|
messageId: params.messageId,
|
|
conversationId: params.conversationId,
|
|
parentMessageId: params.parentMessageId,
|
|
thread_id: params.thread_id,
|
|
/* For messages, use the assistant_id instead of model */
|
|
model: params.assistant_id,
|
|
content: params.content,
|
|
sender: 'Assistant',
|
|
isCreatedByUser: false,
|
|
text: params.text,
|
|
unfinished: false,
|
|
// tokenCount,
|
|
iconURL: params.iconURL,
|
|
spec: params.spec,
|
|
});
|
|
|
|
const announceable = isAnnounceableReply({
|
|
messageId: message?.messageId,
|
|
content: params.content,
|
|
text: params.text,
|
|
isTemporary: ctx.isTemporary,
|
|
});
|
|
|
|
const savedConvo = await saveConvo(
|
|
{ ...ctx, expiredAt: message?.expiredAt ?? ctx.expiredAt },
|
|
{
|
|
endpoint: params.endpoint,
|
|
conversationId: params.conversationId,
|
|
promptPrefix: params.promptPrefix,
|
|
instructions: params.instructions,
|
|
assistant_id: params.assistant_id,
|
|
model: params.model,
|
|
iconURL: params.iconURL,
|
|
spec: params.spec,
|
|
},
|
|
{
|
|
context: 'api/server/services/Threads/manage.js #saveAssistantMessage',
|
|
/** Judged by the same predicate every other persistence path asks, so an assistant row
|
|
* that renders nothing cannot raise a dot here that opening the chat can never clear.
|
|
* `saveConvo` assigns the timestamp past its own awaited reads, so a catch-up recorded
|
|
* while one of them is in flight cannot outrank this reply. */
|
|
stampReply: announceable,
|
|
...(announceable ? { replyMessageId: message.messageId } : {}),
|
|
...(message?._id != null ? { appendMessageIds: [message._id] } : {}),
|
|
},
|
|
);
|
|
|
|
if (savedConvo != null) {
|
|
req.resolvedConversation = savedConvo;
|
|
}
|
|
|
|
return { message, conversation: savedConvo };
|
|
}
|
|
|
|
/**
|
|
* Records LibreChat messageId to all response messages' metadata
|
|
*
|
|
* @param {Object} params - The parameters for initializing a thread.
|
|
* @param {OpenAIClient} params.openai - The OpenAI client instance.
|
|
* @param {string} params.thread_id - Response thread ID.
|
|
* @param {string} params.messageId - The response `messageId` generated by LibreChat.
|
|
* @param {StepMessage[] | Message[]} params.messages - A list of messages to start the thread with.
|
|
* @return {Promise<ThreadMessage[]>} A promise that resolves to the updated messages
|
|
*/
|
|
async function addThreadMetadata({ openai, thread_id, messageId, messages }) {
|
|
const promises = [];
|
|
for (const message of messages) {
|
|
promises.push(
|
|
openai.beta.threads.messages.update(message.id, {
|
|
thread_id,
|
|
metadata: {
|
|
messageId,
|
|
},
|
|
}),
|
|
);
|
|
}
|
|
|
|
return await Promise.all(promises);
|
|
}
|
|
|
|
/**
|
|
* Synchronizes LibreChat messages to Thread Messages.
|
|
* Updates the LibreChat DB with any missing Thread Messages and
|
|
* updates the missing Thread Messages' metadata with their corresponding db messageId's.
|
|
*
|
|
* Also updates the existing conversation's file_ids with any new file_ids.
|
|
*
|
|
* @param {Object} params - The parameters for synchronizing messages.
|
|
* @param {OpenAIClient} params.openai - The OpenAI client instance.
|
|
* @param {string} params.endpoint - The current endpoint.
|
|
* @param {string} params.thread_id - The current thread ID.
|
|
* @param {TMessage[]} params.dbMessages - The LibreChat DB messages.
|
|
* @param {ThreadMessage[]} params.apiMessages - The thread messages from the API.
|
|
* @param {string} [params.assistant_id] - The current assistant ID.
|
|
* @param {string} params.conversationId - The current conversation ID.
|
|
* @return {Promise<TMessage[]>} A promise that resolves to the updated messages
|
|
*/
|
|
async function syncMessages({
|
|
openai,
|
|
endpoint,
|
|
thread_id,
|
|
dbMessages,
|
|
apiMessages,
|
|
assistant_id,
|
|
conversationId,
|
|
}) {
|
|
let result = [];
|
|
let dbMessageMap = new Map(dbMessages.map((msg) => [msg.messageId, msg]));
|
|
|
|
const modifyPromises = [];
|
|
const recordPromises = [];
|
|
const ctx = {
|
|
userId: openai.req?.user?.id,
|
|
isTemporary: openai.req?.resolvedConversation?.isTemporary ?? openai.req?.body?.isTemporary,
|
|
expiredAt: openai.req?.resolvedConversation?.expiredAt,
|
|
interfaceConfig: openai.req?.config?.interfaceConfig,
|
|
};
|
|
/** The assistant writes this synchronization performed. Their results decide whether a reply
|
|
* is actually in the history: a write that resolved empty must not raise an indicator. */
|
|
const assistantRecordPromises = [];
|
|
|
|
/**
|
|
*
|
|
* Modify API message and save newMessage to DB
|
|
*
|
|
* @param {Object} params - The parameters object
|
|
* @param {TMessage} params.dbMessage
|
|
* @param {dbMessage} params.apiMessage
|
|
*/
|
|
const processNewMessage = async ({ dbMessage, apiMessage }) => {
|
|
const recorded = saveMessage(ctx, { ...dbMessage, user: openai.req.user.id });
|
|
recordPromises.push(recorded);
|
|
if (dbMessage.role === 'assistant') {
|
|
assistantRecordPromises.push(recorded);
|
|
}
|
|
|
|
if (!apiMessage.id.includes('msg_')) {
|
|
return;
|
|
}
|
|
|
|
if (dbMessage.aggregateMessages?.length > 1) {
|
|
modifyPromises.push(
|
|
addThreadMetadata({
|
|
openai,
|
|
thread_id,
|
|
messageId: dbMessage.messageId,
|
|
messages: dbMessage.aggregateMessages,
|
|
}),
|
|
);
|
|
return;
|
|
}
|
|
|
|
modifyPromises.push(
|
|
openai.beta.threads.messages.update(apiMessage.id, {
|
|
thread_id,
|
|
metadata: {
|
|
messageId: dbMessage.messageId,
|
|
},
|
|
}),
|
|
);
|
|
};
|
|
|
|
let lastMessage = null;
|
|
|
|
for (let i = 0; i < apiMessages.length; i++) {
|
|
const apiMessage = apiMessages[i];
|
|
|
|
// Check if the message exists in the database based on metadata
|
|
const dbMessageId = apiMessage.metadata && apiMessage.metadata.messageId;
|
|
let dbMessage = dbMessageMap.get(dbMessageId);
|
|
|
|
if (dbMessage) {
|
|
// If message exists in DB, use its messageId and update parentMessageId
|
|
dbMessage.parentMessageId = lastMessage ? lastMessage.messageId : Constants.NO_PARENT;
|
|
lastMessage = dbMessage;
|
|
result.push(dbMessage);
|
|
continue;
|
|
}
|
|
|
|
if (apiMessage.role === 'assistant' && lastMessage && lastMessage.role === 'assistant') {
|
|
// Aggregate assistant messages
|
|
lastMessage.content = [...lastMessage.content, ...apiMessage.content];
|
|
lastMessage.files = [...(lastMessage.files ?? []), ...(apiMessage.files ?? [])];
|
|
lastMessage.aggregateMessages.push({ id: apiMessage.id });
|
|
} else {
|
|
// Handle new or missing message
|
|
const newMessage = {
|
|
thread_id,
|
|
conversationId,
|
|
messageId: v4(),
|
|
endpoint,
|
|
parentMessageId: lastMessage ? lastMessage.messageId : Constants.NO_PARENT,
|
|
role: apiMessage.role,
|
|
isCreatedByUser: apiMessage.role === 'user',
|
|
// TODO: process generated files in content parts
|
|
content: apiMessage.content,
|
|
aggregateMessages: [{ id: apiMessage.id }],
|
|
model: apiMessage.role === 'user' ? null : apiMessage.assistant_id,
|
|
user: openai.req.user.id,
|
|
unfinished: false,
|
|
};
|
|
|
|
if (apiMessage.file_ids?.length) {
|
|
// TODO: retrieve file objects from API
|
|
newMessage.files = apiMessage.file_ids.map((file_id) => ({ file_id }));
|
|
}
|
|
|
|
/* Assign assistant_id if defined */
|
|
if (assistant_id && apiMessage.role === 'assistant' && !newMessage.model) {
|
|
apiMessage.model = assistant_id;
|
|
newMessage.model = assistant_id;
|
|
}
|
|
|
|
result.push(newMessage);
|
|
lastMessage = newMessage;
|
|
|
|
if (apiMessage.role === 'user') {
|
|
processNewMessage({ dbMessage: newMessage, apiMessage });
|
|
continue;
|
|
}
|
|
}
|
|
|
|
const nextMessage = apiMessages[i + 1];
|
|
const processAssistant = !nextMessage || nextMessage.role === 'user';
|
|
|
|
if (apiMessage.role === 'assistant' && processAssistant) {
|
|
processNewMessage({ dbMessage: lastMessage, apiMessage });
|
|
}
|
|
}
|
|
|
|
const attached_file_ids = apiMessages.reduce((acc, msg) => {
|
|
if (msg.role === 'user' && msg.file_ids?.length) {
|
|
return [...acc, ...msg.file_ids];
|
|
}
|
|
|
|
return acc;
|
|
}, []);
|
|
|
|
await Promise.all(modifyPromises);
|
|
const recordedAssistantReplies = await Promise.all(assistantRecordPromises);
|
|
await Promise.all(recordPromises);
|
|
|
|
const savedConvo = await saveConvo(
|
|
ctx,
|
|
{
|
|
conversationId,
|
|
file_ids: attached_file_ids,
|
|
},
|
|
{ context: 'api/server/services/Threads/manage.js #syncMessages' },
|
|
);
|
|
if (savedConvo != null) {
|
|
openai.req.resolvedConversation = savedConvo;
|
|
}
|
|
|
|
/* Every caller that reaches here recovers assistant output the normal save path never wrote:
|
|
a cancelled run, or one that errored after the model had already produced content. The
|
|
`saveConvo` above carries no reply stamp, so without this the recovered reply would never
|
|
raise its unseen indicator. Only a write that actually persisted counts, and it is
|
|
best-effort, since those messages are already durable.
|
|
A run cancelled before any output still persists the synthetic row `checkMessageGaps`
|
|
built, and the error handlers add their visible content to the returned object rather
|
|
than to that row: stamping it would raise a dot for a message that renders nothing, and
|
|
acknowledgement requires the stamped reply to be on screen. */
|
|
const persistedAssistantReply = recordedAssistantReplies.findLast((message) => message != null);
|
|
await announceReply(
|
|
{ stampConvoLastResponse },
|
|
{
|
|
userId: openai.req.user.id,
|
|
conversationId,
|
|
reply: { ...persistedAssistantReply, isTemporary: ctx.isTemporary },
|
|
context: 'syncMessages',
|
|
},
|
|
);
|
|
|
|
return result;
|
|
}
|
|
|
|
/**
|
|
* Maps messages to their corresponding steps. Steps with message creation will be paired with their messages,
|
|
* while steps without message creation will be returned as is.
|
|
*
|
|
* @param {RunStep[]} steps - An array of steps from the run.
|
|
* @param {Message[]} messages - An array of message objects.
|
|
* @returns {(StepMessage | RunStep)[]} An array where each element is either a step with its corresponding message (StepMessage) or a step without a message (RunStep).
|
|
*/
|
|
function mapMessagesToSteps(steps, messages) {
|
|
// Create a map of messages indexed by their IDs for efficient lookup
|
|
const messageMap = messages.reduce((acc, msg) => {
|
|
acc[msg.id] = msg;
|
|
return acc;
|
|
}, {});
|
|
|
|
// Map each step to its corresponding message, or return the step as is if no message ID is present
|
|
return steps
|
|
.sort((a, b) => a.created_at - b.created_at)
|
|
.map((step) => {
|
|
const messageId = step.step_details?.message_creation?.message_id;
|
|
|
|
if (messageId && messageMap[messageId]) {
|
|
return { step, message: messageMap[messageId] };
|
|
}
|
|
return step;
|
|
});
|
|
}
|
|
|
|
/**
|
|
* Checks for any missing messages; if missing,
|
|
* synchronizes LibreChat messages to Thread Messages
|
|
*
|
|
* @param {Object} params - The parameters for initializing a thread.
|
|
* @param {OpenAIClient} params.openai - The OpenAI client instance.
|
|
* @param {string} params.endpoint - The current endpoint.
|
|
* @param {string} [params.latestMessageId] - Optional: The latest message ID from LibreChat.
|
|
* @param {string} params.thread_id - Response thread ID.
|
|
* @param {string} params.run_id - Response Run ID.
|
|
* @param {string} params.conversationId - LibreChat conversation ID.
|
|
* @return {Promise<TMessage[]>} A promise that resolves to the updated messages
|
|
*/
|
|
async function checkMessageGaps({
|
|
openai,
|
|
endpoint,
|
|
latestMessageId,
|
|
thread_id,
|
|
run_id,
|
|
conversationId,
|
|
}) {
|
|
const promises = [];
|
|
promises.push(openai.beta.threads.messages.list(thread_id, defaultOrderQuery));
|
|
promises.push(openai.beta.threads.runs.steps.list(run_id, { thread_id }));
|
|
/** @type {[{ data: ThreadMessage[] }, { data: RunStep[] }]} */
|
|
const [response, stepsResponse] = await Promise.all(promises);
|
|
|
|
const steps = mapMessagesToSteps(stepsResponse.data, response.data);
|
|
/** @type {ThreadMessage} */
|
|
const currentMessage = {
|
|
id: v4(),
|
|
content: [],
|
|
assistant_id: null,
|
|
created_at: Math.floor(new Date().getTime() / 1000),
|
|
object: 'thread.message',
|
|
role: 'assistant',
|
|
run_id,
|
|
thread_id,
|
|
endpoint,
|
|
metadata: {
|
|
messageId: latestMessageId,
|
|
},
|
|
};
|
|
|
|
for (const step of steps) {
|
|
if (!currentMessage.assistant_id && step.assistant_id) {
|
|
currentMessage.assistant_id = step.assistant_id;
|
|
}
|
|
if (step.message) {
|
|
currentMessage.id = step.message.id;
|
|
currentMessage.created_at = step.message.created_at;
|
|
currentMessage.content = currentMessage.content.concat(step.message.content);
|
|
} else if (step.step_details?.type === 'tool_calls' && step.step_details?.tool_calls?.length) {
|
|
currentMessage.content = currentMessage.content.concat(
|
|
step.step_details?.tool_calls.map((toolCall) => ({
|
|
[ContentTypes.TOOL_CALL]: {
|
|
...toolCall,
|
|
progress: 2,
|
|
},
|
|
type: ContentTypes.TOOL_CALL,
|
|
})),
|
|
);
|
|
}
|
|
}
|
|
|
|
let addedCurrentMessage = false;
|
|
const apiMessages = response.data
|
|
.map((msg) => {
|
|
if (msg.id === currentMessage.id) {
|
|
addedCurrentMessage = true;
|
|
return currentMessage;
|
|
}
|
|
return msg;
|
|
})
|
|
.sort((a, b) => new Date(a.created_at) - new Date(b.created_at));
|
|
|
|
if (!addedCurrentMessage) {
|
|
apiMessages.push(currentMessage);
|
|
}
|
|
|
|
const dbMessages = await getMessages({ conversationId, user: openai.req.user.id });
|
|
const assistant_id = dbMessages?.[0]?.model;
|
|
|
|
const syncedMessages = await syncMessages({
|
|
openai,
|
|
endpoint,
|
|
thread_id,
|
|
dbMessages,
|
|
apiMessages,
|
|
assistant_id,
|
|
conversationId,
|
|
});
|
|
|
|
return Object.values(
|
|
[...dbMessages, ...syncedMessages].reduce(
|
|
(acc, message) => ({ ...acc, [message.messageId]: message }),
|
|
{},
|
|
),
|
|
);
|
|
}
|
|
|
|
/**
|
|
* Records token usage for a given completion request.
|
|
* @param {Object} params - The parameters for initializing a thread.
|
|
* @param {number} params.prompt_tokens - The number of prompt tokens used.
|
|
* @param {number} params.completion_tokens - The number of completion tokens used.
|
|
* @param {string} params.model - The model used by the assistant run.
|
|
* @param {string} params.user - The user's ID.
|
|
* @param {string} params.conversationId - LibreChat conversation ID.
|
|
* @param {string} [params.context='message'] - The context of the usage. Defaults to 'message'.
|
|
* @param {AppConfig['transactions']} [params.transactions] - Resolved transactions config.
|
|
* @return {Promise<void>}
|
|
*/
|
|
const recordUsage = async ({
|
|
prompt_tokens,
|
|
completion_tokens,
|
|
model,
|
|
user,
|
|
conversationId,
|
|
context = 'message',
|
|
transactions,
|
|
}) => {
|
|
await spendTokens(
|
|
{
|
|
user,
|
|
model,
|
|
context,
|
|
conversationId,
|
|
transactions,
|
|
},
|
|
{ promptTokens: prompt_tokens, completionTokens: completion_tokens },
|
|
);
|
|
};
|
|
|
|
const uniqueCitationStart = '^====||===';
|
|
const uniqueCitationEnd = '==|||||^';
|
|
|
|
/**
|
|
* Sorts, processes, and flattens messages to a single string.
|
|
*
|
|
* @param {object} params - The parameters for processing messages.
|
|
* @param {OpenAIClient} params.openai - The OpenAI client instance.
|
|
* @param {RunClient} params.client - The LibreChat client that manages the run: either refers to `OpenAI` or `StreamRunManager`.
|
|
* @param {ThreadMessage[]} params.messages - An array of messages.
|
|
* @returns {Promise<{messages: ThreadMessage[], text: string, edited: boolean}>} The sorted messages, the flattened text, and whether it was edited.
|
|
*/
|
|
async function processMessages({ openai, client, messages = [] }) {
|
|
const sorted = messages.sort((a, b) => a.created_at - b.created_at);
|
|
|
|
let text = '';
|
|
let edited = false;
|
|
const sources = new Map();
|
|
const fileRetrievalPromises = [];
|
|
|
|
for (const message of sorted) {
|
|
message.files = [];
|
|
for (const content of message.content) {
|
|
const type = content.type;
|
|
const contentType = content[type];
|
|
const currentFileId = contentType?.file_id;
|
|
|
|
if (type === ContentTypes.IMAGE_FILE && !client.processedFileIds.has(currentFileId)) {
|
|
fileRetrievalPromises.push(
|
|
retrieveAndProcessFile({
|
|
openai,
|
|
client,
|
|
file_id: currentFileId,
|
|
basename: `${currentFileId}.png`,
|
|
})
|
|
.then((file) => {
|
|
client.processedFileIds.add(currentFileId);
|
|
message.files.push(file);
|
|
})
|
|
.catch((error) => {
|
|
console.error(`Failed to retrieve file: ${error.message}`);
|
|
}),
|
|
);
|
|
continue;
|
|
}
|
|
|
|
let currentText = contentType?.value ?? '';
|
|
|
|
/** @type {{ annotations: Annotation[] }} */
|
|
const { annotations } = contentType ?? {};
|
|
|
|
if (!annotations?.length) {
|
|
text += currentText;
|
|
continue;
|
|
}
|
|
|
|
const replacements = [];
|
|
const annotationPromises = annotations.map(async (annotation) => {
|
|
const type = annotation.type;
|
|
const annotationType = annotation[type];
|
|
const file_id = annotationType?.file_id;
|
|
const alreadyProcessed = client.processedFileIds.has(file_id);
|
|
|
|
let file;
|
|
let replacementText = '';
|
|
|
|
try {
|
|
if (alreadyProcessed) {
|
|
file = await retrieveAndProcessFile({ openai, client, file_id, unknownType: true });
|
|
} else if (type === AnnotationTypes.FILE_PATH) {
|
|
const basename = path.basename(annotation.text);
|
|
file = await retrieveAndProcessFile({
|
|
openai,
|
|
client,
|
|
file_id,
|
|
basename,
|
|
});
|
|
replacementText = file.filepath;
|
|
} else if (type === AnnotationTypes.FILE_CITATION && file_id) {
|
|
file = await retrieveAndProcessFile({
|
|
openai,
|
|
client,
|
|
file_id,
|
|
unknownType: true,
|
|
});
|
|
if (file && file.filename) {
|
|
if (!sources.has(file.filename)) {
|
|
sources.set(file.filename, sources.size + 1);
|
|
}
|
|
replacementText = `${uniqueCitationStart}${sources.get(
|
|
file.filename,
|
|
)}${uniqueCitationEnd}`;
|
|
}
|
|
}
|
|
|
|
if (file && replacementText) {
|
|
replacements.push({
|
|
start: annotation.start_index,
|
|
end: annotation.end_index,
|
|
text: replacementText,
|
|
});
|
|
edited = true;
|
|
if (!alreadyProcessed) {
|
|
client.processedFileIds.add(file_id);
|
|
message.files.push(file);
|
|
}
|
|
}
|
|
} catch (error) {
|
|
console.error(`Failed to process annotation: ${error.message}`);
|
|
}
|
|
});
|
|
|
|
await Promise.all(annotationPromises);
|
|
|
|
// Apply replacements in reverse order
|
|
replacements.sort((a, b) => b.start - a.start);
|
|
for (const { start, end, text: replacementText } of replacements) {
|
|
currentText = currentText.slice(0, start) + replacementText + currentText.slice(end);
|
|
}
|
|
|
|
text += currentText;
|
|
}
|
|
}
|
|
|
|
await Promise.all(fileRetrievalPromises);
|
|
|
|
// Handle adjacent identical citations with the unique format
|
|
const adjacentCitationRegex = new RegExp(
|
|
`${escapeRegExp(uniqueCitationStart)}(\\d+)${escapeRegExp(
|
|
uniqueCitationEnd,
|
|
)}(\\s*)${escapeRegExp(uniqueCitationStart)}(\\d+)${escapeRegExp(uniqueCitationEnd)}`,
|
|
'g',
|
|
);
|
|
text = text.replace(adjacentCitationRegex, (match, num1, space, num2) => {
|
|
return num1 === num2
|
|
? `${uniqueCitationStart}${num1}${uniqueCitationEnd}`
|
|
: `${uniqueCitationStart}${num1}${uniqueCitationEnd}${space}${uniqueCitationStart}${num2}${uniqueCitationEnd}`;
|
|
});
|
|
|
|
// Remove any remaining adjacent identical citations
|
|
const remainingAdjacentRegex = new RegExp(
|
|
`(${escapeRegExp(uniqueCitationStart)}(\\d+)${escapeRegExp(uniqueCitationEnd)})\\s*\\1+`,
|
|
'g',
|
|
);
|
|
text = text.replace(remainingAdjacentRegex, '$1');
|
|
|
|
// Replace the unique citation format with the final format
|
|
text = text.replace(new RegExp(escapeRegExp(uniqueCitationStart), 'g'), '^');
|
|
text = text.replace(new RegExp(escapeRegExp(uniqueCitationEnd), 'g'), '^');
|
|
|
|
if (sources.size) {
|
|
text += '\n\n';
|
|
Array.from(sources.entries()).forEach(([source, index], arrayIndex) => {
|
|
text += `^${index}.^ ${source}${arrayIndex === sources.size - 1 ? '' : '\n'}`;
|
|
});
|
|
}
|
|
|
|
return { messages: sorted, text, edited };
|
|
}
|
|
|
|
module.exports = {
|
|
initThread,
|
|
recordUsage,
|
|
processMessages,
|
|
saveUserMessage,
|
|
checkMessageGaps,
|
|
addThreadMetadata,
|
|
mapMessagesToSteps,
|
|
saveAssistantMessage,
|
|
};
|