mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-06-10 10:05:18 +00:00
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
* feat: support data retention for normal chats Add retentionMode config variable supporting "all" and "temporary" values. When "all" is set, data retention applies to all chats, not just temporary ones. Adds isTemporary field to conversations for proper filtering. Adapted to new TS method files in packages/data-schemas since upstream moved models out of api/models/. Based on danny-avila/LibreChat#10532 Co-Authored-By: WhammyLeaf <233105313+WhammyLeaf@users.noreply.github.com> (cherry picked from commit30109e90b0) * feat: extend data retention to files, tool calls, and shared links Add expiredAt field and TTL indexes to file, toolCall, and share schemas. Set expiredAt on tool calls, shared links, and file uploads when retentionMode is "all" or chat is temporary. (cherry picked from commit48973752d3) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: lint/test (cherry picked from commit310c514e6a) * fix: address code review feedback for data retention PR Critical: - Fix BookmarkMenu crash: restore optional chaining on conversation - Fix migration hazard: backward-compatible sidebar filter that also checks expiredAt for documents without isTemporary field Major: - Add logging to getRetentionExpiry error path, align with tools.js - Add tests for retentionMode: ALL in saveConvo and saveMessage - Fix share route: apply expiredAt for temporary chats too by querying the conversation's isTemporary flag server-side - Add assertions for getRetentionExpiry mocks in process tests Minor: - Fix ChatRoute isTemporaryChat to be strictly boolean via Boolean() - Fix stale test description (expired -> temporary) - Comment out retentionMode default in example yaml - Simplify verbose if/else to isTemporary === true - Add compound index on { user: 1, isTemporary: 1 } - Remove narrating comment from process.spec.js Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> (cherry picked from commit6bad535f90) * chore: fix typescript (cherry picked from commit826527a46b) * fix: lint (cherry picked from commit77817e80ea) * fix: use mockSanitizeArtifactPath in retention test The 'getRetentionExpiry is called with the request object' test referenced an undefined `mockSanitizeFilename` identifier, breaking both lint (no-undef) and the test suite. Use the existing `mockSanitizeArtifactPath` mock that the surrounding tests already use, since `processCodeOutput` calls `sanitizeArtifactPath` (not `sanitizeFilename`) before invoking `getRetentionExpiry`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> (cherry picked from commit52ea2da66d) * fix: forward isTemporary from client for retention on file uploads and tool calls Server-side `getRetentionExpiry` (file uploads) and the tool-call controller both read `req.body.isTemporary`, but the file upload multipart form and the tool-call payload did not include that field. In `retentionMode: temporary` (default), files uploaded and tool calls created from temporary chats were therefore retained indefinitely. Forward the Recoil `isTemporary` flag in both client paths so the existing server checks can fire correctly. `ToolParams` gains an optional `isTemporary` field. Addresses Codex P1 review feedback on PR #29. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> (cherry picked from commit7e937df05a) * test: stub store.isTemporary in useFileHandling test mocks Previous commit added `useRecoilValue(store.isTemporary)` to the hook. The test file mocks `~/store` with only `ephemeralAgentByConvoId` and does not stub `useRecoilValue`, so all 7 cases threw "Invalid argument to useRecoilValue: expected an atom or selector but got undefined". Add a stub default export with `isTemporary` and a `useRecoilValue` mock returning `false`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> (cherry picked from commiteb1609537d) * fix: harden data retention semantics * fix: provide sweep request context for expired files * fix: preserve temporary flags in all-retention updates * fix: honor assistant versions in retention sweeps * fix: retain non-temporary flags in all mode * fix: hide expired retained records * fix: propagate retained conversation expiry * fix: refresh meili retention cutoff * fix: prevent overlapping file sweeps * fix: show legacy retained conversations * fix: index legacy retained records * fix: harden retention cleanup edge cases * fix: count failed file storage sweeps * fix: preserve legacy temporary retention * fix: assign retention sweep worker deterministically * fix: hide expired shared links on reads * fix: prevent retention refresh after parent expiry * fix: break code output retention import cycle * fix: harden retention review findings * fix: ignore expired share duplicates * fix: reject expired retained share creation * fix: harden retention review edge cases * fix: address retention audit findings * fix: enforce expired conversation shares in all retention * fix: scope temporary upload flag to chat files * fix: address retention review findings * fix: address codex retention review findings * fix: tighten missing storage detection * test: remove unused file process spec bindings --------- Co-authored-by: WhammyLeaf <233105313+WhammyLeaf@users.noreply.github.com> Co-authored-by: Aron Gates <aron@muonspace.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
259 lines
8.9 KiB
JavaScript
259 lines
8.9 KiB
JavaScript
const { nanoid } = require('nanoid');
|
|
const { logger } = require('@librechat/data-schemas');
|
|
const { checkAccess, loadWebSearchAuth } = require('@librechat/api');
|
|
const {
|
|
Tools,
|
|
AuthType,
|
|
Permissions,
|
|
ToolCallTypes,
|
|
PermissionTypes,
|
|
} = require('librechat-data-provider');
|
|
const { getRoleByName, createToolCall, getToolCallsByConvo, getMessage } = require('~/models');
|
|
const { processFileURL, uploadImageBuffer } = require('~/server/services/Files/process');
|
|
const { getRetentionExpiry } = require('~/server/services/Files/retention');
|
|
const { processCodeOutput, runPreviewFinalize } = require('~/server/services/Files/Code/process');
|
|
const { loadAuthValues } = require('~/server/services/Tools/credentials');
|
|
const { loadTools } = require('~/app/clients/tools/util');
|
|
|
|
/**
|
|
* Tools that are callable directly via `POST /tools/:toolId/call`.
|
|
* `execute_code` is the only entry today; the tool runs server-side via
|
|
* the agents library / sandbox service without any per-user credential.
|
|
*/
|
|
const directCallableTools = new Set([Tools.execute_code]);
|
|
|
|
const toolAccessPermType = {
|
|
[Tools.execute_code]: PermissionTypes.RUN_CODE,
|
|
};
|
|
|
|
/**
|
|
* Verifies web search authentication, ensuring each category has at least
|
|
* one fully authenticated service.
|
|
*
|
|
* @param {ServerRequest} req - The request object
|
|
* @param {ServerResponse} res - The response object
|
|
* @returns {Promise<void>} A promise that resolves when the function has completed
|
|
*/
|
|
const verifyWebSearchAuth = async (req, res) => {
|
|
try {
|
|
const appConfig = req.config;
|
|
const userId = req.user.id;
|
|
/** @type {TCustomConfig['webSearch']} */
|
|
const webSearchConfig = appConfig?.webSearch || {};
|
|
const result = await loadWebSearchAuth({
|
|
userId,
|
|
loadAuthValues,
|
|
webSearchConfig,
|
|
throwError: false,
|
|
});
|
|
|
|
return res.status(200).json({
|
|
authenticated: result.authenticated,
|
|
authTypes: result.authTypes,
|
|
});
|
|
} catch (error) {
|
|
console.error('Error in verifyWebSearchAuth:', error);
|
|
return res.status(500).json({ message: error.message });
|
|
}
|
|
};
|
|
|
|
/**
|
|
* @param {ServerRequest} req - The request object, containing information about the HTTP request.
|
|
* @param {ServerResponse} res - The response object, used to send back the desired HTTP response.
|
|
* @returns {Promise<void>} A promise that resolves when the function has completed.
|
|
*/
|
|
const verifyToolAuth = async (req, res) => {
|
|
try {
|
|
const { toolId } = req.params;
|
|
if (toolId === Tools.web_search) {
|
|
return await verifyWebSearchAuth(req, res);
|
|
}
|
|
if (!directCallableTools.has(toolId)) {
|
|
res.status(404).json({ message: 'Tool not found' });
|
|
return;
|
|
}
|
|
/**
|
|
* `execute_code` no longer requires a per-user credential — sandbox
|
|
* auth is handled server-side by the agents library. Always report
|
|
* system-authenticated so the client proceeds straight to the call
|
|
* without a key-entry dialog.
|
|
*
|
|
* Deployment contract: reachability of the sandbox service is the
|
|
* admin's responsibility. This endpoint does not probe the service
|
|
* (a per-auth-check network hop would be too expensive for what is
|
|
* a UI-gate query). If the sandbox is unreachable, the call path
|
|
* surfaces the error at execution time instead of here.
|
|
*/
|
|
res.status(200).json({ authenticated: true, message: AuthType.SYSTEM_DEFINED });
|
|
} catch (error) {
|
|
res.status(500).json({ message: error.message });
|
|
}
|
|
};
|
|
|
|
/**
|
|
* @param {ServerRequest} req - The request object, containing information about the HTTP request.
|
|
* @param {ServerResponse} res - The response object, used to send back the desired HTTP response.
|
|
* @param {NextFunction} next - The next middleware function to call.
|
|
* @returns {Promise<void>} A promise that resolves when the function has completed.
|
|
*/
|
|
const callTool = async (req, res) => {
|
|
try {
|
|
const appConfig = req.config;
|
|
const { toolId = '' } = req.params;
|
|
if (!directCallableTools.has(toolId)) {
|
|
logger.warn(`[${toolId}/call] User ${req.user.id} attempted call to invalid tool`);
|
|
res.status(404).json({ message: 'Tool not found' });
|
|
return;
|
|
}
|
|
|
|
const { partIndex, blockIndex, messageId, conversationId, ...args } = req.body;
|
|
if (!messageId) {
|
|
logger.warn(`[${toolId}/call] User ${req.user.id} attempted call without message ID`);
|
|
res.status(400).json({ message: 'Message ID required' });
|
|
return;
|
|
}
|
|
|
|
const message = await getMessage({ user: req.user.id, messageId });
|
|
if (!message) {
|
|
logger.debug(`[${toolId}/call] User ${req.user.id} attempted call with invalid message ID`);
|
|
res.status(404).json({ message: 'Message not found' });
|
|
return;
|
|
}
|
|
logger.debug(`[${toolId}/call] User: ${req.user.id}`);
|
|
let hasAccess = true;
|
|
if (toolAccessPermType[toolId]) {
|
|
hasAccess = await checkAccess({
|
|
user: req.user,
|
|
permissionType: toolAccessPermType[toolId],
|
|
permissions: [Permissions.USE],
|
|
getRoleByName,
|
|
});
|
|
}
|
|
if (!hasAccess) {
|
|
logger.warn(
|
|
`[${toolAccessPermType[toolId]}] Forbidden: Insufficient permissions for User ${req.user.id}: ${Permissions.USE}`,
|
|
);
|
|
return res.status(403).json({ message: 'Forbidden: Insufficient permissions' });
|
|
}
|
|
const { loadedTools } = await loadTools({
|
|
user: req.user.id,
|
|
tools: [toolId],
|
|
functions: true,
|
|
options: {
|
|
req,
|
|
returnMetadata: true,
|
|
processFileURL,
|
|
uploadImageBuffer,
|
|
},
|
|
webSearch: appConfig.webSearch,
|
|
fileStrategy: appConfig.fileStrategy,
|
|
imageOutputType: appConfig.imageOutputType,
|
|
});
|
|
|
|
const tool = loadedTools[0];
|
|
const toolCallId = `${req.user.id}_${nanoid()}`;
|
|
const result = await tool.invoke({
|
|
args,
|
|
name: toolId,
|
|
id: toolCallId,
|
|
type: ToolCallTypes.TOOL_CALL,
|
|
});
|
|
|
|
const { content, artifact } = result;
|
|
const toolCallData = {
|
|
toolId,
|
|
messageId,
|
|
partIndex,
|
|
blockIndex,
|
|
conversationId,
|
|
result: content,
|
|
user: req.user.id,
|
|
...(await getRetentionExpiry(req)),
|
|
};
|
|
|
|
if (!artifact || !artifact.files || toolId !== Tools.execute_code) {
|
|
createToolCall(toolCallData).catch((error) => {
|
|
logger.error(`Error creating tool call: ${error.message}`);
|
|
});
|
|
return res.status(200).json({
|
|
result: content,
|
|
});
|
|
}
|
|
|
|
const artifactPromises = [];
|
|
for (const file of artifact.files) {
|
|
/* Files flagged `inherited` by codeapi are unchanged passthroughs of
|
|
* inputs the caller already owns (skill files, prior downloaded inputs,
|
|
* inherited .dirkeep markers). Re-downloading them is wasted work and
|
|
* 403s when the file is scoped to a different entity (e.g. skill
|
|
* entity_id) than the user's session key. They remain available for
|
|
* subsequent tool calls via primeInvokedSkills / session inheritance. */
|
|
if (file.inherited) {
|
|
continue;
|
|
}
|
|
const { id, name } = file;
|
|
artifactPromises.push(
|
|
(async () => {
|
|
const result = await processCodeOutput({
|
|
req,
|
|
id,
|
|
name,
|
|
messageId,
|
|
toolCallId,
|
|
conversationId,
|
|
session_id: artifact.session_id,
|
|
});
|
|
const fileMetadata = result?.file ?? null;
|
|
const finalize = result?.finalize;
|
|
if (!fileMetadata) {
|
|
return null;
|
|
}
|
|
/* This endpoint is non-streaming and its contract is "give
|
|
* me the artifacts" — return the persisted record immediately
|
|
* (with `status: 'pending'` for office buckets) and run the
|
|
* preview render in the background. The client polls
|
|
* `/api/files/:file_id/preview` for the resolved record.
|
|
* No `onResolved` — there's no live stream to write to here. */
|
|
runPreviewFinalize({
|
|
finalize,
|
|
fileId: fileMetadata.file_id,
|
|
previewRevision: result?.previewRevision,
|
|
});
|
|
return fileMetadata;
|
|
})().catch((error) => {
|
|
logger.error('Error processing code output:', error);
|
|
return null;
|
|
}),
|
|
);
|
|
}
|
|
const attachments = await Promise.all(artifactPromises);
|
|
toolCallData.attachments = attachments;
|
|
createToolCall(toolCallData).catch((error) => {
|
|
logger.error(`Error creating tool call: ${error.message}`);
|
|
});
|
|
res.status(200).json({
|
|
result: content,
|
|
attachments,
|
|
});
|
|
} catch (error) {
|
|
logger.error('Error calling tool', error);
|
|
res.status(500).json({ message: 'Error calling tool' });
|
|
}
|
|
};
|
|
|
|
const getToolCalls = async (req, res) => {
|
|
try {
|
|
const { conversationId } = req.query;
|
|
const toolCalls = await getToolCallsByConvo(conversationId, req.user.id);
|
|
res.status(200).json(toolCalls);
|
|
} catch (error) {
|
|
logger.error('Error getting tool calls', error);
|
|
res.status(500).json({ message: 'Error getting tool calls' });
|
|
}
|
|
};
|
|
|
|
module.exports = {
|
|
callTool,
|
|
getToolCalls,
|
|
verifyToolAuth,
|
|
};
|