LibreChat/api/server/services/Endpoints/agents/skillDeps.js
Danny Avila 20cd00c492
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
🖼️ feat: Return Sandbox Images From read_file as Viewable Artifacts (#14277)
* 🖼️ feat: Return Sandbox Images From `read_file` as Viewable Artifacts

The code-execution sandbox `read_file` path refused every image
extension because it reads files via `cat` over codeapi's JSON `/exec`
transport, which lossily corrupts non-UTF-8 bytes. The skill-file read
path already surfaced images as artifacts; this brings the sandbox path
to parity so an agent can actually see a chart/screenshot it reads.

- `readSandboxImage` (process.js): a Python base64 reader over `/exec`
  with an in-sandbox size guard so oversize images never cross the wire;
  base64 is ASCII-safe where `cat` corrupts.
- `handleSandboxImageRead` (handlers.ts): byte-integrity check (guards
  against a truncated `/exec` stdout), MIME resolved purely from the
  magic-byte sniff (extension only routes; a mislabeled non-image falls
  back to the bash hint), and graceful degradation on every failure mode.
- Shared `buildImageArtifactResult` used by both read paths; the result's
  `artifact.content` image_url reaches the UI (tool-end callbacks save it
  as an attachment) and the LLM (SDK folds it into the model-visible
  message for Anthropic/OpenAI/Google).

*  test: Sync read_file code-only description assertions with image wording

* 🛡️ fix: Harden sandbox image reads (regular-file guard, completeness check)

Addresses Codex review on PR #14277:

- readSandboxImage now os.stat's the target and rejects non-regular files
  (FIFOs, sockets, /dev/* symlinks) via stat.S_ISREG, and bounds the read at
  limit+1 bytes — a device/FIFO can no longer stream unbounded into memory
  until the request times out.
- handleSandboxImageRead validates completeness (not just the magic header):
  PNG must end with the IEND trailer and WebP's RIFF size must match the byte
  length, so a truncated/interrupted image degrades to the bash hint instead
  of being sent as a corrupt image_url. JPEG/GIF stay header-level (they can
  carry trailing metadata; a strict end-marker would risk false rejections).

* 🩹 fix: Chunk sandbox image reads to fit the runner stdout cap

Inlining any real image failed with "is an image file (.png) and cannot
be read as text". Root cause: readSandboxImage base64-encodes the file to
STDOUT, but the runner caps stdout at SANDBOX_OUTPUT_MAX_SIZE (1024 bytes
by default) and SIGKILLs the job on overflow (status OL), truncating the
JSON mid-base64. The parse then threw and the handler degraded to the
binary hint. The in-sandbox MAX_BINARY_BYTES=5MB guard never fired because
the *transport*, not the file size, is the real ceiling: a 5MB image needs
~6.8MB of stdout. Reproduced against a live MicroVM — a 186KB matplotlib
PNG died with 'stdout length exceeded' at exactly the 65536-byte cap.

Read the file in windows instead: each /exec pulls  raw bytes at an
offset and base64s only that slice, so every response stays under the cap
regardless of how the runner is configured; the chunks are reassembled and
verified against the sandbox-reported total. Verified end-to-end on a real
MicroVM: 25KB and 186KB PNGs both round-trip byte-exact (sha256 match).

Also:
- Detect the truncation explicitly (status OL) and name the fixable cause
  (chunk size / SANDBOX_OUTPUT_MAX_SIZE) instead of "unexpected output".
- Parse the LAST stdout line so a shell banner can't break the read, and
  include a stdout snippet when it genuinely is unparseable.
- LIBRECHAT_CODE_IMAGE_CHUNK_BYTES (default 32KB) tunes the window.
- Tests drive the real reader against a mocked /exec transport rather than
  mocking readSandboxImage, which is why the existing suite stayed green
  through this bug.

* 🎯 fix: Cap sandbox inline images at 1MB, separate from skill-file reads

The sandbox and skill-file image paths shared MAX_BINARY_BYTES (5MB), but
their transports differ: skill files stream from storage, while sandbox
bytes come back base64 over /exec stdout under the runner's output cap, so
the reader windows the file and cost scales in round-trips (~160 at 5MB vs
~32 at 1MB). Nothing is gained by allowing more — vision providers
downsample to ~1.5-2k px regardless, so multi-MB originals buy no fidelity
while grinding through round-trips.

Give the sandbox path its own MAX_SANDBOX_INLINE_IMAGE_BYTES (1MB), used
for both the read cap and the over-limit message (which previously quoted
5MB while the reader enforced something else). Skill-file reads keep 5MB.

Verified against a live MicroVM: a 186KB PNG round-trips byte-exact, and a
1.4MB file returns tooLarge in a single round-trip with zero bytes
transferred, degrading to the existing bash_tool hint.
2026-07-16 07:27:33 -04:00

386 lines
13 KiB
JavaScript

const crypto = require('crypto');
const { getStrategyFunctions } = require('~/server/services/Files/strategies');
const { batchUploadCodeEnvFiles } = require('~/server/services/Files/Code/crud');
const {
getSessionInfo,
checkIfActive,
readSandboxFile,
readSandboxImage,
writeSandboxFile,
} = require('~/server/services/Files/Code/process');
const {
checkAccess,
getStorageMetadata,
resolveRequestTenantId,
enrichWithSkillConfigurable,
mergeDeploymentSkillIds,
createDeploymentSkillMethods,
isDeploymentSkillFileSource,
getDeploymentSkillDownloadStream,
} = require('@librechat/api');
const {
Permissions,
FileContext,
ResourceType,
PermissionBits,
AccessRoleIds,
PrincipalType,
PermissionTypes,
isEphemeralAgentId,
} = require('librechat-data-provider');
const { checkPermission, grantPermission } = require('~/server/services/PermissionService');
const { getFileStrategy } = require('~/server/utils/getFileStrategy');
const db = require('~/models');
const deploymentSkillMethods = createDeploymentSkillMethods({
getSkillById: db.getSkillById,
getSkillByName: db.getSkillByName,
listSkillsByAccess: db.listSkillsByAccess,
listAlwaysApplySkills: db.listAlwaysApplySkills,
listSkillFiles: db.listSkillFiles,
getSkillFileByPath: db.getSkillFileByPath,
updateSkillFileContent: db.updateSkillFileContent,
updateSkillFileCodeEnvIds: db.updateSkillFileCodeEnvIds,
});
function getSkillDbMethods() {
return deploymentSkillMethods;
}
function withDeploymentSkillIds(ids = []) {
return mergeDeploymentSkillIds(ids);
}
function getSkillStrategyFunctions(source) {
if (isDeploymentSkillFileSource(source)) {
return {
getDownloadStream: (_req, filepath) => getDeploymentSkillDownloadStream(filepath),
};
}
return getStrategyFunctions(source);
}
function resolveSkillStorage(req, { isImage = false } = {}) {
const source = getFileStrategy(req.config, { context: FileContext.skill_file, isImage });
const strategy = getStrategyFunctions(source);
if (!strategy.saveBuffer) {
throw new Error(`Storage backend "${source}" does not support file writes`);
}
return { saveBuffer: strategy.saveBuffer, source };
}
function basename(relativePath) {
const slash = relativePath.lastIndexOf('/');
return slash === -1 ? relativePath : relativePath.slice(slash + 1);
}
async function saveSkillFileContent({ req, skillId, relativePath, content, mimeType }) {
const existingFile = await db.getSkillFileByPath(skillId, relativePath);
const tenantId = resolveRequestTenantId(req);
const fileId = crypto.randomUUID();
const filename = basename(relativePath);
const storageFileName = `${fileId}__${filename}`;
const buffer = Buffer.from(content, 'utf8');
const storage = resolveSkillStorage(req, { isImage: mimeType.startsWith('image/') });
const filepath = await storage.saveBuffer({
userId: req.user.id,
buffer,
fileName: storageFileName,
basePath: 'uploads',
tenantId,
});
const storageMetadata = getStorageMetadata({ filepath, source: storage.source });
let result;
try {
result = await db.upsertSkillFile({
skillId,
relativePath,
file_id: fileId,
filename,
filepath,
...storageMetadata,
source: storage.source,
mimeType,
bytes: buffer.length,
isExecutable: false,
author: req.user._id ?? req.user.id,
tenantId,
});
if (!result) {
const error = new Error('Skill file save failed to persist metadata');
error.code = 'SKILL_FILE_UPSERT_NOT_FOUND';
throw error;
}
} catch (error) {
const { deleteFile } = getStrategyFunctions(storage.source);
if (deleteFile) {
await deleteFile(req, { filepath, user: req.user.id, tenantId }).catch(() => undefined);
}
throw error;
}
if (existingFile && existingFile.filepath !== filepath) {
const { deleteFile } = getStrategyFunctions(existingFile.source);
if (deleteFile) {
deleteFile(req, {
filepath: existingFile.filepath,
storageKey: existingFile.storageKey,
storageRegion: existingFile.storageRegion,
user: existingFile.author ?? req.user.id,
tenantId: existingFile.tenantId ?? tenantId,
}).catch(() => undefined);
}
}
return { bytes: result.bytes, relativePath: result.relativePath };
}
function canCreateSkill({ req }) {
return checkAccess({
req,
user: req.user,
permissionType: PermissionTypes.SKILLS,
permissions: [Permissions.USE, Permissions.CREATE],
getRoleByName: db.getRoleByName,
});
}
function canEditSkill({ req, skillId }) {
return checkPermission({
userId: req.user.id,
role: req.user.role,
resourceType: ResourceType.SKILL,
resourceId: skillId,
requiredPermission: PermissionBits.EDIT,
});
}
function isAgentSkillsEnabledForRun({ agent, skillsCapabilityEnabled, ephemeralSkillsToggle }) {
if (!skillsCapabilityEnabled) {
return false;
}
if (isEphemeralAgentId(agent.id)) {
if (agent.skills_enabled === false) {
return false;
}
if (agent.skills_enabled === true) {
return true;
}
return ephemeralSkillsToggle === true;
}
return agent.skills_enabled === true;
}
function canAuthorSkillFiles({
agent,
scopedEditableSkillIds = [],
skillCreateAllowed,
skillsCapabilityEnabled,
ephemeralSkillsToggle,
}) {
return (
isAgentSkillsEnabledForRun({ agent, skillsCapabilityEnabled, ephemeralSkillsToggle }) &&
(scopedEditableSkillIds.length > 0 || skillCreateAllowed === true)
);
}
function grantSkillOwner({ req, skillId }) {
return grantPermission({
principalType: PrincipalType.USER,
principalId: req.user.id,
resourceType: ResourceType.SKILL,
resourceId: skillId,
accessRoleId: AccessRoleIds.SKILL_OWNER,
grantedBy: req.user.id,
});
}
function getAuthorSkillByName({ req, name }) {
const author = req.user?._id ?? req.user?.id;
if (!author) {
return null;
}
return db.getAuthorSkillByName({
name,
author,
tenantId: resolveRequestTenantId(req),
});
}
/**
* Builds the `skillPrimedIdsByName` map threaded through
* `buildAgentToolContext`. Centralized here so every runtime route shares
* one source of truth — if `ResolvedManualSkill` ever renames `_id` or
* gains new identifying fields, only this helper changes.
*
* Combines both manual (`$`-popover) primes AND always-apply primes so
* `read_file` can:
* - Relax the `disable-model-invocation: true` gate for either source
* (the body is already in context; blocking its own files would be
* nonsensical).
* - Pin same-name collision lookups to the exact `_id` the resolver
* primed (otherwise a newer same-name duplicate could shadow the
* body/file pair within a single turn).
*
* On the rare overlap (a name appears in both arrays because upstream
* dedup was skipped), manual wins — manual invocation is explicit user
* intent and carries the authoritative `_id` for this turn.
*
* Returns `undefined` (not `{}`) when both arrays are empty, so the
* downstream `enrichWithSkillConfigurable` cleanly omits the field from
* `mergedConfigurable` rather than threading an empty object.
*
* @param {Array<{ name: string, _id: { toString(): string } }> | undefined} manualSkillPrimes
* @param {Array<{ name: string, _id: { toString(): string } }> | undefined} alwaysApplySkillPrimes
* @returns {Record<string, string> | undefined}
*/
function buildSkillPrimedIdsByName(manualSkillPrimes, alwaysApplySkillPrimes) {
const manualCount = manualSkillPrimes?.length ?? 0;
const alwaysApplyCount = alwaysApplySkillPrimes?.length ?? 0;
if (manualCount === 0 && alwaysApplyCount === 0) {
return undefined;
}
const out = {};
/* Order matters on the edge case where the same name appears in both
lists: always-apply goes in first, then manual overwrites — manual
wins because it's explicit user intent for this turn. */
if (alwaysApplyCount > 0) {
for (const p of alwaysApplySkillPrimes) {
out[p.name] = p._id.toString();
}
}
if (manualCount > 0) {
for (const p of manualSkillPrimes) {
out[p.name] = p._id.toString();
}
}
return out;
}
/**
* Builds the per-agent context consumed by ON_TOOL_EXECUTE. Keeping this
* shape in one Adapter gives every runtime path the same configurable
* fields and the same primed-skill pinning behavior.
*
* @param {object} params
* @param {object} params.agent
* @param {object} params.config
* @param {Record<string, import('@librechat/api').LCAvailableTools>} [params.config.mcpAvailableTools]
* @param {import('@librechat/api').RequestScopedMCPConnectionStore} [params.config.requestScopedConnections]
* @returns {object}
*/
function buildAgentToolContext({ agent, config }) {
return {
agent,
/** Per-agent resolved endpoint token/pricing config. Retained here because
* `agentToolContexts` is the one map that holds every agent — including
* pure subagents pruned from `agentConfigs` — so usage can be priced with
* the producing agent's config in multi-endpoint graphs. */
endpointTokenConfig: config.endpointTokenConfig,
toolRegistry: config.toolRegistry,
backgroundToolNames: config.backgroundToolNames,
mcpAvailableTools: config.mcpAvailableTools,
requestScopedConnections: config.requestScopedConnections,
userMCPAuthMap: config.userMCPAuthMap,
tool_resources: config.tool_resources,
actionsEnabled: config.actionsEnabled,
accessibleSkillIds: config.accessibleSkillIds,
activeSkillNames: config.activeSkillNames,
codeEnvAvailable: config.codeEnvAvailable,
skillAuthoringAvailable: config.skillAuthoringAvailable,
fileAuthoringToolNames: config.fileAuthoringToolNames,
skillPrimedIdsByName:
buildSkillPrimedIdsByName(config.manualSkillPrimes, config.alwaysApplySkillPrimes) ?? {},
};
}
function hasOwn(value, key) {
return Object.prototype.hasOwnProperty.call(value ?? {}, key);
}
/**
* Applies per-agent runtime context to a loadToolsForExecution result.
*
* @param {object} params
* @param {{ loadedTools: unknown[], configurable?: Record<string, unknown> }} params.result
* @param {object} params.req
* @param {object | undefined} params.ctx
* @param {object | undefined} [params.fallback]
* @returns {{ loadedTools: unknown[], configurable: Record<string, unknown> }}
*/
function enrichLoadedToolsWithAgentContext({ result, req, ctx = {}, fallback = {} }) {
const codeEnvAvailable = hasOwn(ctx, 'codeEnvAvailable')
? ctx.codeEnvAvailable === true
: fallback.codeEnvAvailable === true;
const skillAuthoringAvailable = hasOwn(ctx, 'skillAuthoringAvailable')
? ctx.skillAuthoringAvailable === true
: fallback.skillAuthoringAvailable === true;
return enrichWithSkillConfigurable({
result,
context: {
req,
codeEnvAvailable,
accessibleSkillIds: ctx.accessibleSkillIds ?? fallback.accessibleSkillIds,
skillPrimedIdsByName: ctx.skillPrimedIdsByName ?? fallback.skillPrimedIdsByName,
activeSkillNames: ctx.activeSkillNames ?? fallback.activeSkillNames,
skillAuthoringAvailable,
fileAuthoringToolNames: ctx.fileAuthoringToolNames ?? fallback.fileAuthoringToolNames,
},
});
}
/** Skill-related properties for ToolExecuteOptions (stable references, allocated once). */
const skillToolDeps = {
getSkillByName: deploymentSkillMethods.getSkillByName,
getAuthorSkillByName,
createSkill: db.createSkill,
updateSkill: db.updateSkill,
deleteSkill: db.deleteSkill,
canCreateSkill,
canEditSkill,
grantSkillOwner,
saveSkillFileContent,
listSkillFiles: deploymentSkillMethods.listSkillFiles,
getStrategyFunctions: getSkillStrategyFunctions,
batchUploadCodeEnvFiles,
getSessionInfo,
checkIfActive,
updateSkillFileCodeEnvIds: deploymentSkillMethods.updateSkillFileCodeEnvIds,
getSkillFileByPath: deploymentSkillMethods.getSkillFileByPath,
updateSkillFileContent: deploymentSkillMethods.updateSkillFileContent,
/**
* `read_file` falls back to a sandbox `cat` for `/mnt/data/...` paths
* and for `{firstSegment}/...` paths whose first segment isn't a known
* skill name. The handler routes through this when the agent has code
* execution enabled; the codeapi base URL comes from
* `LIBRECHAT_CODE_BASEURL` and the sandbox session id is forwarded by
* the agents-side `ToolNode` via `tc.codeSessionContext`.
*/
readSandboxFile,
/**
* Companion to `readSandboxFile` for the raster-image case: pulls the
* bytes base64-encoded (size-guarded in-sandbox) so `read_file` can
* return an image the model can see instead of refusing it as binary.
*/
readSandboxImage,
writeSandboxFile,
};
function getSkillToolDeps() {
return skillToolDeps;
}
module.exports = {
getSkillToolDeps,
canAuthorSkillFiles,
isAgentSkillsEnabledForRun,
getSkillDbMethods,
withDeploymentSkillIds,
getSkillStrategyFunctions,
enrichWithSkillConfigurable,
buildSkillPrimedIdsByName,
buildAgentToolContext,
enrichLoadedToolsWithAgentContext,
};