mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-10-25 11:47:32 +00:00
🧱 refactor: typed CodeEnvRef + kind discriminator + principal-aware sandbox cache (#12960)
* 🧱 refactor: typed CodeEnvRef + kind discriminator + tenant-aware sandbox cache Final cutover for the LibreChat ↔ codeapi sandbox file identity. Replaces the magic string `${session_id}/${file_id}?entity_id=...` with a typed, discriminated `CodeEnvRef`. Pre-release lockstep deploy with codeapi #1455 and agents #148; no legacy aliases retained. ## Final shape ```ts type CodeEnvRef = | { kind: 'skill'; id: string; storage_session_id: string; file_id: string; version: number } | { kind: 'agent'; id: string; storage_session_id: string; file_id: string } | { kind: 'user'; id: string; storage_session_id: string; file_id: string }; ``` `kind` drives codeapi's sessionKey: `<tenant>:<kind>:<id>[✌️<version>]` for shared kinds, `<tenant>:user:<userId>` for user-private (auth context provides `userId`). `version` is statically required for `kind: 'skill'` and forbidden otherwise via discriminated union — constraint holds at compile time on every consumer, not just codeapi's runtime validator. `id` is sessionKey-meaningful for `'skill'` / `'agent'`; informational only for `'user'` (codeapi resolves user identity from auth context). ## What changed - `packages/data-provider/src/codeEnvRef.ts` — discriminated union + `CODE_ENV_KINDS` const-tuple keeps the runtime list and TS union locked together. - Schemas: `metadata.codeEnvRef` and `SkillFile.codeEnvRef` enums tightened to `['skill', 'agent', 'user']`. - `primeSkillFiles` writes `kind: 'skill'`, `id: skill._id`, `version: skill.version`. Cache-hit path reads `codeEnvRef` directly. Bumping `skill.version` on edit naturally invalidates the prior cache entry under the new sessionKey. - `processCodeOutput` writes `kind: 'user'`, `id: req.user.id`. Output bucket is always user-scoped, regardless of which skill the execution invoked. New regression test pins the asymmetry. - `primeFiles` reupload preserves `kind`/`id`/`version?` from the existing ref so a skill-cache-miss reupload doesn't silently demote to user bucket. - `crud.js` upload functions (`uploadCodeEnvFile` / `batchUploadCodeEnvFiles`) thread `kind`/`id`/`version?` to the multipart form (codeapi #1455 option α). Without these on the wire, codeapi falls back to user bucketing and skill-cache invalidation never fires. Client-side validation mirrors codeapi's validator. - `Files/process.js` — chat attachments use `kind: 'user'`; agent setup files use `kind: 'agent'`. - Drops `entity_id` everywhere (struct, schema sub-docs, write paths, upload form fields). Drops `'system'` from the kind enum (no emitter ever existed). ## Test plan - [x] `cd packages/data-provider && npx jest src/codeEnvRef.spec` — 4 / 4 - [x] `cd packages/data-schemas && npx jest` — 1447 / 1447 - [x] `cd packages/api && npx jest src/agents` — 81 / 81 in skillFiles + handlers + resources - [x] `cd api && npx jest server/services/Files server/controllers/agents` — 436 / 436 - [x] `cd api && npx jest server/services/Files/Code` — 98 / 98 (incl. new "outputs are user-scoped regardless of which skill the execution invoked" regression and "reupload forwards kind/id/version from existing ref") - [x] `npx tsc --noEmit -p packages/data-{provider,schemas}/tsconfig.json && npx tsc --noEmit -p packages/api/tsconfig.json` — clean (only pre-existing unrelated dev errors in storage/balance, untouched here) ## Deploy notes - **24h cache-miss burst** on first deploy. Inputs (skill caches re-prime under new sessionKey shape) and outputs (any pre-Phase C skill-output cached files become unreadable). Bounded by codeapi's 24h TTL. - **Lockstep with codeapi #1455 and agents #148.** Either repo can land first since no aliases to drain, but the three deploys must overlap within the same maintenance window. - **`@librechat/agents` bump to `3.1.79-dev.0`** required after agents #148 lands and is published. ## What this enables Auth bridge work (JWT-based tenant/user identity between LC and codeapi) — codeapi now derives sessionKey purely from `req.codeApiAuthContext.{ tenantId, userId}`, so the next chapter is replacing the header-asserted user identity with a verified-claim path. * 🩹 fix: persist execute_code uploads under codeEnvRef metadata key Codex review P1 (chatgpt-codex-connector). `Files/process.js` was storing the upload result under `metadata.fileIdentifier` even though: - `uploadCodeEnvFile` now returns `{ storage_session_id, file_id }`, not the legacy magic string. - The post-cutover schema (`File.metadata.codeEnvRef`) only declares `codeEnvRef` — mongoose strict mode silently strips unknown keys. - All readers (`primeFiles`, `getCodeFilesByIds`, `categorizeFileForToolResources`, controller filtering) check `metadata.codeEnvRef`. Net effect of the bug: chat-attached and agent-setup execute_code files would lose their sandbox reference on save, and primeFiles would skip them on subsequent code-execution turns — the file blob would still be available locally but never re-mounted in the sandbox. Fix: construct the full `CodeEnvRef` (`{ kind, id, storage_session_id, file_id }`) at the write site and persist under `metadata.codeEnvRef`. `BaseClient`'s "is this a code-env file" presence check accepts the new shape alongside the legacy `fileIdentifier` for back-compat with any pre-cutover records still in the database. Mirrors the same change in `processAttachments.spec.ts` (which re-implements the BaseClient logic for testability). New regression tests in `process.spec.js` cover three cases: - chat attachments (`messageAttachment=true`) → `kind: 'user'` - agent setup (`messageAttachment=false`) → `kind: 'agent'` - legacy `fileIdentifier` key is NOT persisted (would be schema-stripped) * 🩹 fix: read storage_session_id on primed file refs (Codex P1) Codex review (chatgpt-codex-connector). After Phase B's per-file `session_id` → `storage_session_id` rename, `primeFiles` emits the new field — but `seedCodeFilesIntoSessions` was still reading `files[0].session_id` for the representative session and `f.session_id` for the dedupe key. In runs with only primed attachments (no skill seed), `representativeSessionId` was `undefined`, the function returned the unchanged map, and `seedCodeFilesIntoSessions` silently dropped the entire batch. The first `execute_code` call then started without `_injected_files` and the agent couldn't see prior-turn artifacts. Fix: - `codeFilesSession.ts`: read `f.storage_session_id` for both the dedupe key and the representative session id. JSDoc updated to match the new field name. - `callbacks.js`: the two output-file persistence paths read `file.session_id` to pass to `processCodeOutput` — switch to `file.storage_session_id`. The original comment explicitly says this should be the STORAGE session, which is exactly the field Phase B renamed. - `codeFilesSession.spec.ts`: fixture builder uses `storage_session_id` and `kind: 'user'` to match the post-cutover `CodeEnvFile` shape. Lockstep coordination: this matches the post-bump shape of `@librechat/agents` 3.1.79+. CI tsc errors against the currently-pinned 3.1.78 are expected and resolve when the dep bumps in this PR before merge. * 📦 chore: Bump `@librechat/agents` to version 3.1.80-dev.0 in package-lock and package.json files * 🪪 fix: thread kind/id/version through codeapi /download URLs (Phase C α) Symmetric fix for the upload-side wire change in 537725a. Codeapi's `sessionAuth` middleware now requires `kind`/`id`/`version?` on every download/freshness URL — without them it 400s with "kind must be one of: skill, agent, user" before serving the file. Three sites construct codeapi-side URLs that go through `sessionAuth`: - `processCodeOutput` (`Files/Code/process.js`): `/download/<sess>/<id>` for freshly-generated sandbox outputs. Always `kind: 'user'` + `id: req.user.id` — code-output files are always user-private, regardless of which skill the run invoked. - `getSessionInfo` (`Files/Code/process.js`): `/sessions/<sess>/objects/<id>` for the 23h freshness check. Pulls kind/id/version straight off the `codeEnvRef` already in scope — skill files stay skill-bucketed, user files stay user-bucketed. - `/code/download/:session_id/:fileId` LC route (`routes/files/files.js`): proxies to codeapi for manual downloads. Code-output files only on this route, so `kind: 'user'` + `id: req.user.id`. The `getCodeOutputDownloadStream` helper in `crud.js` now takes an `identity` param, validated by a `buildCodeEnvDownloadQuery` helper that mirrors `appendCodeEnvFileIdentity`'s shape rules: kind required from the closed `{skill, agent, user}` set, version required for 'skill' and forbidden otherwise. Bad callers fail fast on the client instead of round-tripping a 400. Also cleans up two log-noise sources reported alongside the 400: - `logAxiosError` in `packages/api/src/utils/axios.ts` was dumping `error.response.data` raw. With `responseType: 'arraybuffer'` that's a `Buffer` (~4 chars per byte after JSON-serialization); with `responseType: 'stream'` it's a `Readable` whose internal state serializes the entire ring buffer + socket. New `renderResponseData` decodes small buffers as UTF-8 (truncated past 2KB) and stubs streams as `'[stream]'`. Diagnostics stay useful, log lines stop being megabytes. - `/code/download` route's catch was bare `logger.error('...', error)`, bypassing the redactor. Switched to `logAxiosError` so it benefits from the same buffer/stream handling. Tests updated to match the new contract: - crud.spec: `getCodeOutputDownloadStream` fixtures pass `userIdentity`; new cases cover skill identity (with version), bad kind rejection, skill-without-version rejection. - process.spec: `getSessionInfo` test passes a full `codeEnvRef` object. * ♻️ refactor: extract codeEnv identity helpers into packages/api Per the project convention that new backend code lives in TypeScript under `packages/api`, moves `appendCodeEnvFileIdentity` and `buildCodeEnvDownloadQuery` from `api/server/services/Files/Code/crud.js` into a new `packages/api/src/files/code/identity.ts` module. Both helpers are pure validators that mirror codeapi's `parseUploadSessionKeyInput` server-side rules (closed kind set, `version` required for `'skill'` and forbidden otherwise) — they deserve TS support and a dedicated spec rather than living as JSDoc-typed helpers in the legacy `/api` workspace. The new module: - Exports a `CodeEnvIdentity` interface using the `librechat-data-provider` `CodeEnvKind` discriminated union. - Adds 13 unit tests in `identity.spec.ts` covering the validation matrix (skill+version, agent, user, and every rejection path) plus URL encoding for the download query. - Re-exported from `packages/api/src/files/code/index.ts` alongside `classify`, `extract`, and `form`. Consumer updates: - `api/server/services/Files/Code/crud.js`: drops the local helpers and imports them from `@librechat/api`. Net -64 lines. - `api/server/services/Files/Code/process.js`: same. - Test mocks for `@librechat/api` in three spec files now stub the helpers' validation behavior locally rather than pulling them through `requireActual` (which would drag in provider-config init-time side effects). The package's `exports` field only surfaces the root barrel, so leaf imports aren't reachable from legacy `/api` test setup. No runtime behavior change. Identity validation rules and emitted form/query shapes are byte-for-byte identical pre/post. * 🪪 fix: emit resource_id alongside id on _injected_files (skill 403 fix) Companion to codeapi #1455 fix and agents 3.1.80-dev.1 — the wire shape for shared-kind files now requires `resource_id` distinct from the storage `id`. Without this LC change, codeapi's sessionKey re-derivation on every shared-kind /exec rejects with 403 session_key_mismatch: cached: legacy:skill:69dcf561...✌️59 (signed at upload, skill _id) derived: legacy:skill:ysPwEURuPk-...✌️59 (storage nanoid) Emit sites updated: - `primeInvokedSkills` cache-hit path: `resource_id: ref.id` (the persisted skill `_id` from `codeEnvRef.id`); `id: ref.file_id` unchanged (storage uuid). - `primeInvokedSkills` fresh-upload path: `resource_id: skill._id.toString()` on every primed file (the `allPrimedFiles` builder type now carries the field). - `processCodeOutput`'s `pushFile` (Code/process.js): `resource_id: ref.id` — for `kind: 'user'` this is informational (codeapi derives sessionKey from auth context) but emitted for shape uniformity with shared kinds. Bumps `@librechat/agents` to `^3.1.80-dev.1` (the version that ships the matching `CodeEnvFile.resource_id` field). ## Test plan - [x] `cd packages/api && npx jest src/agents` — 67 / 67 pass (skillFiles fixtures updated to assert `resource_id` on the emitted CodeSessionContext.files). - [x] `cd api && npx jest server/services/Files server/controllers/agents` — 445 / 445 pass (process.spec fixtures updated for the reupload + cache-hit emission). - [x] `npx tsc --noEmit -p packages/api/tsconfig.json` — clean. * fix(skill-tool-call): carry resource_id through primeSkillFiles → artifact Codeapi was 400ing every /exec following a `handle_skill` tool call with `resource_id is invalid` (`type: 'undefined'`). Both code paths in `primeSkillFiles` (cache-hit + fresh-upload) returned files without `resource_id`/`kind`/`version`, and the artifact in `handlers.ts` forwarded the stripped shape into `tc.codeSessionContext.files` → `_injected_files`. `primeInvokedSkills` (the NL-detected loader) had already been fixed end-to-end; this commit aligns the tool-invoked path with the same contract: `resource_id` = `skill._id.toString()`, `kind: 'skill'`, `version` = the skill's monotonic counter. Tests added to `skillFiles.spec.ts` lock the contract on `primeSkillFiles` directly so future refactors can't silently drop the resource identity again. * fix(handlers.spec): align session_id → storage_session_id rename + kind discriminator Pre-existing TS errors against the post-rename `CodeEnvFile` shape: the test file still used `session_id` on per-file objects (renamed to `storage_session_id` in agents Phase B/C) and was missing the `kind` discriminator the discriminated union requires. Both inputs and the matching `expect.toEqual(...)` mirrors updated together so the runtime equality check still holds. Lines 723-732 stay as-is — they sit behind `as unknown as ToolCallRequest` and TS already skipped them. * chore: fix `@librechat/agents`, correct version to 3.1.80-dev.0 in package.json files * chore: bump `@librechat/agents` to version 3.1.80-dev.1 in package.json and package-lock.json * chore: bump `@librechat/agents` to version 3.1.80-dev.2 * feat(observability): trace file priming chain from primeCodeFiles to _injected_files Diagnosing the user-upload "files=[] on first /exec" bug requires seeing where in the LC chain a file ref disappears. Prior to this patch the chain (primeCodeFiles → primedCodeFiles → initialSessions → CodeSessionContext → _injected_files) was opaque end-to-end: - primeCodeFiles silently dropped files without `metadata.codeEnvRef` - reuploadFile catches all errors and continues with no signal - the handlers.ts handoff to codeapi never logged what it was sending After this patch, a single grep on `[primeCodeFiles]` plus `[code-env:inject]` shows the full per-file path: [primeCodeFiles] in: file_ids=N resourceFiles=M [primeCodeFiles] file=<id> path=skip reason=no-codeenvref filename=... [primeCodeFiles] file=<id> path=cache-hit-by-session storage_session_id=... [primeCodeFiles] file=<id> path=reupload reason=no-uploadtime ... [primeCodeFiles] file=<id> path=reupload reason=stale ... [primeCodeFiles] file=<id> path=reupload-success oldSession=... newSession=... newFileId=... [primeCodeFiles] file=<id> path=reupload-failed session=... [primeCodeFiles] file=<id> path=fresh-active storage_session_id=... [primeCodeFiles] out: returned=N skippedNoRef=M reuploadFailures=K [code-env:inject] tool=<name> files=N missingResourceId=K (debug) [code-env:inject] M/N files missing resource_id ... (warn) [code-env:inject] tool=<name> _injected_files=0 ... (warn) The boundary log warns when LC sends zero injected files on a code-execution tool call — that's the user's actual symptom showing up at the LC side instead of having to correlate against codeapi's `Request received { files: [] }`. Tag chosen as `[code-env:inject]` rather than `[handoff:exec]` to avoid collision with the app-level "handoff" semantic (subagent handoff workflow). Structural cleanup in primeFiles: replaced the `if (ref) { ... }` nesting with an early `if (!ref) continue` so the per-path instrumentation hooks land at top-level scope instead of indented inside a conditional. Behavior unchanged; pushFile / reuploadFile identical. Spec fixtures (handlers.spec.ts, codeFilesSession.spec.ts) updated to include `resource_id` on `CodeEnvFile` literals — required by the post-3.1.80-dev.2 type now installed. ## Test plan - [x] `cd packages/api && npx jest src/agents/handlers.spec.ts src/agents/codeFilesSession.spec.ts src/agents/skillFiles.spec.ts` — 69/69 pass - [x] `cd api && npx jest server/services/Files/Code/process.spec.js` — 84/84 pass - [x] `npx tsc --noEmit -p packages/api` — clean - [x] `npx eslint` on all four touched files — clean * chore: add CONSOLE_JSON_STRING_LENGTH to .env.example for JSON log string length configuration * fix(files): align codeapi upload filename with LC's sanitized DB filename User-attached files for code execution were uploading to codeapi under `file.originalname` (raw upload filename, may contain spaces / special chars) while LC's DB record stored the sanitized form (`sanitizeFilename(file.originalname)`, underscores). Codeapi preserves whatever filename the upload sent, so the sandbox saw `/mnt/data/<originalname>` while LC's `primeFiles` toolContext text + `_injected_files.name` referenced `file.filename` (sanitized). Visible failure: agent gets system prompt saying /mnt/data/librechat_code_api_-_active_customer_-_2025-11-05.xlsx …tries that path, hits `FileNotFoundError`, then notices the sandbox's actual `Available files` line says /mnt/data/librechat code api - active customer - 2025-11-05.xlsx …retries with spaces, succeeds. Wastes a tool call per upload and leaks raw filenames into model context. Fix: sanitize once and use the sanitized form in both the codeapi upload AND the LC DB record. Sandbox path = LC toolContext text = in-memory ref name. No drift. Reupload path (`Code/process.js` line 867 `filename: file.filename`) already uses the sanitized DB name, so it stays consistent with the fresh-upload path after this change. ## Test plan - [x] `cd api && npx jest server/services/Files/process` — 32/32 pass - [x] `npx eslint` on the touched file — clean * chore: bump `@librechat/agents` to version 3.1.80-dev.3 in package.json and package-lock.json
This commit is contained in:
Danny Avila
parent
9441563b95
commit
93c4ef4ba8
40 changed files with 1937 additions and 456 deletions
|
|
@ -6,10 +6,19 @@ import {
|
|||
type CodeFilesAgent,
|
||||
} from './codeFilesSession';
|
||||
|
||||
const file = (id: string, session_id: string, name: string): CodeEnvFile => ({
|
||||
const file = (id: string, storage_session_id: string, name: string): CodeEnvFile => ({
|
||||
id,
|
||||
session_id,
|
||||
/* `resource_id` is informational for `kind: 'user'` (codeapi derives
|
||||
* the sessionKey from the auth-context user, not this field) but
|
||||
* still required by the type for shape uniformity. The id stand-in
|
||||
* is fine here because no test asserts on its value. */
|
||||
resource_id: id,
|
||||
storage_session_id,
|
||||
name,
|
||||
/* `kind` drives codeapi's sessionKey shape; defaults to `'user'` for
|
||||
* primed code-execution attachments at the test boundary (most are
|
||||
* chat uploads). Tests that need shared kinds set it explicitly. */
|
||||
kind: 'user',
|
||||
});
|
||||
|
||||
describe('seedCodeFilesIntoSessions', () => {
|
||||
|
|
@ -75,13 +84,18 @@ describe('seedCodeFilesIntoSessions', () => {
|
|||
expect(entry.session_id).toBe('skill-sess');
|
||||
});
|
||||
|
||||
it('skips seeding when incoming files have no session_id (defensive)', () => {
|
||||
const fileWithoutSession = { id: 'x', session_id: '', name: 'orphan.csv' } as CodeEnvFile;
|
||||
it('skips seeding when incoming files have no storage_session_id (defensive)', () => {
|
||||
const fileWithoutSession = {
|
||||
id: 'x',
|
||||
storage_session_id: '',
|
||||
name: 'orphan.csv',
|
||||
kind: 'user',
|
||||
} as CodeEnvFile;
|
||||
const result = seedCodeFilesIntoSessions([fileWithoutSession], undefined);
|
||||
expect(result).toBeUndefined();
|
||||
});
|
||||
|
||||
it('dedupes incoming files that share session_id + id with existing entries', () => {
|
||||
it('dedupes incoming files that share storage_session_id + id with existing entries', () => {
|
||||
/**
|
||||
* Regression for Codex review #2: shared conversation files commonly
|
||||
* appear in multiple agents' `primedCodeFiles`. Without dedupe,
|
||||
|
|
@ -128,7 +142,7 @@ describe('seedCodeFilesIntoSessions', () => {
|
|||
const result = seedCodeFilesIntoSessions([a, a, b], undefined);
|
||||
const entry = result!.get(Constants.EXECUTE_CODE) as CodeSessionContext;
|
||||
expect(entry.files).toHaveLength(2);
|
||||
expect(entry.files!.map((f) => f.session_id).sort()).toEqual(['sess-A', 'sess-B']);
|
||||
expect(entry.files!.map((f) => f.storage_session_id).sort()).toEqual(['sess-A', 'sess-B']);
|
||||
});
|
||||
});
|
||||
|
||||
|
|
@ -267,8 +281,8 @@ describe('buildInitialToolSessions', () => {
|
|||
expect(entry.session_id).toBe('sess-PRIMARY');
|
||||
/* All three agents still contributed their files into the merged set. */
|
||||
expect(entry.files!.map((f) => f.name).sort()).toEqual(['a.txt', 'b.txt', 'top.txt']);
|
||||
/* And the per-file session_ids are preserved (ToolNode injects per-file). */
|
||||
const byName = new Map(entry.files!.map((f) => [f.name, f.session_id]));
|
||||
/* And the per-file storage_session_ids are preserved (ToolNode injects per-file). */
|
||||
const byName = new Map(entry.files!.map((f) => [f.name, f.storage_session_id]));
|
||||
expect(byName.get('top.txt')).toBe('sess-PRIMARY');
|
||||
expect(byName.get('a.txt')).toBe('sess-A');
|
||||
expect(byName.get('b.txt')).toBe('sess-B');
|
||||
|
|
|
|||
|
|
@ -22,21 +22,24 @@ export interface CodeFilesAgent {
|
|||
* after the first call returns one, so primed files were silently dropped.
|
||||
*
|
||||
* Files from `primeFiles` (api/server/services/Files/Code/process.js) carry
|
||||
* per-file `session_id`s. The map's representative `session_id` is taken from
|
||||
* the first incoming file (matching `primeInvokedSkills`); per-file ids on the
|
||||
* `files` array are what `ToolNode` actually uses (`file.session_id ?? codeSession.session_id`).
|
||||
* per-file `storage_session_id`s (the long-lived storage bucket id). The
|
||||
* map's representative top-level `session_id` is seeded from the first
|
||||
* incoming file's `storage_session_id` (matching `primeInvokedSkills`),
|
||||
* since no execution session exists yet at seed time. Per-file ids on the
|
||||
* `files` array are what `ToolNode` actually uses
|
||||
* (`file.storage_session_id ?? execSessionId`).
|
||||
*
|
||||
* When an entry already exists (e.g. seeded by `primeInvokedSkills` for skill
|
||||
* files), incoming files are appended after the existing ones. The pre-existing
|
||||
* representative `session_id` is preserved so a partial-cache/fresh-prime
|
||||
* collision doesn't shift which session id `ToolNode` picks for the call.
|
||||
*
|
||||
* Files are deduplicated by `session_id + id` as the stable identity key.
|
||||
* Multiple agents in the same run commonly carry the same primed
|
||||
* code-execution resources (shared conversation files), and without dedupe
|
||||
* `_injected_files` would grow proportionally to agent count and inflate
|
||||
* every `/exec` POST. First-seen wins so the original ordering / source
|
||||
* is preserved.
|
||||
* Files are deduplicated by `storage_session_id + id` as the stable
|
||||
* identity key. Multiple agents in the same run commonly carry the same
|
||||
* primed code-execution resources (shared conversation files), and without
|
||||
* dedupe `_injected_files` would grow proportionally to agent count and
|
||||
* inflate every `/exec` POST. First-seen wins so the original ordering /
|
||||
* source is preserved.
|
||||
*/
|
||||
export function seedCodeFilesIntoSessions(
|
||||
files: CodeEnvFile[] | undefined,
|
||||
|
|
@ -50,15 +53,15 @@ export function seedCodeFilesIntoSessions(
|
|||
const prior = sessions.get(Constants.EXECUTE_CODE) as CodeSessionContext | undefined;
|
||||
|
||||
/**
|
||||
* Compose `(session_id, id)` as a stable identity. `name` alone isn't
|
||||
* sufficient — two distinct primed uploads can share a filename
|
||||
* (different sessions, different file_ids). The composite stays
|
||||
* Compose `(storage_session_id, id)` as a stable identity. `name` alone
|
||||
* isn't sufficient — two distinct primed uploads can share a filename
|
||||
* (different storage sessions, different file_ids). The composite stays
|
||||
* cheap to compute and the keys are short uuids.
|
||||
*/
|
||||
const seenKeys = new Set<string>();
|
||||
const mergedFiles: FileRefs = [];
|
||||
const pushIfFresh = (f: { id?: string; session_id?: string; name?: string }): void => {
|
||||
const key = `${f.session_id ?? ''}\0${f.id ?? ''}`;
|
||||
const pushIfFresh = (f: { id?: string; storage_session_id?: string; name?: string }): void => {
|
||||
const key = `${f.storage_session_id ?? ''}\0${f.id ?? ''}`;
|
||||
if (seenKeys.has(key)) return;
|
||||
seenKeys.add(key);
|
||||
mergedFiles.push(f as FileRefs[number]);
|
||||
|
|
@ -68,7 +71,14 @@ export function seedCodeFilesIntoSessions(
|
|||
}
|
||||
for (const f of files) pushIfFresh(f);
|
||||
|
||||
const representativeSessionId = prior?.session_id ?? files[0].session_id;
|
||||
/* Representative top-level `session_id` for the seed CodeSessionContext.
|
||||
* No execution session exists yet at seed time, so the first incoming
|
||||
* file's `storage_session_id` stands in until the first `/exec` call
|
||||
* returns a real execution session id. ToolNode reads per-file
|
||||
* `storage_session_id` for actual injection — the representative is
|
||||
* informational rather than load-bearing. Mirrors the same convention
|
||||
* used in `primeInvokedSkills`. */
|
||||
const representativeSessionId = prior?.session_id ?? files[0].storage_session_id;
|
||||
if (!representativeSessionId) {
|
||||
return existing;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -72,8 +72,20 @@ describe('createToolExecuteHandler', () => {
|
|||
codeSessionContext: {
|
||||
session_id: 'prev-session-abc',
|
||||
files: [
|
||||
{ session_id: 'prev-session-abc', id: 'f1', name: 'data.parquet' },
|
||||
{ session_id: 'prev-session-abc', id: 'f2', name: 'chart.png' },
|
||||
{
|
||||
storage_session_id: 'prev-session-abc',
|
||||
id: 'f1',
|
||||
resource_id: 'user_alice',
|
||||
name: 'data.parquet',
|
||||
kind: 'user',
|
||||
},
|
||||
{
|
||||
storage_session_id: 'prev-session-abc',
|
||||
id: 'f2',
|
||||
resource_id: 'user_alice',
|
||||
name: 'chart.png',
|
||||
kind: 'user',
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
|
|
@ -84,8 +96,20 @@ describe('createToolExecuteHandler', () => {
|
|||
expect(capturedConfigs).toHaveLength(1);
|
||||
expect(capturedConfigs[0].session_id).toBe('prev-session-abc');
|
||||
expect(capturedConfigs[0]._injected_files).toEqual([
|
||||
{ session_id: 'prev-session-abc', id: 'f1', name: 'data.parquet' },
|
||||
{ session_id: 'prev-session-abc', id: 'f2', name: 'chart.png' },
|
||||
{
|
||||
storage_session_id: 'prev-session-abc',
|
||||
id: 'f1',
|
||||
resource_id: 'user_alice',
|
||||
name: 'data.parquet',
|
||||
kind: 'user',
|
||||
},
|
||||
{
|
||||
storage_session_id: 'prev-session-abc',
|
||||
id: 'f2',
|
||||
resource_id: 'user_alice',
|
||||
name: 'chart.png',
|
||||
kind: 'user',
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
|
|
@ -141,7 +165,15 @@ describe('createToolExecuteHandler', () => {
|
|||
args: { lang: 'python', code: 'step_1()' },
|
||||
codeSessionContext: {
|
||||
session_id: 'session-A',
|
||||
files: [{ session_id: 'session-A', id: 'fa', name: 'a.csv' }],
|
||||
files: [
|
||||
{
|
||||
storage_session_id: 'session-A',
|
||||
id: 'fa',
|
||||
resource_id: 'user_alice',
|
||||
name: 'a.csv',
|
||||
kind: 'user',
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
{
|
||||
|
|
@ -150,7 +182,15 @@ describe('createToolExecuteHandler', () => {
|
|||
args: { lang: 'python', code: 'step_2()' },
|
||||
codeSessionContext: {
|
||||
session_id: 'session-A',
|
||||
files: [{ session_id: 'session-A', id: 'fa', name: 'a.csv' }],
|
||||
files: [
|
||||
{
|
||||
storage_session_id: 'session-A',
|
||||
id: 'fa',
|
||||
resource_id: 'user_alice',
|
||||
name: 'a.csv',
|
||||
kind: 'user',
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
];
|
||||
|
|
@ -161,7 +201,13 @@ describe('createToolExecuteHandler', () => {
|
|||
for (const config of capturedConfigs) {
|
||||
expect(config.session_id).toBe('session-A');
|
||||
expect(config._injected_files).toEqual([
|
||||
{ session_id: 'session-A', id: 'fa', name: 'a.csv' },
|
||||
{
|
||||
storage_session_id: 'session-A',
|
||||
id: 'fa',
|
||||
resource_id: 'user_alice',
|
||||
name: 'a.csv',
|
||||
kind: 'user',
|
||||
},
|
||||
]);
|
||||
}
|
||||
});
|
||||
|
|
@ -177,7 +223,15 @@ describe('createToolExecuteHandler', () => {
|
|||
args: { query: 'test' },
|
||||
codeSessionContext: {
|
||||
session_id: 'should-be-ignored',
|
||||
files: [{ session_id: 'x', id: 'y', name: 'z' }],
|
||||
files: [
|
||||
{
|
||||
storage_session_id: 'x',
|
||||
id: 'y',
|
||||
resource_id: 'user_alice',
|
||||
name: 'z',
|
||||
kind: 'user',
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
];
|
||||
|
|
@ -205,6 +259,7 @@ describe('createToolExecuteHandler', () => {
|
|||
name: 'pii-redactor',
|
||||
body: 'restricted body',
|
||||
fileCount: 0,
|
||||
version: 1,
|
||||
disableModelInvocation: true,
|
||||
}));
|
||||
const handler = createSkillHandler(getSkillByName);
|
||||
|
|
@ -247,6 +302,7 @@ describe('createToolExecuteHandler', () => {
|
|||
name: 'normal-skill',
|
||||
body: 'body',
|
||||
fileCount: 0,
|
||||
version: 1,
|
||||
}));
|
||||
const handler = createSkillHandler(getSkillByName);
|
||||
|
||||
|
|
@ -276,6 +332,7 @@ describe('createToolExecuteHandler', () => {
|
|||
name: 'maybe-disabled',
|
||||
body: 'body',
|
||||
fileCount: 0,
|
||||
version: 1,
|
||||
}));
|
||||
const handler = createSkillHandler(getSkillByName);
|
||||
|
||||
|
|
@ -302,6 +359,7 @@ describe('createToolExecuteHandler', () => {
|
|||
name: 'maybe-disabled-read',
|
||||
body: '# Body',
|
||||
fileCount: 0,
|
||||
version: 1,
|
||||
}));
|
||||
const handler = createToolExecuteHandler({
|
||||
loadTools: jest.fn(async () => ({
|
||||
|
|
@ -342,6 +400,7 @@ describe('createToolExecuteHandler', () => {
|
|||
name: 'manually-primed',
|
||||
body: '# Body',
|
||||
fileCount: 0,
|
||||
version: 1,
|
||||
}));
|
||||
const handler = createToolExecuteHandler({
|
||||
loadTools: jest.fn(async () => ({
|
||||
|
|
@ -391,6 +450,7 @@ describe('createToolExecuteHandler', () => {
|
|||
name: 'pii-redactor',
|
||||
body: 'restricted body',
|
||||
fileCount: 0,
|
||||
version: 1,
|
||||
disableModelInvocation: true,
|
||||
}));
|
||||
const handler = createToolExecuteHandler({
|
||||
|
|
@ -421,6 +481,7 @@ describe('createToolExecuteHandler', () => {
|
|||
name: 'normal-skill',
|
||||
body: '# Body',
|
||||
fileCount: 0,
|
||||
version: 1,
|
||||
}));
|
||||
const handler = createToolExecuteHandler({
|
||||
loadTools: jest.fn(async () => ({
|
||||
|
|
@ -454,6 +515,7 @@ describe('createToolExecuteHandler', () => {
|
|||
name: 'manual-only-skill',
|
||||
body: '# Use references/docs.md for details',
|
||||
fileCount: 0,
|
||||
version: 1,
|
||||
disableModelInvocation: true,
|
||||
}));
|
||||
const handler = createToolExecuteHandler({
|
||||
|
|
@ -489,6 +551,7 @@ describe('createToolExecuteHandler', () => {
|
|||
name: 'other-disabled-skill',
|
||||
body: 'restricted',
|
||||
fileCount: 0,
|
||||
version: 1,
|
||||
disableModelInvocation: true,
|
||||
}));
|
||||
const handler = createToolExecuteHandler({
|
||||
|
|
@ -526,6 +589,7 @@ describe('createToolExecuteHandler', () => {
|
|||
name: 'always-applied-legal',
|
||||
body: '# Cite references/policy.md when advising',
|
||||
fileCount: 0,
|
||||
version: 1,
|
||||
disableModelInvocation: true,
|
||||
}));
|
||||
const handler = createToolExecuteHandler({
|
||||
|
|
@ -566,6 +630,7 @@ describe('createToolExecuteHandler', () => {
|
|||
name: 'collides',
|
||||
body: '# primed body',
|
||||
fileCount: 0,
|
||||
version: 1,
|
||||
}));
|
||||
const handler = createToolExecuteHandler({
|
||||
loadTools: jest.fn(async () => ({
|
||||
|
|
@ -615,6 +680,7 @@ describe('createToolExecuteHandler', () => {
|
|||
name: 'brand-guidelines',
|
||||
body: 'skill body',
|
||||
fileCount: 2,
|
||||
version: 1,
|
||||
}));
|
||||
/* `loadTools` injects `codeEnvAvailable` into the returned
|
||||
`configurable`, which mirrors production flow through
|
||||
|
|
@ -789,6 +855,7 @@ describe('createToolExecuteHandler', () => {
|
|||
name: 'primed-only-skill',
|
||||
body: '# Primed skill body',
|
||||
fileCount: 1,
|
||||
version: 1,
|
||||
}));
|
||||
const readSandboxFile = jest.fn();
|
||||
const getSkillFileByPath = jest.fn(async () => ({
|
||||
|
|
@ -888,6 +955,7 @@ describe('createToolExecuteHandler', () => {
|
|||
name: 'real-skill',
|
||||
body: '# Real Body',
|
||||
fileCount: 0,
|
||||
version: 1,
|
||||
}));
|
||||
const readSandboxFile = jest.fn();
|
||||
const handler = makeReadFileHandler({
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@ import type {
|
|||
ToolExecuteBatchRequest,
|
||||
} from '@librechat/agents';
|
||||
import { Types } from 'mongoose';
|
||||
import type { CodeEnvRef } from 'librechat-data-provider';
|
||||
import type { StructuredToolInterface } from '@librechat/agents/langchain/tools';
|
||||
import type { SkillFileRecord } from './skillFiles';
|
||||
import type { ServerRequest } from '~/types';
|
||||
|
|
@ -67,6 +68,11 @@ export interface ToolExecuteOptions {
|
|||
body: string;
|
||||
name: string;
|
||||
_id: Types.ObjectId;
|
||||
/** Monotonic counter on the skill record. Threaded into
|
||||
* `codeEnvRef.version` so codeapi's sessionKey scopes the cache
|
||||
* per-revision; bumping the version on edit invalidates the
|
||||
* prior cache entry. */
|
||||
version: number;
|
||||
fileCount: number;
|
||||
/**
|
||||
* Set when the skill author opted out of model invocation. The handler
|
||||
|
|
@ -83,22 +89,30 @@ export interface ToolExecuteOptions {
|
|||
getDownloadStream?: (req: ServerRequest, filepath: string) => Promise<NodeJS.ReadableStream>;
|
||||
[key: string]: unknown;
|
||||
};
|
||||
/** Batch uploads files to the code execution environment */
|
||||
/** Batch uploads files to the code execution environment. `kind`/`id`/
|
||||
* `version?` carry the resource identity codeapi uses to derive the
|
||||
* sessionKey for the batch's storage bucket. */
|
||||
batchUploadCodeEnvFiles?: (params: {
|
||||
req: ServerRequest;
|
||||
files: Array<{ stream: NodeJS.ReadableStream; filename: string }>;
|
||||
entity_id?: string;
|
||||
}) => Promise<{ session_id: string; files: Array<{ fileId: string; filename: string }> }>;
|
||||
kind: 'skill' | 'agent' | 'user';
|
||||
id: string;
|
||||
version?: number;
|
||||
read_only?: boolean;
|
||||
}) => Promise<{
|
||||
storage_session_id: string;
|
||||
files: Array<{ fileId: string; filename: string }>;
|
||||
}>;
|
||||
/** Checks if a code env file is still active. Returns lastModified or null. */
|
||||
getSessionInfo?: (fileIdentifier: string) => Promise<string | null>;
|
||||
getSessionInfo?: (ref: CodeEnvRef) => Promise<string | null>;
|
||||
/** 23-hour freshness check */
|
||||
checkIfActive?: (dateString: string) => boolean;
|
||||
/** Persists codeEnvIdentifiers on skill files after upload */
|
||||
/** Persists `codeEnvRef` on skill files after upload */
|
||||
updateSkillFileCodeEnvIds?: (
|
||||
updates: Array<{
|
||||
skillId: Types.ObjectId | string;
|
||||
relativePath: string;
|
||||
codeEnvIdentifier: string;
|
||||
codeEnvRef: CodeEnvRef;
|
||||
}>,
|
||||
) => Promise<void>;
|
||||
/** Loads a skill file by path (for read_file tool) */
|
||||
|
|
@ -888,7 +902,19 @@ async function handleSkillToolCall(
|
|||
|
||||
const contentText = `Skill "${args.skillName}" loaded. Follow the instructions below.`;
|
||||
let artifact:
|
||||
| { session_id: string; files: Array<{ id: string; session_id: string; name: string }> }
|
||||
| {
|
||||
session_id: string;
|
||||
files: Array<{
|
||||
id: string;
|
||||
/** Resource id (skill `_id`). codeapi requires this distinct
|
||||
* from the storage `id` to scope sessionKey by resource. */
|
||||
resource_id: string;
|
||||
name: string;
|
||||
storage_session_id: string;
|
||||
kind?: 'skill' | 'agent' | 'user';
|
||||
version?: number;
|
||||
}>;
|
||||
}
|
||||
| undefined;
|
||||
|
||||
// Prime skill files to code env — only when the `execute_code` capability
|
||||
|
|
@ -916,7 +942,26 @@ async function handleSkillToolCall(
|
|||
updateSkillFileCodeEnvIds,
|
||||
});
|
||||
if (primeResult) {
|
||||
artifact = primeResult;
|
||||
/* `session_id` at the top of the artifact is the (representative)
|
||||
* execution session — ToolNode reads it for CodeSessionContext
|
||||
* continuity. Per-file storage lives on each file's
|
||||
* `storage_session_id`. Skill files carry `kind: 'skill'` and
|
||||
* the skill's version so codeapi's sessionKey scopes the
|
||||
* cache per-revision. */
|
||||
artifact = {
|
||||
session_id: primeResult.storage_session_id,
|
||||
files: primeResult.files.map((f) => ({
|
||||
id: f.id,
|
||||
/* `resource_id` (skill `_id`) is what codeapi feeds into
|
||||
* `<tenant>:skill:<id>:v:<version>` — without it the next
|
||||
* /exec authorizer sees `resource_id: undefined` and 400s. */
|
||||
resource_id: f.resource_id,
|
||||
name: f.name,
|
||||
storage_session_id: f.storage_session_id,
|
||||
kind: 'skill',
|
||||
version: skill.version,
|
||||
})),
|
||||
};
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
|
|
@ -1015,6 +1060,56 @@ export function createToolExecuteHandler(options: ToolExecuteOptions): EventHand
|
|||
toolCallConfig.session_id = tc.codeSessionContext.session_id;
|
||||
if (tc.codeSessionContext.files && tc.codeSessionContext.files.length > 0) {
|
||||
toolCallConfig._injected_files = tc.codeSessionContext.files;
|
||||
/* Last LC-controlled point before the wire. Mirrors
|
||||
* codeapi's validator context so the two log sides
|
||||
* correlate on a single grep. */
|
||||
const refs = tc.codeSessionContext.files as Array<{
|
||||
id?: unknown;
|
||||
resource_id?: unknown;
|
||||
storage_session_id?: unknown;
|
||||
kind?: unknown;
|
||||
version?: unknown;
|
||||
name?: unknown;
|
||||
}>;
|
||||
const summary = refs.map((f) => ({
|
||||
kind: f.kind,
|
||||
hasResourceId: typeof f.resource_id === 'string' && !!f.resource_id,
|
||||
hasStorageSessionId:
|
||||
typeof f.storage_session_id === 'string' && !!f.storage_session_id,
|
||||
hasVersion: typeof f.version === 'number',
|
||||
}));
|
||||
const missingResourceId = summary.filter((s) => !s.hasResourceId).length;
|
||||
logger.debug(
|
||||
`[code-env:inject] tool=${tc.name} files=${refs.length} ` +
|
||||
`missingResourceId=${missingResourceId}`,
|
||||
{ summary },
|
||||
);
|
||||
if (missingResourceId > 0) {
|
||||
logger.warn(
|
||||
`[code-env:inject] ${missingResourceId}/${refs.length} files missing resource_id ` +
|
||||
`for tool=${tc.name} — codeapi will reject with 400`,
|
||||
{ summary },
|
||||
);
|
||||
}
|
||||
} else {
|
||||
/* Empty `_injected_files` on a code-execution tool
|
||||
* call. Almost always means the seeding chain
|
||||
* (primeCodeFiles → initialSessions →
|
||||
* CodeSessionContext) dropped the file upstream.
|
||||
* `session_id` is still emitted; agents falls
|
||||
* through to the `/files/<sid>` legacy fetch
|
||||
* which is post-cutover broken (returns 400).
|
||||
* Pair with `[primeCodeFiles]` traces below to
|
||||
* locate the layer that lost the ref. */
|
||||
logger.warn(
|
||||
`[code-env:inject] tool=${tc.name} _injected_files=0 — sandbox will see no input files`,
|
||||
{
|
||||
tool: tc.name,
|
||||
session_id: tc.codeSessionContext.session_id,
|
||||
codeSessionContextHasFiles: tc.codeSessionContext.files !== undefined,
|
||||
codeSessionContextFileCount: tc.codeSessionContext.files?.length ?? 0,
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -127,7 +127,7 @@ describe('primeResources', () => {
|
|||
embedded: false,
|
||||
usage: 0,
|
||||
metadata: {
|
||||
fileIdentifier: 'python-script',
|
||||
codeEnvRef: { kind: 'user', id: 'user1', storage_session_id: 'sess', file_id: 'fid' },
|
||||
},
|
||||
},
|
||||
];
|
||||
|
|
@ -328,7 +328,7 @@ describe('primeResources', () => {
|
|||
embedded: false,
|
||||
usage: 0,
|
||||
metadata: {
|
||||
fileIdentifier: 'python-script',
|
||||
codeEnvRef: { kind: 'user', id: 'user1', storage_session_id: 'sess', file_id: 'fid' },
|
||||
},
|
||||
},
|
||||
];
|
||||
|
|
@ -542,7 +542,7 @@ describe('primeResources', () => {
|
|||
embedded: false,
|
||||
usage: 0,
|
||||
metadata: {
|
||||
fileIdentifier: 'python-script',
|
||||
codeEnvRef: { kind: 'user', id: 'user1', storage_session_id: 'sess', file_id: 'fid' },
|
||||
},
|
||||
};
|
||||
|
||||
|
|
@ -713,7 +713,7 @@ describe('primeResources', () => {
|
|||
embedded: false,
|
||||
usage: 0,
|
||||
metadata: {
|
||||
fileIdentifier: 'python-script',
|
||||
codeEnvRef: { kind: 'user', id: 'user1', storage_session_id: 'sess', file_id: 'fid' },
|
||||
},
|
||||
};
|
||||
|
||||
|
|
@ -728,7 +728,7 @@ describe('primeResources', () => {
|
|||
embedded: false,
|
||||
usage: 0,
|
||||
metadata: {
|
||||
fileIdentifier: 'python-script',
|
||||
codeEnvRef: { kind: 'user', id: 'user1', storage_session_id: 'sess', file_id: 'fid' },
|
||||
},
|
||||
};
|
||||
|
||||
|
|
@ -878,7 +878,7 @@ describe('primeResources', () => {
|
|||
embedded: false,
|
||||
usage: 0,
|
||||
metadata: {
|
||||
fileIdentifier: 'python-script',
|
||||
codeEnvRef: { kind: 'user', id: 'user1', storage_session_id: 'sess', file_id: 'fid' },
|
||||
},
|
||||
};
|
||||
|
||||
|
|
@ -1128,7 +1128,7 @@ describe('primeResources', () => {
|
|||
embedded: false,
|
||||
usage: 0,
|
||||
metadata: {
|
||||
fileIdentifier: 'python-script',
|
||||
codeEnvRef: { kind: 'user', id: 'user1', storage_session_id: 'sess', file_id: 'fid' },
|
||||
},
|
||||
},
|
||||
{
|
||||
|
|
|
|||
|
|
@ -80,7 +80,7 @@ const addFileToResource = ({
|
|||
/**
|
||||
* Categorizes a file into the appropriate tool resource based on its properties
|
||||
* Files are categorized as:
|
||||
* - execute_code: Files with fileIdentifier metadata
|
||||
* - execute_code: Files with a code-environment ref (`codeEnvRef`)
|
||||
* - file_search: Files marked as embedded
|
||||
* - image_edit: Image files in the request file set with dimensions
|
||||
* @param params - Parameters object
|
||||
|
|
@ -100,7 +100,7 @@ const categorizeFileForToolResources = ({
|
|||
requestFileSet: Set<string>;
|
||||
processedResourceFiles: Set<string>;
|
||||
}): void => {
|
||||
if (file.metadata?.fileIdentifier) {
|
||||
if (file.metadata?.codeEnvRef) {
|
||||
addFileToResource({
|
||||
file,
|
||||
resourceType: EToolResources.execute_code,
|
||||
|
|
|
|||
|
|
@ -14,15 +14,16 @@ jest.mock('./run', () => ({
|
|||
|
||||
import { Readable } from 'stream';
|
||||
import { Types } from 'mongoose';
|
||||
import { primeInvokedSkills } from './skillFiles';
|
||||
import type { PrimeInvokedSkillsDeps } from './skillFiles';
|
||||
import { primeInvokedSkills, primeSkillFiles } from './skillFiles';
|
||||
import type { PrimeInvokedSkillsDeps, PrimeSkillFilesParams } from './skillFiles';
|
||||
|
||||
const SKILL_ID = new Types.ObjectId();
|
||||
const SKILL_VERSION = 7;
|
||||
|
||||
function makeDeps(overrides: Partial<PrimeInvokedSkillsDeps> = {}): PrimeInvokedSkillsDeps {
|
||||
const listSkillFiles = jest.fn().mockResolvedValue([]);
|
||||
const batchUploadCodeEnvFiles = jest.fn().mockResolvedValue({
|
||||
session_id: 'session-x',
|
||||
storage_session_id: 'session-x',
|
||||
files: [],
|
||||
});
|
||||
return {
|
||||
|
|
@ -34,6 +35,7 @@ function makeDeps(overrides: Partial<PrimeInvokedSkillsDeps> = {}): PrimeInvoked
|
|||
_id: SKILL_ID,
|
||||
name: 'brand-guidelines',
|
||||
body: 'skill body',
|
||||
version: SKILL_VERSION,
|
||||
fileCount: 2,
|
||||
}),
|
||||
listSkillFiles,
|
||||
|
|
@ -79,14 +81,10 @@ describe('primeInvokedSkills — execute_code capability gate', () => {
|
|||
];
|
||||
const listSkillFiles = jest.fn().mockResolvedValue(fileRecords);
|
||||
const getStrategyFunctions = jest.fn().mockReturnValue({
|
||||
/* A real empty Readable — matches the production contract for
|
||||
`getDownloadStream` and works even if `batchUploadCodeEnvFiles`
|
||||
is later replaced with a partially-real implementation that
|
||||
iterates the stream. */
|
||||
getDownloadStream: jest.fn().mockResolvedValue(Readable.from(Buffer.from(''))),
|
||||
});
|
||||
const batchUploadCodeEnvFiles = jest.fn().mockResolvedValue({
|
||||
session_id: 'session-42',
|
||||
storage_session_id: 'session-42',
|
||||
files: [{ fileId: 'file-1', filename: 'brand-guidelines/references/style.md' }],
|
||||
});
|
||||
|
||||
|
|
@ -101,17 +99,17 @@ describe('primeInvokedSkills — execute_code capability gate', () => {
|
|||
|
||||
expect(batchUploadCodeEnvFiles).toHaveBeenCalledTimes(1);
|
||||
const [uploadArgs] = batchUploadCodeEnvFiles.mock.calls[0];
|
||||
/* Phase 8 deprecation: LibreChat no longer threads an apiKey through
|
||||
the sandbox upload path. The agents library / sandbox service owns
|
||||
auth internally. */
|
||||
expect(uploadArgs).not.toHaveProperty('apiKey');
|
||||
expect(uploadArgs.entity_id).toBe(SKILL_ID.toString());
|
||||
/* Skill files are infrastructure inputs; the read_only flag tells codeapi
|
||||
to seal them so any sandboxed-code modifications are dropped instead
|
||||
of surfaced as ghost generated artifacts. */
|
||||
expect(uploadArgs).not.toHaveProperty('entity_id');
|
||||
expect(uploadArgs.read_only).toBe(true);
|
||||
/* One uploaded file per `fileRecords` entry plus the synthetic
|
||||
SKILL.md that `primeSkillFiles` always prepends. */
|
||||
/* Resource identity for codeapi's sessionKey: skill files share
|
||||
* cross-user-within-tenant under `<tenant>:skill:<id>:v:<version>`.
|
||||
* Without these on the upload, codeapi falls back to user bucketing
|
||||
* and skill-cache invalidation (driven by version bump on edit)
|
||||
* never fires. See codeapi #1455 (option α). */
|
||||
expect(uploadArgs.kind).toBe('skill');
|
||||
expect(uploadArgs.id).toBe(SKILL_ID.toString());
|
||||
expect(uploadArgs.version).toBe(SKILL_VERSION);
|
||||
expect(uploadArgs.files).toHaveLength(fileRecords.length + 1);
|
||||
expect(uploadArgs.files.map((f: { filename: string }) => f.filename)).toEqual(
|
||||
expect.arrayContaining(['brand-guidelines/SKILL.md', 'brand-guidelines/references/style.md']),
|
||||
|
|
@ -128,12 +126,12 @@ describe('primeInvokedSkills — execute_code capability gate', () => {
|
|||
expect(deps.getSkillByName).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('forwards entity_id on every file in initialSessions after fresh upload', async () => {
|
||||
/* Regression: codeapi now resolves sessionKey per-file using `entity_id`.
|
||||
* Skill files must carry `entity_id=<skillId>` through priming so that
|
||||
* a subsequent execute mixing skill files and a user attachment can be
|
||||
* authorized — both files in the same call resolve to their own scope
|
||||
* instead of collapsing onto a single request-level entity. */
|
||||
it('writes kind/version/storage_session_id on every cached file after a fresh upload', async () => {
|
||||
/* Per-file refs in `CodeSessionContext.files` carry the resource
|
||||
* identity (kind=skill, id=skillId, version=skill.version) so
|
||||
* codeapi can derive the right sessionKey. Cache scope is per
|
||||
* (tenant, kind, id, version) — bumping the skill version
|
||||
* naturally invalidates the prior cache. */
|
||||
const listSkillFiles = jest.fn().mockResolvedValue([
|
||||
{
|
||||
relativePath: 'references/style.md',
|
||||
|
|
@ -147,7 +145,7 @@ describe('primeInvokedSkills — execute_code capability gate', () => {
|
|||
getDownloadStream: jest.fn().mockResolvedValue(Readable.from(Buffer.from(''))),
|
||||
});
|
||||
const batchUploadCodeEnvFiles = jest.fn().mockResolvedValue({
|
||||
session_id: 'session-42',
|
||||
storage_session_id: 'session-42',
|
||||
files: [{ fileId: 'file-1', filename: 'brand-guidelines/references/style.md' }],
|
||||
});
|
||||
|
||||
|
|
@ -164,9 +162,16 @@ describe('primeInvokedSkills — execute_code capability gate', () => {
|
|||
expect(codeSession?.files).toEqual([
|
||||
{
|
||||
id: 'file-1',
|
||||
/* `resource_id` is the skill `_id` — drives codeapi's
|
||||
* sessionKey re-derivation. Distinct from `id` (the storage
|
||||
* file_id). The split fixes the shared-kind 403 mismatch
|
||||
* where codeapi computed sessionKey from the storage nanoid
|
||||
* instead of the skill _id. */
|
||||
resource_id: SKILL_ID.toString(),
|
||||
name: 'brand-guidelines/references/style.md',
|
||||
session_id: 'session-42',
|
||||
entity_id: SKILL_ID.toString(),
|
||||
storage_session_id: 'session-42',
|
||||
kind: 'skill',
|
||||
version: SKILL_VERSION,
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
|
@ -175,10 +180,10 @@ describe('primeInvokedSkills — execute_code capability gate', () => {
|
|||
/* Concurrency regression: when many users hit the same skill at
|
||||
* once, fire-and-forget keeps the cache in miss-steady-state for
|
||||
* the burst — User N's prime reads SkillFile docs before User N-1's
|
||||
* persist commits, sees no `codeEnvIdentifier`, re-uploads, and
|
||||
* fires its own forget that User N+1 also races. Awaiting the
|
||||
* persist before the prime resolves ensures the next concurrent
|
||||
* caller observes the cache pointer instead of racing a write. */
|
||||
* persist commits, sees no codeEnvRef, re-uploads, and fires its
|
||||
* own forget that User N+1 also races. Awaiting the persist before
|
||||
* the prime resolves ensures the next concurrent caller observes
|
||||
* the cache pointer instead of racing a write. */
|
||||
const fileRecords = [
|
||||
{
|
||||
relativePath: 'references/style.md',
|
||||
|
|
@ -193,7 +198,7 @@ describe('primeInvokedSkills — execute_code capability gate', () => {
|
|||
getDownloadStream: jest.fn().mockResolvedValue(Readable.from(Buffer.from(''))),
|
||||
});
|
||||
const batchUploadCodeEnvFiles = jest.fn().mockResolvedValue({
|
||||
session_id: 'session-42',
|
||||
storage_session_id: 'session-42',
|
||||
files: [{ fileId: 'file-1', filename: 'brand-guidelines/references/style.md' }],
|
||||
});
|
||||
|
||||
|
|
@ -235,17 +240,23 @@ describe('primeInvokedSkills — execute_code capability gate', () => {
|
|||
{
|
||||
skillId: SKILL_ID,
|
||||
relativePath: 'references/style.md',
|
||||
codeEnvIdentifier: `session-42/file-1?entity_id=${SKILL_ID.toString()}`,
|
||||
codeEnvRef: {
|
||||
kind: 'skill',
|
||||
id: SKILL_ID.toString(),
|
||||
storage_session_id: 'session-42',
|
||||
file_id: 'file-1',
|
||||
version: SKILL_VERSION,
|
||||
},
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('parses entity_id off codeEnvIdentifier in the cached-skills hot path', async () => {
|
||||
it('reads codeEnvRef in the cached-skills hot path', async () => {
|
||||
/* When all skill files are still active in codeapi, primeInvokedSkills
|
||||
* skips the batch upload entirely and reconstructs file refs from each
|
||||
* skill file's persisted `codeEnvIdentifier`. The query string carries
|
||||
* `?entity_id=<skillId>`, which must survive into `_injected_files` so
|
||||
* downstream authorization still uses the skill's scope. */
|
||||
* skips the batch upload entirely and reconstructs file refs from
|
||||
* each skill file's persisted `codeEnvRef`. The kind/version travel
|
||||
* through to `CodeSessionContext.files` so the next exec carries
|
||||
* them on `_injected_files` for codeapi's sessionKey derivation. */
|
||||
const listSkillFiles = jest.fn().mockResolvedValue([
|
||||
{
|
||||
relativePath: 'references/style.md',
|
||||
|
|
@ -253,7 +264,13 @@ describe('primeInvokedSkills — execute_code capability gate', () => {
|
|||
filepath: '/storage/brand-guidelines/references/style.md',
|
||||
source: 's3',
|
||||
bytes: 256,
|
||||
codeEnvIdentifier: `session-cached/file-cached?entity_id=${SKILL_ID.toString()}`,
|
||||
codeEnvRef: {
|
||||
kind: 'skill',
|
||||
id: SKILL_ID.toString(),
|
||||
storage_session_id: 'session-cached',
|
||||
file_id: 'file-cached',
|
||||
version: SKILL_VERSION,
|
||||
},
|
||||
},
|
||||
]);
|
||||
const batchUploadCodeEnvFiles = jest.fn();
|
||||
|
|
@ -261,7 +278,7 @@ describe('primeInvokedSkills — execute_code capability gate', () => {
|
|||
codeEnvAvailable: true,
|
||||
listSkillFiles,
|
||||
batchUploadCodeEnvFiles,
|
||||
getSessionInfo: jest.fn().mockResolvedValue('2026-05-05T00:00:00Z'),
|
||||
getSessionInfo: jest.fn().mockResolvedValue('2026-05-06T00:00:00Z'),
|
||||
checkIfActive: jest.fn().mockReturnValue(true),
|
||||
});
|
||||
|
||||
|
|
@ -272,9 +289,118 @@ describe('primeInvokedSkills — execute_code capability gate', () => {
|
|||
expect(codeSession?.files).toEqual([
|
||||
{
|
||||
id: 'file-cached',
|
||||
/* From the cache-hit path: pulls `resource_id` directly off
|
||||
* the persisted `codeEnvRef.id` (the skill `_id`). */
|
||||
resource_id: SKILL_ID.toString(),
|
||||
name: 'brand-guidelines/references/style.md',
|
||||
session_id: 'session-cached',
|
||||
entity_id: SKILL_ID.toString(),
|
||||
storage_session_id: 'session-cached',
|
||||
kind: 'skill',
|
||||
version: SKILL_VERSION,
|
||||
},
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
/* The tool-invoked skill loader (`handle_skill` -> `primeSkillFiles`)
|
||||
* is a separate code path from `primeInvokedSkills` (the NL-detected
|
||||
* loader). Both feed `_injected_files` on the next /exec; both must
|
||||
* carry `resource_id` end-to-end, otherwise codeapi 400s with
|
||||
* `resource_id is invalid` (`type: 'undefined'`). Tests below lock
|
||||
* that contract on the lower-level helper directly. */
|
||||
describe('primeSkillFiles — resource identity propagation', () => {
|
||||
beforeEach(() => {
|
||||
jest.clearAllMocks();
|
||||
});
|
||||
|
||||
function makeSkillFilesDeps(
|
||||
overrides: Partial<PrimeSkillFilesParams> = {},
|
||||
): PrimeSkillFilesParams {
|
||||
return {
|
||||
skill: {
|
||||
_id: SKILL_ID,
|
||||
name: 'brand-guidelines',
|
||||
body: 'skill body',
|
||||
version: SKILL_VERSION,
|
||||
},
|
||||
skillFiles: [],
|
||||
req: { user: { id: 'user-1' } } as PrimeSkillFilesParams['req'],
|
||||
getStrategyFunctions: jest.fn().mockReturnValue({
|
||||
getDownloadStream: jest.fn().mockResolvedValue(Readable.from(Buffer.from(''))),
|
||||
}),
|
||||
batchUploadCodeEnvFiles: jest.fn().mockResolvedValue({
|
||||
storage_session_id: 'session-fresh',
|
||||
files: [
|
||||
{ fileId: 'file-fresh', filename: 'brand-guidelines/references/style.md' },
|
||||
{ fileId: 'file-skillmd', filename: 'brand-guidelines/SKILL.md' },
|
||||
],
|
||||
}),
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
it('fresh-upload path: emits resource_id=skill._id, kind=skill, version on each file', async () => {
|
||||
const deps = makeSkillFilesDeps({
|
||||
skillFiles: [
|
||||
{
|
||||
relativePath: 'references/style.md',
|
||||
filename: 'style.md',
|
||||
filepath: '/storage/brand-guidelines/references/style.md',
|
||||
source: 's3',
|
||||
bytes: 256,
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
const result = await primeSkillFiles(deps);
|
||||
|
||||
expect(result?.files).toEqual([
|
||||
{
|
||||
id: 'file-fresh',
|
||||
resource_id: SKILL_ID.toString(),
|
||||
storage_session_id: 'session-fresh',
|
||||
name: 'brand-guidelines/references/style.md',
|
||||
kind: 'skill',
|
||||
version: SKILL_VERSION,
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('cache-hit path: re-derives resource_id/kind/version from persisted codeEnvRef', async () => {
|
||||
const cachedRef = {
|
||||
kind: 'skill' as const,
|
||||
id: SKILL_ID.toString(),
|
||||
storage_session_id: 'session-cached',
|
||||
file_id: 'file-cached',
|
||||
version: SKILL_VERSION,
|
||||
};
|
||||
const batchUploadCodeEnvFiles = jest.fn();
|
||||
const deps = makeSkillFilesDeps({
|
||||
skillFiles: [
|
||||
{
|
||||
relativePath: 'references/style.md',
|
||||
filename: 'style.md',
|
||||
filepath: '/storage/brand-guidelines/references/style.md',
|
||||
source: 's3',
|
||||
bytes: 256,
|
||||
codeEnvRef: cachedRef,
|
||||
},
|
||||
],
|
||||
batchUploadCodeEnvFiles,
|
||||
getSessionInfo: jest.fn().mockResolvedValue('2026-05-06T00:00:00Z'),
|
||||
checkIfActive: jest.fn().mockReturnValue(true),
|
||||
});
|
||||
|
||||
const result = await primeSkillFiles(deps);
|
||||
|
||||
expect(batchUploadCodeEnvFiles).not.toHaveBeenCalled();
|
||||
expect(result?.files).toEqual([
|
||||
{
|
||||
id: 'file-cached',
|
||||
resource_id: SKILL_ID.toString(),
|
||||
storage_session_id: 'session-cached',
|
||||
name: 'brand-guidelines/references/style.md',
|
||||
kind: 'skill',
|
||||
version: SKILL_VERSION,
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ import { Readable } from 'stream';
|
|||
import { Constants } from '@librechat/agents';
|
||||
import { logger } from '@librechat/data-schemas';
|
||||
import type { ToolSessionMap, CodeSessionContext } from '@librechat/agents';
|
||||
import type { CodeEnvRef } from 'librechat-data-provider';
|
||||
import type { Types } from 'mongoose';
|
||||
import type { ServerRequest } from '~/types';
|
||||
import { extractInvokedSkillsFromPayload } from './run';
|
||||
|
|
@ -12,11 +13,19 @@ export interface SkillFileRecord {
|
|||
filepath: string;
|
||||
source: string;
|
||||
bytes: number;
|
||||
codeEnvIdentifier?: string;
|
||||
codeEnvRef?: CodeEnvRef;
|
||||
}
|
||||
|
||||
export interface PrimeSkillFilesParams {
|
||||
skill: { body: string; name: string; _id: Types.ObjectId | string };
|
||||
skill: {
|
||||
body: string;
|
||||
name: string;
|
||||
_id: Types.ObjectId | string;
|
||||
/** Monotonic counter on the skill record. Bumped on every edit
|
||||
* (frontmatter / body / file upsert). Threaded into `codeEnvRef.version`
|
||||
* so codeapi's sessionKey scopes the cache per-revision. */
|
||||
version: number;
|
||||
};
|
||||
skillFiles: SkillFileRecord[];
|
||||
req: ServerRequest;
|
||||
getStrategyFunctions: (source: string) => {
|
||||
|
|
@ -26,44 +35,53 @@ export interface PrimeSkillFilesParams {
|
|||
batchUploadCodeEnvFiles: (params: {
|
||||
req: ServerRequest;
|
||||
files: Array<{ stream: NodeJS.ReadableStream; filename: string }>;
|
||||
entity_id?: string;
|
||||
/** Resource kind that owns the batch's storage session. Drives codeapi's
|
||||
* sessionKey derivation (`<tenant>:<kind>:<id>[:v:<version>]`). */
|
||||
kind: 'skill' | 'agent' | 'user';
|
||||
/** Resource id (skillId / agentId / userId). */
|
||||
id: string;
|
||||
/** Required when `kind === 'skill'`; forbidden otherwise. */
|
||||
version?: number;
|
||||
/** When true, codeapi tags every file in the batch as infrastructure
|
||||
* (read-only inputs that must never surface as generated artifacts,
|
||||
* even if sandboxed code mutates the bytes on disk). */
|
||||
read_only?: boolean;
|
||||
}) => Promise<{
|
||||
session_id: string;
|
||||
storage_session_id: string;
|
||||
files: Array<{ fileId: string; filename: string }>;
|
||||
}>;
|
||||
/** Checks if a code env file is still active. Returns lastModified timestamp or null. */
|
||||
getSessionInfo?: (fileIdentifier: string) => Promise<string | null>;
|
||||
getSessionInfo?: (ref: CodeEnvRef) => Promise<string | null>;
|
||||
/** 23-hour freshness check */
|
||||
checkIfActive?: (dateString: string) => boolean;
|
||||
/** Persists codeEnvIdentifier on skill files after upload. Implementations
|
||||
/** Persists `codeEnvRef` on skill files after upload. Implementations
|
||||
* warn-log on partial writes (matchedCount/modifiedCount mismatch)
|
||||
* internally — caller can fire-and-forget without losing visibility. */
|
||||
updateSkillFileCodeEnvIds?: (
|
||||
updates: Array<{
|
||||
skillId: Types.ObjectId | string;
|
||||
relativePath: string;
|
||||
codeEnvIdentifier: string;
|
||||
codeEnvRef: CodeEnvRef;
|
||||
}>,
|
||||
) => Promise<{ matchedCount: number; modifiedCount: number } | void>;
|
||||
}
|
||||
|
||||
export interface PrimeSkillFilesResult {
|
||||
session_id: string;
|
||||
files: Array<{ id: string; session_id: string; name: string; entity_id?: string }>;
|
||||
}
|
||||
|
||||
/** Parses `entity_id` out of a persisted `codeEnvIdentifier`'s query string.
|
||||
* Returns `undefined` when the identifier has no query string or no
|
||||
* `entity_id` (legacy user-attachment uploads). */
|
||||
function parseEntityIdFromCodeEnvIdentifier(identifier: string): string | undefined {
|
||||
const queryString = identifier.split('?')[1];
|
||||
if (!queryString) return undefined;
|
||||
const value = new URLSearchParams(queryString).get('entity_id');
|
||||
return value && value.length > 0 ? value : undefined;
|
||||
/** Representative storage session id (first file's). */
|
||||
storage_session_id: string;
|
||||
files: Array<{
|
||||
/** Storage file id (the per-file uuid file_server returned at upload). */
|
||||
id: string;
|
||||
/** Resource id — the entity that owns the storage session. For skill
|
||||
* files this is `skill._id.toString()`. Distinct from `id`; codeapi
|
||||
* derives the sessionKey from `resource_id` (shared cache scope) but
|
||||
* validates upload presence under `id` (per-file storage key). */
|
||||
resource_id: string;
|
||||
storage_session_id: string;
|
||||
name: string;
|
||||
kind: 'skill' | 'agent' | 'user';
|
||||
version?: number;
|
||||
}>;
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -90,29 +108,27 @@ export async function primeSkillFiles(
|
|||
updateSkillFileCodeEnvIds,
|
||||
} = params;
|
||||
|
||||
// Check if ALL existing sessions are still active before reusing cached refs.
|
||||
// Files normally share one session, but a partial bulkWrite failure during
|
||||
// updateSkillFileCodeEnvIds can leave mixed session IDs. Checking every
|
||||
// distinct session prevents serving stale identifiers that 404 in code env.
|
||||
/* Cache-hit path: every skillFile carries a `codeEnvRef` from the
|
||||
* previous prime. Check freshness against codeapi for every distinct
|
||||
* storage session; if all are still active, reuse without
|
||||
* re-uploading. The skill version is part of the ref — when the
|
||||
* skill is edited, the upsert clears the ref and forces a fresh
|
||||
* upload on the next prime. */
|
||||
if (getSessionInfo && checkIfActive && skillFiles.length > 0) {
|
||||
// All files must have identifiers for the cache to be complete.
|
||||
// Any missing identifier means partial persistence — fall through to re-upload.
|
||||
const allHaveIds = skillFiles.every((sf) => sf.codeEnvIdentifier);
|
||||
if (allHaveIds) {
|
||||
const sessionIds = new Set(
|
||||
skillFiles.map((sf) => (sf.codeEnvIdentifier as string).split('?')[0].split('/')[0]),
|
||||
);
|
||||
const allHaveRefs = skillFiles.every((sf) => sf.codeEnvRef !== undefined);
|
||||
if (allHaveRefs) {
|
||||
const refsBySession = new Map<string, CodeEnvRef>();
|
||||
for (const sf of skillFiles) {
|
||||
const ref = sf.codeEnvRef;
|
||||
if (ref && !refsBySession.has(ref.storage_session_id)) {
|
||||
refsBySession.set(ref.storage_session_id, ref);
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
const checkResults = await Promise.all(
|
||||
Array.from(sessionIds).map(async (sid) => {
|
||||
const representative = skillFiles.find((sf) =>
|
||||
sf.codeEnvIdentifier!.startsWith(`${sid}/`),
|
||||
);
|
||||
if (!representative) {
|
||||
return false;
|
||||
}
|
||||
const lastModified = await getSessionInfo(representative.codeEnvIdentifier!);
|
||||
Array.from(refsBySession.values()).map(async (ref) => {
|
||||
const lastModified = await getSessionInfo(ref);
|
||||
return !!(lastModified && checkIfActive(lastModified));
|
||||
}),
|
||||
);
|
||||
|
|
@ -121,22 +137,30 @@ export async function primeSkillFiles(
|
|||
if (allActive) {
|
||||
const files: PrimeSkillFilesResult['files'] = [];
|
||||
for (const sf of skillFiles) {
|
||||
const identifier = sf.codeEnvIdentifier as string;
|
||||
const [sid, fid] = identifier.split('?')[0].split('/');
|
||||
const entity_id = parseEntityIdFromCodeEnvIdentifier(identifier);
|
||||
const ref = sf.codeEnvRef;
|
||||
if (!ref) continue;
|
||||
/* Cache-hit refs already carry resource identity (kind / id /
|
||||
* version) — pull them through so the artifact emitted by
|
||||
* `handle_skill` and forwarded to `_injected_files` includes
|
||||
* `resource_id`. Without this the next /exec sends
|
||||
* `resource_id: undefined` and codeapi 400s. The discriminated
|
||||
* union pins `version` to the skill branch only — destructure
|
||||
* before the spread so TS accepts the conditional pull. */
|
||||
files.push({
|
||||
id: fid,
|
||||
session_id: sid,
|
||||
id: ref.file_id,
|
||||
resource_id: ref.id,
|
||||
storage_session_id: ref.storage_session_id,
|
||||
name: `${skill.name}/${sf.relativePath}`,
|
||||
...(entity_id != null ? { entity_id } : {}),
|
||||
kind: ref.kind,
|
||||
...(ref.kind === 'skill' ? { version: ref.version } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
if (files.length > 0) {
|
||||
logger.debug(
|
||||
`[primeSkillFiles] All ${sessionIds.size} session(s) active for skill "${skill.name}", reusing ${files.length} files`,
|
||||
`[primeSkillFiles] All ${refsBySession.size} session(s) active for skill "${skill.name}", reusing ${files.length} files`,
|
||||
);
|
||||
return { session_id: files[0].session_id, files };
|
||||
return { storage_session_id: files[0].storage_session_id, files };
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
|
|
@ -183,7 +207,13 @@ export async function primeSkillFiles(
|
|||
const result = await batchUploadCodeEnvFiles({
|
||||
req,
|
||||
files: filesToUpload,
|
||||
entity_id: entityId,
|
||||
/* Resource identity for codeapi's sessionKey: skill files share
|
||||
* cross-user-within-tenant under `<tenant>:skill:<id>:v:<version>`.
|
||||
* Bumping `skill.version` on edit naturally invalidates the prior
|
||||
* cache entry under the new sessionKey. */
|
||||
kind: 'skill',
|
||||
id: entityId,
|
||||
version: skill.version,
|
||||
/* Skill files are infrastructure: SKILL.md + bundled scripts/schemas/
|
||||
* docs that the agent reads but should never edit. Tag the upload as
|
||||
* read-only so codeapi seals the inputs (chmod 444 in-sandbox) and
|
||||
|
|
@ -194,15 +224,20 @@ export async function primeSkillFiles(
|
|||
read_only: true,
|
||||
});
|
||||
// Exclude SKILL.md from the returned files array — it is uploaded to disk
|
||||
// for bash access but has no codeEnvIdentifier (cannot be cached). Omitting
|
||||
// it here keeps the fresh-upload and cache-hit code paths consistent.
|
||||
const files = result.files
|
||||
// for bash access but has no codeEnvRef (cannot be cached). Omitting it
|
||||
// here keeps the fresh-upload and cache-hit code paths consistent.
|
||||
const files: PrimeSkillFilesResult['files'] = result.files
|
||||
.filter((f) => !f.filename.endsWith('/SKILL.md'))
|
||||
.map((f) => ({
|
||||
id: f.fileId,
|
||||
session_id: result.session_id,
|
||||
/* `resource_id` is the skill `_id` (the entity codeapi scopes
|
||||
* the sessionKey on). Distinct from `id` (the per-file storage
|
||||
* uuid) — both are required on the request. */
|
||||
resource_id: entityId,
|
||||
storage_session_id: result.storage_session_id,
|
||||
name: f.filename,
|
||||
entity_id: entityId,
|
||||
kind: 'skill',
|
||||
version: skill.version,
|
||||
}));
|
||||
|
||||
// Treat partial upload failures as a priming failure — missing bundled
|
||||
|
|
@ -220,39 +255,48 @@ export async function primeSkillFiles(
|
|||
}
|
||||
|
||||
/**
|
||||
* Persist codeEnvIdentifiers on skill files. Awaited (not
|
||||
* fire-and-forget) so the next prime — which can start within
|
||||
* milliseconds when many users hit the same skill concurrently —
|
||||
* sees the cache pointer instead of racing the read against an
|
||||
* in-flight write. Without the await, a fire-and-forget under
|
||||
* concurrency stays in cache-miss steady-state for the duration
|
||||
* of the burst (each user's prime reads stale, re-uploads, then
|
||||
* fires its own forget that the next user also misses). Latency
|
||||
* cost is ~10–50ms on the prime that does the upload; subsequent
|
||||
* primes save an entire batch upload. Failures don't fail the
|
||||
* prime — the file refs returned to the caller are still valid.
|
||||
* Persist codeEnvRefs on skill files. Awaited (not fire-and-forget)
|
||||
* so the next prime — which can start within milliseconds when
|
||||
* many users hit the same skill concurrently — sees the cache
|
||||
* pointer instead of racing the read against an in-flight write.
|
||||
* Without the await, a fire-and-forget under concurrency stays in
|
||||
* cache-miss steady-state for the duration of the burst (each
|
||||
* user's prime reads stale, re-uploads, then fires its own forget
|
||||
* that the next user also misses). Latency cost is ~10–50ms on
|
||||
* the prime that does the upload; subsequent primes save an entire
|
||||
* batch upload. Failures don't fail the prime — the file refs
|
||||
* returned to the caller are still valid.
|
||||
*/
|
||||
if (updateSkillFileCodeEnvIds) {
|
||||
const updates = result.files
|
||||
.filter((f) => !f.filename.endsWith('/SKILL.md'))
|
||||
.map((f) => ({
|
||||
skillId: skill._id,
|
||||
relativePath: f.filename.slice(f.filename.indexOf('/') + 1),
|
||||
codeEnvIdentifier: `${result.session_id}/${f.fileId}?entity_id=${entityId}`,
|
||||
}));
|
||||
.map((f) => {
|
||||
const ref: CodeEnvRef = {
|
||||
kind: 'skill',
|
||||
id: entityId,
|
||||
storage_session_id: result.storage_session_id,
|
||||
file_id: f.fileId,
|
||||
version: skill.version,
|
||||
};
|
||||
return {
|
||||
skillId: skill._id,
|
||||
relativePath: f.filename.slice(f.filename.indexOf('/') + 1),
|
||||
codeEnvRef: ref,
|
||||
};
|
||||
});
|
||||
if (updates.length > 0) {
|
||||
try {
|
||||
await updateSkillFileCodeEnvIds(updates);
|
||||
} catch (err: unknown) {
|
||||
logger.warn(
|
||||
'[primeSkillFiles] Failed to persist codeEnvIdentifiers:',
|
||||
'[primeSkillFiles] Failed to persist codeEnvRefs:',
|
||||
err instanceof Error ? err.message : err,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return { session_id: result.session_id, files };
|
||||
return { storage_session_id: result.storage_session_id, files };
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
`[primeSkillFiles] Batch upload failed for skill "${skill.name}":`,
|
||||
|
|
@ -274,7 +318,13 @@ export interface PrimeInvokedSkillsDeps {
|
|||
getSkillByName: (
|
||||
name: string,
|
||||
accessibleIds: Types.ObjectId[],
|
||||
) => Promise<{ body: string; name: string; _id: Types.ObjectId; fileCount: number } | null>;
|
||||
) => Promise<{
|
||||
body: string;
|
||||
name: string;
|
||||
_id: Types.ObjectId;
|
||||
version: number;
|
||||
fileCount: number;
|
||||
} | null>;
|
||||
listSkillFiles: (skillId: Types.ObjectId | string) => Promise<SkillFileRecord[]>;
|
||||
getStrategyFunctions: PrimeSkillFilesParams['getStrategyFunctions'];
|
||||
batchUploadCodeEnvFiles: PrimeSkillFilesParams['batchUploadCodeEnvFiles'];
|
||||
|
|
@ -324,6 +374,7 @@ export async function primeInvokedSkills(
|
|||
body: string;
|
||||
name: string;
|
||||
_id: Types.ObjectId;
|
||||
version: number;
|
||||
fileCount: number;
|
||||
}> = [];
|
||||
for (const r of resolveResults) {
|
||||
|
|
@ -353,23 +404,24 @@ export async function primeInvokedSkills(
|
|||
// ALL distinct sessions for freshness. If all are active, return cached
|
||||
// references with zero re-uploads. If any expired, re-upload everything.
|
||||
if (deps.getSessionInfo && deps.checkIfActive) {
|
||||
const allFiles = fileListResults.flatMap((r) => r.files);
|
||||
const allFilesWithIds = allFiles.filter((f) => f.codeEnvIdentifier);
|
||||
const allResolved = fileListResults.flatMap((r) =>
|
||||
r.files.map((f) => ({ skillName: r.skill.name, file: f, ref: f.codeEnvRef })),
|
||||
);
|
||||
const resolvedWithRef = allResolved.filter((x) => x.ref !== undefined);
|
||||
|
||||
// Only use cache when ALL files have identifiers (no partial persistence)
|
||||
if (allFilesWithIds.length > 0 && allFilesWithIds.length === allFiles.length) {
|
||||
const sessionIds = new Set(
|
||||
allFilesWithIds.map((f) => f.codeEnvIdentifier!.split('?')[0].split('/')[0]),
|
||||
);
|
||||
// Only use cache when ALL files have refs (no partial persistence)
|
||||
if (resolvedWithRef.length > 0 && resolvedWithRef.length === allResolved.length) {
|
||||
const refsBySession = new Map<string, CodeEnvRef>();
|
||||
for (const { ref } of resolvedWithRef) {
|
||||
if (ref && !refsBySession.has(ref.storage_session_id)) {
|
||||
refsBySession.set(ref.storage_session_id, ref);
|
||||
}
|
||||
}
|
||||
|
||||
const checkResults = await Promise.all(
|
||||
Array.from(sessionIds).map(async (sid) => {
|
||||
const representative = allFilesWithIds.find((f) =>
|
||||
f.codeEnvIdentifier!.startsWith(`${sid}/`),
|
||||
);
|
||||
if (!representative) return true;
|
||||
Array.from(refsBySession.values()).map(async (ref) => {
|
||||
try {
|
||||
const lastModified = await deps.getSessionInfo?.(representative.codeEnvIdentifier!);
|
||||
const lastModified = await deps.getSessionInfo?.(ref);
|
||||
return !!(lastModified && deps.checkIfActive?.(lastModified));
|
||||
} catch {
|
||||
return false;
|
||||
|
|
@ -379,30 +431,33 @@ export async function primeInvokedSkills(
|
|||
const allActive = checkResults.every(Boolean);
|
||||
|
||||
if (allActive) {
|
||||
const cachedFiles = fileListResults.flatMap((r) =>
|
||||
r.files
|
||||
.filter((f) => f.codeEnvIdentifier)
|
||||
.map((f) => {
|
||||
const identifier = f.codeEnvIdentifier as string;
|
||||
const [sid, fid] = identifier.split('?')[0].split('/');
|
||||
const entity_id = parseEntityIdFromCodeEnvIdentifier(identifier);
|
||||
return {
|
||||
id: fid,
|
||||
name: `${r.skill.name}/${f.relativePath}`,
|
||||
session_id: sid,
|
||||
...(entity_id != null ? { entity_id } : {}),
|
||||
};
|
||||
}),
|
||||
);
|
||||
/* `id` is the STORAGE file_id (the per-file uuid the
|
||||
* file_server registered the upload under); `resource_id`
|
||||
* is the entity that owns the storage session — the
|
||||
* skill's `_id` here. codeapi's auth layer needs both:
|
||||
* `id` for the upload-existence check, `resource_id` for
|
||||
* sessionKey re-derivation (`<tenant>:skill:<resource_id>:v:<version>`).
|
||||
* Conflating them sent the storage nanoid through the
|
||||
* sessionKey switch and 403'd every shared-kind /exec. */
|
||||
const cachedFiles = resolvedWithRef.map(({ skillName, file, ref }) => ({
|
||||
id: ref!.file_id,
|
||||
resource_id: ref!.id,
|
||||
name: `${skillName}/${file.relativePath}`,
|
||||
storage_session_id: ref!.storage_session_id,
|
||||
kind: ref!.kind,
|
||||
...(ref!.kind === 'skill' ? { version: ref!.version } : {}),
|
||||
}));
|
||||
if (cachedFiles.length > 0) {
|
||||
logger.debug(
|
||||
`[primeInvokedSkills] All ${sessionIds.size} session(s) active, reusing ${cachedFiles.length} cached files`,
|
||||
`[primeInvokedSkills] All ${refsBySession.size} session(s) active, reusing ${cachedFiles.length} cached files`,
|
||||
);
|
||||
sessions = new Map();
|
||||
// session_id is a representative value. ToolNode uses per-file
|
||||
// session_id from the files array (file.session_id ?? codeSession.session_id).
|
||||
/* `session_id` at the top of CodeSessionContext is the
|
||||
* (representative) execution session — ToolNode reads it
|
||||
* for continuity. Per-file storage is on each
|
||||
* `files[i].storage_session_id`. */
|
||||
sessions.set(Constants.EXECUTE_CODE, {
|
||||
session_id: cachedFiles[0].session_id,
|
||||
session_id: cachedFiles[0].storage_session_id,
|
||||
files: cachedFiles,
|
||||
lastUpdated: Date.now(),
|
||||
} satisfies CodeSessionContext);
|
||||
|
|
@ -412,14 +467,18 @@ export async function primeInvokedSkills(
|
|||
}
|
||||
}
|
||||
|
||||
// Per-skill upload: each skill gets its own session with entity_id=skillId.
|
||||
// primeSkillFiles handles freshness caching per-skill, so only expired
|
||||
// skills re-upload. The code env handles mixed session_ids natively.
|
||||
// Per-skill upload: each skill gets its own storage session keyed
|
||||
// by `(kind: 'skill', id: skillId, version: skill.version)`.
|
||||
// primeSkillFiles handles freshness caching per-skill, so only
|
||||
// expired skills re-upload. CodeAPI handles mixed
|
||||
// storage_session_ids natively.
|
||||
const allPrimedFiles: Array<{
|
||||
id: string;
|
||||
resource_id: string;
|
||||
name: string;
|
||||
session_id: string;
|
||||
entity_id?: string;
|
||||
storage_session_id: string;
|
||||
kind: 'skill';
|
||||
version: number;
|
||||
}> = [];
|
||||
const primeResults = await Promise.allSettled(
|
||||
fileListResults.map(async ({ skill, files }) => {
|
||||
|
|
@ -441,9 +500,14 @@ export async function primeInvokedSkills(
|
|||
for (const f of r.value.result.files) {
|
||||
allPrimedFiles.push({
|
||||
id: f.id,
|
||||
/* `resource_id` is the skill's `_id` — drives codeapi's
|
||||
* sessionKey re-derivation. See cachedFiles above for the
|
||||
* full id-vs-resource_id rationale. */
|
||||
resource_id: r.value.skill._id.toString(),
|
||||
name: f.name,
|
||||
session_id: f.session_id,
|
||||
...(f.entity_id != null ? { entity_id: f.entity_id } : {}),
|
||||
storage_session_id: f.storage_session_id,
|
||||
kind: 'skill',
|
||||
version: r.value.skill.version,
|
||||
});
|
||||
}
|
||||
} else if (r.status === 'rejected') {
|
||||
|
|
@ -453,11 +517,11 @@ export async function primeInvokedSkills(
|
|||
|
||||
if (allPrimedFiles.length > 0) {
|
||||
sessions = new Map();
|
||||
// session_id is a representative value (first skill's session). ToolNode
|
||||
// uses per-file session_id from the files array (file.session_id ??
|
||||
// codeSession.session_id), so mixed sessions work correctly.
|
||||
/* `session_id` at the top of CodeSessionContext is the
|
||||
* (representative) execution session. Per-file storage is on
|
||||
* each file's `storage_session_id`. */
|
||||
sessions.set(Constants.EXECUTE_CODE, {
|
||||
session_id: allPrimedFiles[0].session_id,
|
||||
session_id: allPrimedFiles[0].storage_session_id,
|
||||
files: allPrimedFiles,
|
||||
lastUpdated: Date.now(),
|
||||
} satisfies CodeSessionContext);
|
||||
|
|
|
|||
116
packages/api/src/files/code/identity.spec.ts
Normal file
116
packages/api/src/files/code/identity.spec.ts
Normal file
|
|
@ -0,0 +1,116 @@
|
|||
import FormData from 'form-data';
|
||||
import { appendCodeEnvFileIdentity, buildCodeEnvDownloadQuery } from './identity';
|
||||
|
||||
describe('appendCodeEnvFileIdentity', () => {
|
||||
/* Validation rules mirror codeapi's `parseUploadSessionKeyInput`
|
||||
* server-side validator. Asserting client-side gives bad callers a
|
||||
* synchronous failure with file path + line number, vs. round-tripping
|
||||
* a 400 from a multipart POST. */
|
||||
|
||||
it('appends kind/id/version on the multipart form for skill uploads', () => {
|
||||
const form = new FormData();
|
||||
appendCodeEnvFileIdentity(form, { kind: 'skill', id: 'skill-42', version: 7 });
|
||||
|
||||
/* Spy on the form internally — `getBuffer()` chokes on stream
|
||||
* entries other tests append, but the form has only the three
|
||||
* scalar fields here so dump-and-grep is safe. */
|
||||
const body = form.getBuffer().toString();
|
||||
expect(body).toContain('name="kind"');
|
||||
expect(body).toContain('skill');
|
||||
expect(body).toContain('name="id"');
|
||||
expect(body).toContain('skill-42');
|
||||
expect(body).toContain('name="version"');
|
||||
expect(body).toContain('7');
|
||||
});
|
||||
|
||||
it('omits version on the multipart form for non-skill kinds', () => {
|
||||
const form = new FormData();
|
||||
appendCodeEnvFileIdentity(form, { kind: 'agent', id: 'agent-9' });
|
||||
expect(form.getBuffer().toString()).not.toContain('name="version"');
|
||||
});
|
||||
|
||||
it('rejects unknown kinds before mutating the form', () => {
|
||||
const form = new FormData();
|
||||
expect(() =>
|
||||
appendCodeEnvFileIdentity(form, {
|
||||
kind: 'system' as 'user',
|
||||
id: 'x',
|
||||
}),
|
||||
).toThrow(/invalid kind/);
|
||||
/* form-data emits multipart boundaries even on an empty form, so
|
||||
* "did we append" is a substring check, not buffer equality. */
|
||||
expect(form.getBuffer().toString()).not.toContain('name="kind"');
|
||||
expect(form.getBuffer().toString()).not.toContain('name="id"');
|
||||
});
|
||||
|
||||
it('rejects skill identity without a version', () => {
|
||||
expect(() =>
|
||||
appendCodeEnvFileIdentity(new FormData(), { kind: 'skill', id: 'skill-42' }),
|
||||
).toThrow(/skill.*version/);
|
||||
});
|
||||
|
||||
it('rejects version on non-skill kinds', () => {
|
||||
expect(() =>
|
||||
appendCodeEnvFileIdentity(new FormData(), {
|
||||
kind: 'agent',
|
||||
id: 'agent-9',
|
||||
version: 3,
|
||||
}),
|
||||
).toThrow(/version.*skill/);
|
||||
});
|
||||
|
||||
it('rejects missing id', () => {
|
||||
expect(() => appendCodeEnvFileIdentity(new FormData(), { kind: 'user', id: '' })).toThrow(
|
||||
/missing id/,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildCodeEnvDownloadQuery', () => {
|
||||
it('builds the canonical user-private query string', () => {
|
||||
expect(buildCodeEnvDownloadQuery({ kind: 'user', id: 'user-123' })).toBe(
|
||||
'?kind=user&id=user-123',
|
||||
);
|
||||
});
|
||||
|
||||
it('appends version when kind is skill', () => {
|
||||
expect(buildCodeEnvDownloadQuery({ kind: 'skill', id: 'skill-abc', version: 7 })).toBe(
|
||||
'?kind=skill&id=skill-abc&version=7',
|
||||
);
|
||||
});
|
||||
|
||||
it('omits version on agent kind', () => {
|
||||
expect(buildCodeEnvDownloadQuery({ kind: 'agent', id: 'agent-xyz' })).toBe(
|
||||
'?kind=agent&id=agent-xyz',
|
||||
);
|
||||
});
|
||||
|
||||
it('URL-encodes ids that contain special characters', () => {
|
||||
/* `id` shouldn't normally contain unsafe chars (it's a uuid or
|
||||
* mongo ObjectId), but the URLSearchParams plumbing handles it
|
||||
* uniformly so a future call site that passes through a
|
||||
* user-supplied identifier doesn't accidentally produce a
|
||||
* malformed URL. */
|
||||
expect(buildCodeEnvDownloadQuery({ kind: 'user', id: 'a b/c?d' })).toMatch(
|
||||
/\?kind=user&id=a\+b%2Fc%3Fd/,
|
||||
);
|
||||
});
|
||||
|
||||
it('throws on unknown kind', () => {
|
||||
expect(() => buildCodeEnvDownloadQuery({ kind: 'system' as 'user', id: 'x' })).toThrow(
|
||||
/invalid kind/,
|
||||
);
|
||||
});
|
||||
|
||||
it('throws on skill missing version', () => {
|
||||
expect(() => buildCodeEnvDownloadQuery({ kind: 'skill', id: 'skill-42' })).toThrow(
|
||||
/skill.*version/,
|
||||
);
|
||||
});
|
||||
|
||||
it('throws on version with non-skill kind', () => {
|
||||
expect(() => buildCodeEnvDownloadQuery({ kind: 'agent', id: 'agent-9', version: 3 })).toThrow(
|
||||
/version.*skill/,
|
||||
);
|
||||
});
|
||||
});
|
||||
78
packages/api/src/files/code/identity.ts
Normal file
78
packages/api/src/files/code/identity.ts
Normal file
|
|
@ -0,0 +1,78 @@
|
|||
import { CODE_ENV_KINDS } from 'librechat-data-provider';
|
||||
import type { CodeEnvKind } from 'librechat-data-provider';
|
||||
import type FormData from 'form-data';
|
||||
|
||||
/**
|
||||
* Resource identity required by codeapi's sessionKey resolver. The
|
||||
* shape is symmetric across the upload form fields and the URL query
|
||||
* params on download/freshness routes — codeapi normalizes both into
|
||||
* the same `parseUploadSessionKeyInput`. Validation rules (closed
|
||||
* kind set, `version` required for `'skill'` and forbidden otherwise)
|
||||
* mirror codeapi's validator so a bad caller fails fast on the client
|
||||
* instead of round-tripping a 400. See codeapi #1455 / Phase C.
|
||||
*
|
||||
* `id` is sessionKey-meaningful for `'skill'` / `'agent'` and
|
||||
* informational-only for `'user'` (codeapi resolves user identity
|
||||
* from auth context). It's required uniformly here for shape
|
||||
* consistency with the runtime validator.
|
||||
*/
|
||||
export interface CodeEnvIdentity {
|
||||
kind: CodeEnvKind;
|
||||
id: string;
|
||||
/** Required when `kind === 'skill'`; forbidden otherwise. */
|
||||
version?: number;
|
||||
}
|
||||
|
||||
const VALID_KINDS = new Set<string>(CODE_ENV_KINDS);
|
||||
|
||||
const validateIdentity = (identity: CodeEnvIdentity, callerLabel: string): void => {
|
||||
const { kind, id, version } = identity;
|
||||
if (!kind || !VALID_KINDS.has(kind)) {
|
||||
throw new Error(`${callerLabel}: invalid kind "${kind}"`);
|
||||
}
|
||||
if (!id) {
|
||||
throw new Error(`${callerLabel}: missing id for kind "${kind}"`);
|
||||
}
|
||||
if (kind === 'skill' && version == null) {
|
||||
throw new Error(`${callerLabel}: kind "skill" requires a numeric version`);
|
||||
}
|
||||
if (kind !== 'skill' && version != null) {
|
||||
throw new Error(`${callerLabel}: version is only valid for kind "skill"`);
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Append `kind`/`id`/`version?` form fields to a multipart upload.
|
||||
* Used by `uploadCodeEnvFile` and `batchUploadCodeEnvFiles` so codeapi
|
||||
* can route the upload to the correct sessionKey bucket
|
||||
* (`<tenant>:<kind>:<id>[:v:<version>]` for shared kinds,
|
||||
* `<tenant>:user:<authContext.userId>` for `'user'`).
|
||||
*
|
||||
* Mutates the form in place and throws synchronously on bad input.
|
||||
*/
|
||||
export function appendCodeEnvFileIdentity(form: FormData, identity: CodeEnvIdentity): void {
|
||||
validateIdentity(identity, 'appendCodeEnvFileIdentity');
|
||||
form.append('kind', identity.kind);
|
||||
form.append('id', identity.id);
|
||||
if (identity.version != null) {
|
||||
form.append('version', String(identity.version));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the `?kind=...&id=...&version=...` query string codeapi's
|
||||
* `sessionAuth` middleware requires on `/download/<session>/<file>`
|
||||
* and `/sessions/<session>/objects/<file>` URLs. Without these,
|
||||
* codeapi 400s with "kind must be one of: skill, agent, user" before
|
||||
* serving the file.
|
||||
*
|
||||
* Returns a string with leading `?`; concatenate onto a path.
|
||||
*/
|
||||
export function buildCodeEnvDownloadQuery(identity: CodeEnvIdentity): string {
|
||||
validateIdentity(identity, 'buildCodeEnvDownloadQuery');
|
||||
const params = new URLSearchParams({ kind: identity.kind, id: identity.id });
|
||||
if (identity.version != null) {
|
||||
params.set('version', String(identity.version));
|
||||
}
|
||||
return `?${params.toString()}`;
|
||||
}
|
||||
|
|
@ -1,3 +1,4 @@
|
|||
export * from './classify';
|
||||
export * from './extract';
|
||||
export * from './form';
|
||||
export * from './identity';
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@ function categorizeFile(
|
|||
type?: string | null;
|
||||
source?: string;
|
||||
embedded?: boolean;
|
||||
metadata?: { fileIdentifier?: string };
|
||||
metadata?: { fileIdentifier?: string; codeEnvRef?: unknown };
|
||||
},
|
||||
isBedrock: boolean,
|
||||
mergedFileConfig: FileConfig | undefined,
|
||||
|
|
@ -26,7 +26,11 @@ function categorizeFile(
|
|||
if (source === FileSources.text) {
|
||||
return 'skipped';
|
||||
}
|
||||
if (file.embedded === true || file.metadata?.fileIdentifier != null) {
|
||||
if (
|
||||
file.embedded === true ||
|
||||
file.metadata?.codeEnvRef != null ||
|
||||
file.metadata?.fileIdentifier != null
|
||||
) {
|
||||
return 'skipped';
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,37 @@
|
|||
import { Buffer } from 'buffer';
|
||||
import axios from 'axios';
|
||||
import { logger } from '@librechat/data-schemas';
|
||||
import type { AxiosInstance, AxiosProxyConfig, AxiosError } from 'axios';
|
||||
|
||||
/**
|
||||
* Render `error.response.data` as a small, log-safe value. Axios surfaces
|
||||
* the response body in whichever native shape the request asked for, so
|
||||
* `responseType: 'arraybuffer'` yields a `Buffer` (raw bytes — JSON-
|
||||
* serializes as `{type: 'Buffer', data: [123, 34, ...]}`, ~4 chars per
|
||||
* byte) and `responseType: 'stream'` yields a `Readable` (whose internal
|
||||
* state JSON-serializes the full readableState ring buffer + socket
|
||||
* fields, easily megabytes per error). Both are useless for diagnostics
|
||||
* and drown the log line. Decode small buffers as UTF-8 (truncated) and
|
||||
* stub streams entirely.
|
||||
*/
|
||||
const renderResponseData = (data: unknown): unknown => {
|
||||
if (data == null) return data;
|
||||
if (Buffer.isBuffer(data)) {
|
||||
const MAX = 2048;
|
||||
const text = data.subarray(0, MAX).toString('utf8');
|
||||
return data.length > MAX ? `${text}…[+${data.length - MAX} bytes]` : text;
|
||||
}
|
||||
// Readable streams (responseType: 'stream') and other piped sources.
|
||||
if (
|
||||
typeof data === 'object' &&
|
||||
data !== null &&
|
||||
typeof (data as { pipe?: unknown }).pipe === 'function'
|
||||
) {
|
||||
return '[stream]';
|
||||
}
|
||||
return data;
|
||||
};
|
||||
|
||||
/**
|
||||
* Logs Axios errors based on the error object and a custom message.
|
||||
* @param options - The options object.
|
||||
|
|
@ -33,7 +63,7 @@ export const logAxiosError = ({
|
|||
logger.error(logMessage, {
|
||||
status,
|
||||
headers,
|
||||
data,
|
||||
data: renderResponseData(data),
|
||||
stack,
|
||||
});
|
||||
} else if (axios.isAxiosError(error) && error.request) {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue