mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-10-31 19:17:13 +00:00
fe30e3db32
* 🧹 fix: Stop Undeletable Files Starving the Retention Sweep `getExpiredFiles` returns the oldest `expiredAt` first, capped at `limit`, and `processDeleteRequest` leaves the record in place when storage deletion fails. Nothing records the failure, so the same files come back at the head of the next batch an hour later, forever: no backoff, no cap, and — once `limit` of them cannot be deleted — no file that expires afterwards is ever swept again. On the deployment behind #15511 that is ~29k stranded objects permanently occupying a 100-slot queue, which is why fixing the Code Interpreter side alone (LibreChat-AI/code-interpreter#85) would not have resumed deletion there. Failures are now recorded on the file. `deletionRetryAt` holds it back with a backoff doubling from one sweep interval to a day, and `deletionAttempts` retires it from the query entirely once it reaches `FILE_RETENTION_SWEEP_MAX_ATTEMPTS` (10, so roughly five days of retries). Both fields are absent on existing records and absence means "never attempted", so nothing already in the collection changes eligibility. The record itself is kept rather than deleted — the reference is what an operator needs to reconcile a bucket the sweep could not clear, and dropping it would restore the silence that made this leak invisible. Raising `FILE_RETENTION_SWEEP_MAX_ATTEMPTS` re-admits everything previously given up on, which is the supported way to resume once the storage-side failure is fixed; the give-up is logged with that instruction. The counter is incremented server-side with `$inc` so concurrent sweeps on separate nodes cannot overwrite each other's progress toward the cap, and a failure to record a failure is logged and skipped rather than aborting the rest of the batch. * 🧭 fix: Delete Code Environment Files Through the Route That Exists `deleteCodeEnvFile` tried `/sessions/:sid/objects/:fid` before falling back to `/files/:sid/:fid`. Both have been there since #13424, but only the second is mounted by any released codeapi — the first gained DELETE in LibreChat-AI/code-interpreter#85 — so every deletion paid a guaranteed 404 and a wasted round trip, and #15511 read that 404 as the whole bug. Call `/files/:sid/:fid` directly. It is the safe direction to collapse toward: codeapi has mounted it since its first release, so this works against older deployments as well as post-#85 ones, whereas keeping the other path would not. Collapsing the loop tightens two behaviours that only existed to serve it. A 405 now surfaces instead of being swallowed on the way to a second attempt; there is no second route to try, and a service that refuses the method should say so. A 404 is still treated as "already gone" — that is the only thing it can now mean — but it is logged rather than passed over in silence, because a 404 caused by a misconfigured base URL looks identical and this branch drops the file's metadata record either way. * 🔒 fix: Settle sweep retry state from the write, not the read Two findings from the review of0e52924. - `recordFailure` derived the attempt number by adding one to the count the batch had queried. Two nodes sweeping the same file read the same value, so both believed themselves to be the same attempt: each `$inc` landed, the stored count crossed `FILE_RETENTION_SWEEP_MAX_ATTEMPTS`, and neither caller ever saw the threshold. The file drops out of the query — the starvation guard still holds — but the give-up is never reported, and that log line is the operator's only notice, and carries the instruction for resuming. Return the count from the increment itself so every caller gets a distinct attempt number and exactly one observes the cap. The same staleness shortened the backoff, so `deferExpiredFile` now writes with `$max`: a deferral can only move later, and a node that computed a shorter delay cannot pull the file forward past one another node already committed. - The backoff doubled from a hard-coded hour while claiming to start from one sweep interval. At the default they coincide; away from it the schedule stops meaning anything — on a six-hour sweep the first three attempts all land on consecutive passes, and on a five-minute one the first retry skips twelve. Derive the base from `FILE_RETENTION_SWEEP_INTERVAL_MS`, floored at a minute so a pathologically short interval cannot spend the whole give-up budget on a transient outage. * 🧯 fix: Keep the give-up notice and the retry budget honest Four findings from the review ofee1bac5. - The give-up was reported after the deferral write, inside the same catch. Once the increment lands the counter is durable, so `getExpiredFiles` already excludes the file; a deferral that then failed left only the generic recording error and dropped the one line naming the file and saying how to resume. Report it as soon as the increment returns, and let the deferral fail on its own. - Retry deadlines were measured from `Date.now()` after the deletion I/O, but `startExpiredFileSweep` arms its interval before the sweep runs. A one-interval delay therefore expired just *after* the next scheduled pass, which skipped the file and pushed its first retry out by a whole extra interval — and the same drift applied to every delay that is an exact multiple. Anchor deadlines to the sweep's start instead. - The threshold used `>=`, so once two nodes pushed a file past the cap every one of them past it logged the give-up: one error per replica per exhausted file rather than one actionable notice. Only the attempt that lands exactly on the cap reports it. - Retry state outlived the content it described. `processCodeOutput` reuses a record for a repeated `(filename, conversationId)` — new bytes, new storage key, and a fresh `expiredAt` from `getRetentionExpiry` — while `createFile` and `updateFile` set only supplied fields. A record carried to the cap by its previous content stayed excluded from the sweep forever, stranding the new object exactly as this PR set out to prevent; a partially failed one started the new object's budget already spent. Both write paths now clear the fields: the budget belongs to the storage a record currently points at. The retry-state fixtures in `file.spec.ts` were seeded through `createFile`, which now clears them, so they drive the real methods the sweep uses. * 🎯 fix: Clear the retry budget only when a retention lifecycle starts Both findings from the review of8d26f9b, and both are consequences of that commit's reset rather than of the original change. - The reset was unconditional on every `createFile`/`updateFile`, on the reasoning that those paths write content. Two of them do not. `prepareImages{Local,Azure,Firebase}` call `updateFile({ file_id })` with nothing but the id — a TTL touch — every time an existing image is encoded for another chat, and the deferred preview uses the same method to transition `status`. Either handed a stranded record a fresh set of attempts and another give-up notice, on repeat, defeating the cap for the files most likely to be stranded. Gate it on the write actually setting `expiredAt`. The budget belongs to a retention lifecycle: that is the write which starts a new one, it is what `processCodeOutput` supplies when it repurposes a record, and a record with no retention deadline is never swept, so the fields are inert there anyway. - Both retry writes bumped `updatedAt`. `processCodeOutput` falls back to `updatedAt` as the writer-order stamp for records that predate `metadata.sourceDispatchedAt`, so a failed sweep landing mid-harvest read as a newer content writer and the harvest dropped its attachment. Mark them `timestamps: false`, for the reason `claimCodeFile` already does: bookkeeping is not a content write. * 🅿️ refactor: Park exhausted files instead of excluding them Three review rounds in a row found defects in how the give-up cap interacts with record reuse, each in the fix for the last. That is a design error, not a bug list: a permanent exclusion has to be bound precisely to the content lifecycle it was recorded against, and File records outlive their content. `processCodeOutput` repurposes a row for a repeated `(filename, conversationId)`, `createFile`/`updateFile` set only supplied fields, and `getRetentionExpiry` returns `{}` on a lookup failure so the row inherits its old deadline — three separate ways for bookkeeping to survive into a lifecycle it does not describe, each needing its own guard, and the two retry writes needing to be lifecycle-conditional on top. Remove the category instead. `deletionRetryAt` becomes the sweep's only hold, and reaching `FILE_RETENTION_SWEEP_MAX_ATTEMPTS` parks the file for a month rather than excluding it. The bound on the batch is the same — a stranded file costs one slot a month instead of one an hour — but a deadline that outlives its content can only delay the next object, never lose it, so nothing outside the sweep has to reason about this state at all. That deletes more than it adds: - `createFile` and `updateFile` go back to their original form. No reset, so no question of which writes install content, and `prepareImages*` and the deferred preview stop mattering here. - `getExpiredFiles` loses `maxAttempts` and its `$and`; eligibility is one `$or` on the deadline. - The interleaving race between the increment and the deferral degrades from a stranded object to a delayed one. Working through a large backlog is throughput-bound either way: every attempt costs a slot in the bounded batch, so N stranded files need N × `FILE_RETENTION_SWEEP_MAX_ATTEMPTS` passes to settle. Lower that limit when recovering a deployment that has accumulated many.
596 lines
22 KiB
JavaScript
596 lines
22 KiB
JavaScript
const http = require('http');
|
||
const https = require('https');
|
||
const { Readable } = require('stream');
|
||
|
||
const mockAxios = jest.fn();
|
||
mockAxios.post = jest.fn();
|
||
|
||
jest.mock('@librechat/agents', () => ({
|
||
getCodeBaseURL: jest.fn(() => 'https://code-api.example.com'),
|
||
}));
|
||
|
||
/* Inline the identity helpers' validation rules instead of pulling
|
||
* them through `@librechat/api`'s root barrel (which has init-time
|
||
* provider-config side effects that don't matter here) or its leaf
|
||
* module (the package's `exports` field only surfaces the root).
|
||
* The real implementation lives in `packages/api/src/files/code/identity.ts`
|
||
* and has its own dedicated `identity.spec.ts` covering the validation
|
||
* matrix; this stub just mirrors enough behavior for the surrounding
|
||
* crud tests to exercise the upload/download flow. */
|
||
const VALID_KINDS = new Set(['skill', 'agent', 'user']);
|
||
const validateIdentity = ({ kind, id, version }, label) => {
|
||
if (!kind || !VALID_KINDS.has(kind)) throw new Error(`${label}: invalid kind "${kind}"`);
|
||
if (!id) throw new Error(`${label}: missing id for kind "${kind}"`);
|
||
if (kind === 'skill' && version == null) {
|
||
throw new Error(`${label}: kind "skill" requires a numeric version`);
|
||
}
|
||
if (kind !== 'skill' && version != null) {
|
||
throw new Error(`${label}: version is only valid for kind "skill"`);
|
||
}
|
||
};
|
||
|
||
jest.mock('@librechat/api', () => {
|
||
const http = require('http');
|
||
const https = require('https');
|
||
return {
|
||
appendCodeEnvFile: jest.fn((form, stream, filename) => {
|
||
form.append('file', stream, { filename });
|
||
}),
|
||
appendCodeEnvFileIdentity: jest.fn((form, identity) => {
|
||
validateIdentity(identity, 'appendCodeEnvFileIdentity');
|
||
form.append('kind', identity.kind);
|
||
form.append('id', identity.id);
|
||
if (identity.version != null) form.append('version', String(identity.version));
|
||
}),
|
||
buildCodeEnvDownloadQuery: jest.fn((identity) => {
|
||
validateIdentity(identity, 'buildCodeEnvDownloadQuery');
|
||
const params = new URLSearchParams({ kind: identity.kind, id: identity.id });
|
||
if (identity.version != null) params.set('version', String(identity.version));
|
||
return `?${params.toString()}`;
|
||
}),
|
||
logAxiosError: jest.fn(({ message }) => message),
|
||
getCodeApiAuthHeaders: jest.fn(async () => ({})),
|
||
getCodeExecutionBaseUrl: jest.fn((profile) =>
|
||
profile === 'stateful' ? 'https://code-stateful.example.com' : 'https://code-api.example.com',
|
||
),
|
||
codeExecutionHeaders: jest.fn(({ executionProfile, bridgeWorkerId }) => ({
|
||
'X-CodeAPI-Expected-Profile': executionProfile,
|
||
...(bridgeWorkerId ? { 'X-LibreChat-Code-Worker-ID': bridgeWorkerId } : {}),
|
||
})),
|
||
CODE_API_EXPECTED_PROFILE_HEADER: 'X-CodeAPI-Expected-Profile',
|
||
createAxiosInstance: jest.fn(() => mockAxios),
|
||
codeServerHttpAgent: new http.Agent({ keepAlive: false }),
|
||
codeServerHttpsAgent: new https.Agent({ keepAlive: false }),
|
||
};
|
||
});
|
||
|
||
const {
|
||
codeServerHttpAgent,
|
||
codeServerHttpsAgent,
|
||
getCodeApiAuthHeaders,
|
||
getCodeExecutionBaseUrl,
|
||
} = require('@librechat/api');
|
||
const {
|
||
deleteCodeEnvFile,
|
||
getCodeOutputDownloadStream,
|
||
uploadCodeEnvFile,
|
||
batchUploadCodeEnvFiles,
|
||
} = require('./crud');
|
||
|
||
describe('Code CRUD', () => {
|
||
beforeEach(() => {
|
||
jest.clearAllMocks();
|
||
getCodeApiAuthHeaders.mockResolvedValue({});
|
||
});
|
||
|
||
describe('getCodeOutputDownloadStream', () => {
|
||
/* Code-output downloads always carry `kind: 'user'` + `id: <userId>`
|
||
* — codeapi's `sessionAuth` rejects without them post-Phase C. The
|
||
* fixture mirrors what `processCodeOutput` and the `/code/download`
|
||
* route pass in production. */
|
||
const userIdentity = { kind: 'user', id: 'user-123' };
|
||
|
||
it('should pass dedicated keepAlive:false agents to axios', async () => {
|
||
const mockResponse = { data: Readable.from(['chunk']) };
|
||
mockAxios.mockResolvedValue(mockResponse);
|
||
|
||
await getCodeOutputDownloadStream('session-1/file-1', userIdentity);
|
||
|
||
const callConfig = mockAxios.mock.calls[0][0];
|
||
expect(callConfig.httpAgent).toBe(codeServerHttpAgent);
|
||
expect(callConfig.httpsAgent).toBe(codeServerHttpsAgent);
|
||
expect(callConfig.httpAgent).toBeInstanceOf(http.Agent);
|
||
expect(callConfig.httpsAgent).toBeInstanceOf(https.Agent);
|
||
expect(callConfig.httpAgent.keepAlive).toBe(false);
|
||
expect(callConfig.httpsAgent.keepAlive).toBe(false);
|
||
});
|
||
|
||
it('should request stream response from the correct URL', async () => {
|
||
mockAxios.mockResolvedValue({ data: Readable.from(['chunk']) });
|
||
|
||
await getCodeOutputDownloadStream('session-1/file-1', userIdentity);
|
||
|
||
const callConfig = mockAxios.mock.calls[0][0];
|
||
/* URL carries `?kind=user&id=<userId>` so codeapi's `sessionAuth`
|
||
* can reconstruct the matching `<tenant>:user:<userId>` sessionKey
|
||
* (Phase C / option α). */
|
||
expect(callConfig.url).toBe(
|
||
'https://code-api.example.com/download/session-1/file-1?kind=user&id=user-123',
|
||
);
|
||
expect(callConfig.responseType).toBe('stream');
|
||
expect(callConfig.timeout).toBe(15000);
|
||
});
|
||
|
||
it('uses the trusted stateful route and fail-closed profile header', async () => {
|
||
mockAxios.mockResolvedValue({ data: Readable.from(['chunk']) });
|
||
|
||
await getCodeOutputDownloadStream('session-1/file-1', userIdentity, undefined, {
|
||
baseUrl: 'https://code-stateful.example.com',
|
||
executionProfile: 'stateful',
|
||
bridgeWorkerId: 'personal-worker-1',
|
||
});
|
||
|
||
expect(mockAxios).toHaveBeenCalledWith(
|
||
expect.objectContaining({
|
||
url: 'https://code-stateful.example.com/download/session-1/file-1?kind=user&id=user-123',
|
||
headers: expect.objectContaining({
|
||
'X-CodeAPI-Expected-Profile': 'stateful',
|
||
'X-LibreChat-Code-Worker-ID': 'personal-worker-1',
|
||
}),
|
||
}),
|
||
);
|
||
});
|
||
|
||
it('forwards Code API auth headers when a request is provided', async () => {
|
||
const req = { user: { id: 'user-123' } };
|
||
getCodeApiAuthHeaders.mockResolvedValue({ Authorization: 'Bearer codeapi-token' });
|
||
mockAxios.mockResolvedValue({ data: Readable.from(['chunk']) });
|
||
|
||
await getCodeOutputDownloadStream('session-1/file-1', userIdentity, req);
|
||
|
||
const callConfig = mockAxios.mock.calls[0][0];
|
||
expect(getCodeApiAuthHeaders).toHaveBeenCalledWith(req, undefined);
|
||
expect(callConfig.headers.Authorization).toBe('Bearer codeapi-token');
|
||
});
|
||
|
||
it('forwards skill identity (kind/id/version) when re-downloading a primed skill file', async () => {
|
||
mockAxios.mockResolvedValue({ data: Readable.from(['chunk']) });
|
||
|
||
await getCodeOutputDownloadStream('session-2/file-x', {
|
||
kind: 'skill',
|
||
id: 'skill-abc',
|
||
version: 7,
|
||
});
|
||
|
||
const callConfig = mockAxios.mock.calls[0][0];
|
||
expect(callConfig.url).toBe(
|
||
'https://code-api.example.com/download/session-2/file-x?kind=skill&id=skill-abc&version=7',
|
||
);
|
||
});
|
||
|
||
it('rejects skill identity without a version (mirrors codeapi validator)', async () => {
|
||
await expect(
|
||
getCodeOutputDownloadStream('s/f', { kind: 'skill', id: 'skill-abc' }),
|
||
).rejects.toThrow(/skill.*version/);
|
||
expect(mockAxios).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it('rejects unknown kind without dispatching to codeapi', async () => {
|
||
await expect(getCodeOutputDownloadStream('s/f', { kind: 'system', id: 'x' })).rejects.toThrow(
|
||
/invalid kind/,
|
||
);
|
||
expect(mockAxios).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it('should throw on network error', async () => {
|
||
mockAxios.mockRejectedValue(new Error('ECONNREFUSED'));
|
||
|
||
await expect(getCodeOutputDownloadStream('s/f', userIdentity)).rejects.toThrow();
|
||
});
|
||
});
|
||
|
||
describe('deleteCodeEnvFile', () => {
|
||
const req = { user: { id: 'user-123' } };
|
||
const file = {
|
||
metadata: {
|
||
codeEnvRef: {
|
||
kind: 'agent',
|
||
id: 'agent-abc',
|
||
storage_session_id: 'session-1',
|
||
file_id: 'file-1',
|
||
},
|
||
},
|
||
};
|
||
|
||
it('deletes the code environment object with resource identity and auth headers', async () => {
|
||
getCodeApiAuthHeaders.mockResolvedValue({ Authorization: 'Bearer codeapi-token' });
|
||
mockAxios.mockResolvedValue({ status: 204 });
|
||
|
||
await deleteCodeEnvFile(req, file);
|
||
|
||
expect(getCodeApiAuthHeaders).toHaveBeenCalledWith(req, undefined);
|
||
expect(mockAxios).toHaveBeenCalledWith(
|
||
expect.objectContaining({
|
||
method: 'delete',
|
||
url: 'https://code-api.example.com/files/session-1/file-1?kind=agent&id=agent-abc',
|
||
headers: expect.objectContaining({
|
||
Authorization: 'Bearer codeapi-token',
|
||
'User-Agent': 'LibreChat/1.0',
|
||
}),
|
||
httpAgent: codeServerHttpAgent,
|
||
httpsAgent: codeServerHttpsAgent,
|
||
timeout: 15000,
|
||
}),
|
||
);
|
||
});
|
||
|
||
it('deletes a stateful artifact from its originating profile', async () => {
|
||
mockAxios.mockResolvedValue({ status: 204 });
|
||
const statefulFile = {
|
||
metadata: {
|
||
codeEnvRef: {
|
||
...file.metadata.codeEnvRef,
|
||
executionProfile: 'stateful',
|
||
},
|
||
},
|
||
};
|
||
|
||
await deleteCodeEnvFile(req, statefulFile);
|
||
|
||
expect(mockAxios).toHaveBeenCalledWith(
|
||
expect.objectContaining({
|
||
url: 'https://code-stateful.example.com/files/session-1/file-1?kind=agent&id=agent-abc',
|
||
headers: expect.objectContaining({
|
||
'X-CodeAPI-Expected-Profile': 'stateful',
|
||
}),
|
||
}),
|
||
);
|
||
});
|
||
|
||
it('deletes every profile-local object retained for a shared file record', async () => {
|
||
mockAxios.mockResolvedValue({ status: 204 });
|
||
const dualProfileFile = {
|
||
metadata: {
|
||
codeEnvRef: file.metadata.codeEnvRef,
|
||
codeEnvRefs: {
|
||
default: file.metadata.codeEnvRef,
|
||
stateful: {
|
||
...file.metadata.codeEnvRef,
|
||
storage_session_id: 'stateful-session',
|
||
file_id: 'stateful-file',
|
||
executionProfile: 'stateful',
|
||
},
|
||
},
|
||
},
|
||
};
|
||
|
||
await deleteCodeEnvFile(req, dualProfileFile);
|
||
|
||
expect(mockAxios).toHaveBeenCalledTimes(2);
|
||
expect(mockAxios).toHaveBeenNthCalledWith(
|
||
1,
|
||
expect.objectContaining({
|
||
url: expect.stringContaining('/files/session-1/file-1'),
|
||
headers: expect.objectContaining({ 'X-CodeAPI-Expected-Profile': 'default' }),
|
||
}),
|
||
);
|
||
expect(mockAxios).toHaveBeenNthCalledWith(
|
||
2,
|
||
expect.objectContaining({
|
||
url: expect.stringContaining('/files/stateful-session/stateful-file'),
|
||
headers: expect.objectContaining({ 'X-CodeAPI-Expected-Profile': 'stateful' }),
|
||
}),
|
||
);
|
||
});
|
||
|
||
it('skips remote cleanup instead of falling back when a historical route is unmapped', async () => {
|
||
const historicalRoute = 'stateful:0123456789abcdef0123456789abcdef';
|
||
const historicalFile = {
|
||
metadata: {
|
||
codeEnvRefs: {
|
||
[historicalRoute]: {
|
||
...file.metadata.codeEnvRef,
|
||
executionProfile: 'stateful',
|
||
executionRouteKey: historicalRoute,
|
||
},
|
||
},
|
||
},
|
||
};
|
||
|
||
await expect(deleteCodeEnvFile(req, historicalFile)).resolves.toBeUndefined();
|
||
|
||
expect(mockAxios).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it('skips legacy stateful cleanup after its endpoint is retired', async () => {
|
||
getCodeExecutionBaseUrl.mockImplementationOnce(() => {
|
||
throw new Error('LIBRECHAT_CODE_BASEURL_STATEFUL is not configured');
|
||
});
|
||
const legacyStatefulFile = {
|
||
metadata: {
|
||
codeEnvRef: {
|
||
...file.metadata.codeEnvRef,
|
||
executionProfile: 'stateful',
|
||
},
|
||
},
|
||
};
|
||
|
||
await expect(deleteCodeEnvFile(req, legacyStatefulFile)).resolves.toBeUndefined();
|
||
|
||
expect(mockAxios).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it('never calls the file-server path that only newer codeapi mounts', async () => {
|
||
mockAxios.mockResolvedValue({ status: 204 });
|
||
|
||
await deleteCodeEnvFile(req, file);
|
||
|
||
expect(mockAxios).toHaveBeenCalledTimes(1);
|
||
expect(mockAxios).not.toHaveBeenCalledWith(
|
||
expect.objectContaining({
|
||
url: expect.stringContaining('/sessions/session-1/objects/file-1'),
|
||
}),
|
||
);
|
||
});
|
||
|
||
it('surfaces an unsupported delete method instead of retrying elsewhere', async () => {
|
||
mockAxios.mockRejectedValue(
|
||
Object.assign(new Error('method not allowed'), { response: { status: 405 } }),
|
||
);
|
||
|
||
await expect(deleteCodeEnvFile(req, file)).rejects.toThrow('method not allowed');
|
||
expect(mockAxios).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it('skips files without a code environment ref', async () => {
|
||
await deleteCodeEnvFile(req, {});
|
||
|
||
expect(mockAxios).not.toHaveBeenCalled();
|
||
expect(getCodeApiAuthHeaders).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it('treats missing code environment objects as already deleted', async () => {
|
||
mockAxios.mockRejectedValue(
|
||
Object.assign(new Error('missing'), { response: { status: 404 } }),
|
||
);
|
||
|
||
await expect(deleteCodeEnvFile(req, file)).resolves.toBeUndefined();
|
||
expect(mockAxios).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it('throws when code environment deletion fails', async () => {
|
||
mockAxios.mockRejectedValue(
|
||
Object.assign(new Error('unavailable'), { response: { status: 500 } }),
|
||
);
|
||
|
||
await expect(deleteCodeEnvFile(req, file)).rejects.toThrow('unavailable');
|
||
});
|
||
});
|
||
|
||
describe('uploadCodeEnvFile', () => {
|
||
const baseUploadParams = {
|
||
req: { user: { id: 'user-123' } },
|
||
stream: Readable.from(['file-content']),
|
||
filename: 'data.csv',
|
||
kind: 'user',
|
||
id: 'user-123',
|
||
};
|
||
|
||
it('should pass dedicated keepAlive:false agents to axios', async () => {
|
||
mockAxios.post.mockResolvedValue({
|
||
data: {
|
||
message: 'success',
|
||
storage_session_id: 'sess-1',
|
||
files: [{ fileId: 'fid-1', filename: 'data.csv' }],
|
||
},
|
||
});
|
||
|
||
await uploadCodeEnvFile(baseUploadParams);
|
||
|
||
const callConfig = mockAxios.post.mock.calls[0][2];
|
||
expect(callConfig.httpAgent).toBe(codeServerHttpAgent);
|
||
expect(callConfig.httpsAgent).toBe(codeServerHttpsAgent);
|
||
expect(callConfig.httpAgent).toBeInstanceOf(http.Agent);
|
||
expect(callConfig.httpsAgent).toBeInstanceOf(https.Agent);
|
||
expect(callConfig.httpAgent.keepAlive).toBe(false);
|
||
expect(callConfig.httpsAgent.keepAlive).toBe(false);
|
||
});
|
||
|
||
it('should set a timeout on upload requests', async () => {
|
||
mockAxios.post.mockResolvedValue({
|
||
data: {
|
||
message: 'success',
|
||
storage_session_id: 'sess-1',
|
||
files: [{ fileId: 'fid-1', filename: 'data.csv' }],
|
||
},
|
||
});
|
||
|
||
await uploadCodeEnvFile(baseUploadParams);
|
||
|
||
const callConfig = mockAxios.post.mock.calls[0][2];
|
||
expect(callConfig.timeout).toBe(120000);
|
||
});
|
||
|
||
it('should return { storage_session_id, file_id } on success', async () => {
|
||
mockAxios.post.mockResolvedValue({
|
||
data: {
|
||
message: 'success',
|
||
storage_session_id: 'sess-1',
|
||
files: [{ fileId: 'fid-1', filename: 'data.csv' }],
|
||
},
|
||
});
|
||
|
||
const result = await uploadCodeEnvFile(baseUploadParams);
|
||
expect(result).toEqual({ storage_session_id: 'sess-1', file_id: 'fid-1' });
|
||
});
|
||
|
||
it('forwards Code API auth headers on upload requests', async () => {
|
||
getCodeApiAuthHeaders.mockResolvedValue({ Authorization: 'Bearer codeapi-token' });
|
||
mockAxios.post.mockResolvedValue({
|
||
data: {
|
||
message: 'success',
|
||
storage_session_id: 'sess-1',
|
||
files: [{ fileId: 'fid-1', filename: 'data.csv' }],
|
||
},
|
||
});
|
||
|
||
await uploadCodeEnvFile(baseUploadParams);
|
||
|
||
const callConfig = mockAxios.post.mock.calls[0][2];
|
||
expect(getCodeApiAuthHeaders).toHaveBeenCalledWith(baseUploadParams.req, undefined);
|
||
expect(callConfig.headers.Authorization).toBe('Bearer codeapi-token');
|
||
});
|
||
|
||
it('routes uploads through the trusted stateful endpoint and profile header', async () => {
|
||
mockAxios.post.mockResolvedValue({
|
||
data: {
|
||
message: 'success',
|
||
storage_session_id: 'sess-1',
|
||
files: [{ fileId: 'fid-1', filename: 'data.csv' }],
|
||
},
|
||
});
|
||
|
||
await uploadCodeEnvFile({
|
||
...baseUploadParams,
|
||
codeApiBaseUrl: 'https://stateful-code.example.com',
|
||
executionProfile: 'stateful',
|
||
bridgeWorkerId: 'personal-worker-1',
|
||
});
|
||
|
||
const [url, , callConfig] = mockAxios.post.mock.calls[0];
|
||
expect(url).toBe('https://stateful-code.example.com/upload');
|
||
expect(callConfig.headers['X-CodeAPI-Expected-Profile']).toBe('stateful');
|
||
expect(callConfig.headers['X-LibreChat-Code-Worker-ID']).toBe('personal-worker-1');
|
||
expect(getCodeApiAuthHeaders).toHaveBeenCalledWith(baseUploadParams.req, 'personal-worker-1');
|
||
});
|
||
|
||
/* Phase C / option α (codeapi #1455): the upload wire carries the
|
||
* resource identity codeapi uses for sessionKey derivation. Without
|
||
* these on the form, codeapi falls back to user bucketing for every
|
||
* upload and skill-cache invalidation never fires. Validation runs
|
||
* client-side too so a bad caller fails fast instead of round-tripping
|
||
* a 400. */
|
||
describe('codeapi resource identity (kind/id/version)', () => {
|
||
const FormData = require('form-data');
|
||
const successResponse = {
|
||
data: {
|
||
message: 'success',
|
||
storage_session_id: 'sess-1',
|
||
files: [{ fileId: 'fid-1', filename: 'data.csv' }],
|
||
},
|
||
};
|
||
let appendSpy;
|
||
|
||
beforeEach(() => {
|
||
/* Spying on the prototype lets us assert form fields without
|
||
* materializing the multipart body — `form.getBuffer()` would
|
||
* fail on the file-stream entry, but we don't care about the
|
||
* stream here, only the identity fields that ride beside it. */
|
||
appendSpy = jest.spyOn(FormData.prototype, 'append');
|
||
});
|
||
|
||
afterEach(() => {
|
||
appendSpy.mockRestore();
|
||
});
|
||
|
||
const fieldsAppended = () =>
|
||
appendSpy.mock.calls
|
||
.filter((call) => typeof call[1] === 'string' || typeof call[1] === 'number')
|
||
.reduce((acc, [name, value]) => ({ ...acc, [name]: value }), {});
|
||
|
||
it('forwards kind, id, and (when skill) version on the multipart form', async () => {
|
||
mockAxios.post.mockResolvedValue(successResponse);
|
||
|
||
await uploadCodeEnvFile({
|
||
...baseUploadParams,
|
||
kind: 'skill',
|
||
id: 'skill-42',
|
||
version: 7,
|
||
});
|
||
|
||
expect(fieldsAppended()).toEqual({ kind: 'skill', id: 'skill-42', version: '7' });
|
||
});
|
||
|
||
it('omits version on the form for non-skill kinds', async () => {
|
||
mockAxios.post.mockResolvedValue(successResponse);
|
||
|
||
await uploadCodeEnvFile({ ...baseUploadParams, kind: 'agent', id: 'agent-9' });
|
||
|
||
const fields = fieldsAppended();
|
||
expect(fields).toEqual({ kind: 'agent', id: 'agent-9' });
|
||
expect(fields).not.toHaveProperty('version');
|
||
});
|
||
|
||
it('rejects unknown kind without dispatching to codeapi', async () => {
|
||
await expect(
|
||
uploadCodeEnvFile({ ...baseUploadParams, kind: 'system', id: 'x' }),
|
||
).rejects.toThrow(/invalid kind/);
|
||
expect(mockAxios.post).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it('rejects skill upload without a version (mirrors codeapi validator)', async () => {
|
||
await expect(
|
||
uploadCodeEnvFile({ ...baseUploadParams, kind: 'skill', id: 'skill-42' }),
|
||
).rejects.toThrow(/skill.*version/);
|
||
expect(mockAxios.post).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it('rejects version on non-skill kinds (mirrors codeapi validator)', async () => {
|
||
await expect(
|
||
uploadCodeEnvFile({
|
||
...baseUploadParams,
|
||
kind: 'agent',
|
||
id: 'agent-9',
|
||
version: 3,
|
||
}),
|
||
).rejects.toThrow(/version.*skill/);
|
||
expect(mockAxios.post).not.toHaveBeenCalled();
|
||
});
|
||
});
|
||
|
||
it('should throw when server returns non-success message', async () => {
|
||
mockAxios.post.mockResolvedValue({
|
||
data: { message: 'quota_exceeded', storage_session_id: 's', files: [] },
|
||
});
|
||
|
||
await expect(uploadCodeEnvFile(baseUploadParams)).rejects.toThrow('quota_exceeded');
|
||
});
|
||
|
||
it('should throw on network error', async () => {
|
||
mockAxios.post.mockRejectedValue(new Error('ECONNREFUSED'));
|
||
|
||
await expect(uploadCodeEnvFile(baseUploadParams)).rejects.toThrow();
|
||
});
|
||
});
|
||
|
||
describe('batchUploadCodeEnvFiles', () => {
|
||
it('routes batch uploads through the selected bridge worker', async () => {
|
||
const req = { user: { id: 'user-123' } };
|
||
mockAxios.post.mockResolvedValue({
|
||
data: {
|
||
message: 'success',
|
||
storage_session_id: 'sess-1',
|
||
files: [{ status: 'success', fileId: 'fid-1', filename: 'data.csv' }],
|
||
succeeded: 1,
|
||
failed: 0,
|
||
},
|
||
});
|
||
|
||
await batchUploadCodeEnvFiles({
|
||
req,
|
||
files: [{ stream: Readable.from(['file-content']), filename: 'data.csv' }],
|
||
kind: 'user',
|
||
id: 'user-123',
|
||
codeApiBaseUrl: 'https://stateful-code.example.com',
|
||
executionProfile: 'stateful',
|
||
bridgeWorkerId: 'personal-worker-1',
|
||
});
|
||
|
||
const [url, , callConfig] = mockAxios.post.mock.calls[0];
|
||
expect(url).toBe('https://stateful-code.example.com/upload/batch');
|
||
expect(getCodeApiAuthHeaders).toHaveBeenCalledWith(req, 'personal-worker-1');
|
||
expect(callConfig.headers['X-CodeAPI-Expected-Profile']).toBe('stateful');
|
||
expect(callConfig.headers['X-LibreChat-Code-Worker-ID']).toBe('personal-worker-1');
|
||
});
|
||
});
|
||
});
|