Commit graph

81 commits

Author SHA1 Message Date
lia-by-librechat[bot]
c102e2d498
🦪 fix: Preserve Generated File Byte Snapshots (#18009)
* fix: Preserve Generated File Byte Snapshots

* fix: Select Snapshot Generations Before Resource Partitioning

* fix: Select Snapshot Destinations Before File Admission

* fix: Separate Snapshot Capture Identity and File Consumers

* fix: Separate Owned References and Literal Snapshot Mounts

* fix: Clean Up Unfinished Snapshots and Normalize Upload Arbitration

* fix: Keep Shared Snapshot Mounts Ahead of Private Receipts

* test: Keep Snapshot Priority Regressions Type-Safe

* fix: Preserve Literal Snapshot Paths During Legacy Recovery

* fix: Reconcile Legacy Receipt Paths Before Recovery

* fix: Reuse Fresh Unnamed Legacy Code References

* fix: Expire Snapshot Reservations Before Provisioning

---------

Co-authored-by: Lia <lia@librechat.ai>
Co-authored-by: lia-by-librechat[bot] <328778573+lia-by-librechat[bot]@users.noreply.github.com>
2026-11-10 13:45:15 -05:00
Marco Beretta
654ce897e9
🗄️ fix: Skip Attachments Deleted by Storage Retention Instead of Failing the Turn (#17910)
* fix: Skip Attachments Deleted by Storage Retention Instead of Failing the Turn

A file that storage retention removed from an earlier message made every
later turn of the conversation fail, because the encoders rethrew the
storage miss and the user has no way to remove an attachment from history.

History replay now passes a handler to the encoders, which leave missing
files out of the request and report them; the message's file context tells
the model which files were deleted and were not read. Current-turn
attachments keep the strict error, since the user can remove and re-upload
them. Code Interpreter priming names files missing from storage in the
tool context and only requires resource recovery for other failures.

* refactor: Keep Code Reupload Failure Policy in packages/api

Tally priming reupload failures in createCodeReuploadFailures so the
decision that a file deleted from storage is named instead of requiring
resource recovery lives in TypeScript, and add mock e2e scenarios for a
follow-up after an attachment is deleted from storage.

* fix: Classify Code Files as Deleted Only at the Storage Read

A 404 from the Code API upload was counted as a source file deleted from
storage, which skipped resource recovery. openCodeFileSource marks a miss
where the stored object is read, and only that error counts as deleted.

Move the history replay of attachments with tolerated misses into
replayMessageAttachments so BaseClient only calls it.

* fix: Name Local Code Files Whose Stream Reports a Missing Object

Local storage returns a read stream that raises ENOENT only once read, so
openCodeFileSource now holds a stream until its first chunk or error and
names a miss there. The missing-files context assignment moves into
setMissingCodeFilesContext so both tool loaders only call it.

* fix: Close a Held Code File Stream When Priming Is Cancelled

openCodeFileSource waits for a source stream's first chunk; when that
wait fails, as on cancellation, the caller never receives the stream, so
destroy it before rethrowing instead of leaking its file descriptor.
2026-11-08 23:38:47 +01:00
Danny Avila
5e90cc45e4
📎 feat: Copy Chat Attachments into Attached Workspaces (#17376)
* 📎 feat: Copy Chat Attachments into Attached Workspaces

A chat on an attached workspace now copies the user's own attachments into
.librechat/attachments/ before the model runs, through the worker's
import_file operation, and tells the model where they are. A worker without
the operation, a deny file-write policy, or a copy that fails or runs past
its bound leaves a plain note saying the file is not on this machine and why.

* 🧭 fix: Scope Attachment Records by Tenant and Prime Late Subagents

Match the tenant when recording a workspace copy, require write_file so the
folder can always ignore itself, key copies by the live Code API route and
worker, keep a digest when a copy fails so it is not hashed again, stop
waiting on bookkeeping at the deadline, refuse to start on a cancelled turn,
prime lazily initialized subagents through one per-turn primer, and make the
budget configurable as endpoints.agents.workspaceAttachments.timeoutMs.

* 🙈 fix: Copy Attachments Only Where the Folder Ignores Itself

Gate every copy on .librechat/attachments/.gitignore ignoring the whole
folder: create it with *, and accept an existing one only when it already
ignores everything, otherwise report the file as unavailable with how to fix
it. Prime a lazily routed subagent against its own tool context with a
provisioning callback bound to it, and check for a cancelled turn before
arming the deadline timer.

* 🪢 fix: Recheck the Attachment Ignore Rule Each Turn and Share One Request Cap

Drop the process-wide ignore cache and check the folder once per turn, shared
by every agent, rejecting a truncated read. Cap workspace requests per turn
across agents (workspaceAttachments.concurrency, default 3), name converted
images for their stored type, log only safe error metadata, and keep the
bookkeeping write from advancing updatedAt.

* 🔀 chore: Take dev's File Methods Ahead of Merging dev

dev and this branch both add to the same lines of the file methods module.
Take dev's version here so the merge of dev that follows is clean; the next
commit adds recordFileWorkspaceImport back beside dev's new methods.

* 🧩 fix: Ignore Workspace Operations a Newer Worker Advertises

Bring recordFileWorkspaceImport back beside dev's new file methods after the
merge. Drop well-formed operation names this build does not know from a
worker's capabilities, globally and per workspace, instead of rejecting the
whole status, so workers can turn on new operations such as import_file by
default. Known names keep their strict checks, malformed lists are still
refused, and a worker offering only unknown operations offers no workspace
tools rather than failing.
2026-10-23 07:37:50 +02:00
lia-by-librechat[bot]
a51c45bbf3
🌳 fix: Preserve Primary Files Until Secondary Cleanup Succeeds (#17228)
* fix: Preserve primary files until secondary cleanup succeeds

* test: Use a flat deletion-strategy lookup

* fix: Preserve erasure retries and reconcile removed embeddings

* test: Match deletion regression formatting

* fix: Preserve search intent after vector cleanup

* fix: Repair retained project search files before tool execution

* fix: Normalize Retained Project File Runtime Types

* test: Use the Storage Source Enum in Project Repair Coverage

* fix: Confirm Secondary Cleanup and Preserve Repair Policy

* test: Correct Repair Fixture Types and Cleanup Retry Assertions

* fix: Reconcile Code Copies Compatibly and Charge Shared File Budgets

* fix: Reserve Admitted File Budgets Before Deferred Screening

* fix: Retain Concurrent File Copies and Scope Cleanup by Tenant

* fix: Bound Erasure File Checks and Correct Cleanup Fixtures

* fix: Preserve Provider-Only Assistant File Deletion

* fix: Reconcile Primary Vector Deletions Before Finalization

* fix: Finalize Successful Files in Mixed Cleanup Batches

* test: Load Real File Finalizer in Cleanup Integration Tests

* fix: Preserve Project Routing MIME and Fence File Storage Operations

* fix: Validate Provisioning Principal and Type Project Repair Fixtures

* fix: Recover Provisioning Claims and Retain Uncertain Storage Outcomes

* fix: Normalize File Recovery Results and Reconcile Transport Attempts

* test: Preserve Deployment Routes in Host Provisioning Fixtures

* fix: Recover Deletion Claims and Fence Generated Output Writes

* fix: Preserve Generated Output Commit Snapshots

* test: Use Persisted Owner Type in Generated File Fixture

* test: Type Generated Output Fixture with Its Commit Contract

* fix: Preserve Publication Error Handling and Cleanup Test Boundaries

---------

Co-authored-by: Lia <lia@librechat.ai>
2026-10-23 07:00:20 +02:00
Tómas Pálsson
5e4ecbc24c
🧹 fix: Let File Deletes Succeed When the Code API No Longer Serves the Ref's Profile (#16671)
deleteCodeEnvFile rethrew every error except 404. A deployment that only serves
the stateful profile answers a default-profile delete with 409
execution_profile_mismatch on every retry, so the file landed in failedFileIds,
its record and agent references stayed, and agents holding it aborted every code
turn with "Failed to provision N file(s) to the code environment". The mismatch
is now handled like an absent object: logged, and the delete proceeds. Other
409s, 5xx, auth and network errors still fail the delete.
2026-10-04 11:38:31 -04:00
lia-by-librechat[bot]
5674faaa01
🪀 fix: Separate Workspace Admission and Transport Budgets (#16728)
* fix: Separate workspace admission and transport budgets

* fix: Validate workspace run reserves and retry test types

---------

Co-authored-by: Lia <lia@librechat.ai>
2026-10-03 21:22:30 -04:00
lia-by-librechat[bot]
59b772f132
🥽 feat: Configure Attached Defaults and Honor Skill Read Ranges (#16540)
* feat: configure attached command and read defaults

* fix: separate programmatic timeout defaults from ceilings

* fix: retain attached programmatic schema instructions

* fix: honor explicit skill and sandbox read ranges

* fix: remove duplicate command imports after rebase

* fix: retrieve complete sandbox text before paging

---------

Co-authored-by: Lia <lia@librechat.ai>
2026-10-01 19:57:08 -04:00
Danny Avila
3e19b1876e
🎯 feat: Tolerant Workspace Edits with Host-Rendered Conflict Diagnostics (#16520)
* 🎯 feat: Opt Into Tolerant Workspace Edits and Pass Worker Diagnostics Through

* 🛡️ fix: Gate Tolerant Edits Behind Config, Render Worker Conflicts From Parsed Facts, Bound replace_all

* ⚖️ fix: Default Workspace Edits to Tolerant Matching and Stream Exact replace_all

* 🧱 fix: Build Exact replace_all Output in Bounded Chunks and Describe Both Matching Modes

* 🧹 fix: Keep Worker Conflict Bodies Out of Logs and Edit Matching Out of CJS
2026-09-29 16:55:32 -04:00
Danny Avila
867fd5eac5
🛤️ feat: Route Linked Worktree Requests into Per-Worktree Lanes (#16477)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
* feat: Route Linked Worktree Requests into Per-Worktree Lanes

Workers that advertise the git_linked_worktree workspace scope run each
.worktrees/<name> of a registered checkout in its own scheduling lane.
Accept the scope in worker status, map worktree-scoped paths and command
working directories onto the worktree field, and restore the prefix on
returned paths so tool output stays relative to the checkout.

* fix: Advertise Worktree cwd Routing in Model-Facing Bash Definitions

* refactor: Normalize the Lane Flag in TypeScript, Pass It Through /api Unchanged

* feat: Gate Linked Worktree Lanes Behind an Environment configSchema Toggle

* test: Give the Lane Toggle Fixture a Default Environment
2026-09-29 08:13:05 -04:00
Danny Avila
f624ad0876
🔁 fix: Retry Code API Rate Limits That Reject Before a Workspace Call Starts (#16453)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
* 🔁 fix: Retry Code API Rate Limits That Reject Before a Workspace Call Starts

* 🔁 fix: Bound Workspace Rate Limit Retries by Code API Budget

* 🔗 fix: Isolate Code API Retry Default from Utility Graph

* 🧩 fix: Type the Shared Code API Retry Default

* 🔁 fix: Keep Workspace Admission Retry Budgets Independent

---------

Co-authored-by: Lia <lia@librechat.ai>
2026-09-28 15:45:13 -04:00
lia-by-librechat[bot]
bf91d2b59c
🥧 fix: Reserve Execution Time in Workspace Admission (#16421)
* 🥧 fix: Reserve Execution Time in Workspace Admission

* 🥧 fix: Pass Workspace Admission Static Checks

---------

Co-authored-by: Lia <lia@librechat.ai>
2026-09-27 20:19:54 -04:00
Danny Avila
f15c8a0f1b
🗂️ fix: Stop Telling Attached Workspaces That Uploads Are at /mnt/data (#16389) 2026-09-26 22:04:05 -04:00
Danny Avila
2df937dd06
🌳 feat: Isolate Attached Workspaces by Conversation (#16107)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
* feat: bind attached workspaces to conversations

* ⬆️ chore: bump agents to 3.8.8
2026-09-19 23:19:26 -04:00
Danny Avila
c0affede00
🎟️ fix: Preserve BYOM Tool Calls Through Capacity Windows (#16060)
* ⏳ fix: Keep BYOM tools queued across capacity windows

* ⏳ fix: Configure the admission budget and refresh queued credentials

Addresses the Codex review on #16060:

- Add `limits.maxQueueWaitMs` to the code environment config schema so a
  deployment can shorten or disable the five-minute admission budget from
  `librechat.yaml`; omission keeps the merged behavior, `0` surfaces the
  first capacity expiry to the model. Resolved through
  `resolveAttachedWorkspaceQueueWaitMs` and passed by ToolService, matching
  how `maxCommandTimeoutMs` already flows.
- Accept an auth-header supplier in `executeWorkspaceTool` and mint it per
  admission attempt, so a call queued past the Code API token TTL presents
  fresh credentials instead of failing permanently with 401. The attached
  Bash tool and the workspace file operations now pass suppliers.
- Recheck the deadline after a clamped wait and throw the saved rejection
  rather than opening an admission window that could still be admitted
  once the budget is spent.

* fix: Complete workspace admission wiring and caller regressions

* fix: Share protected edit retry budgets and schema limits

---------

Co-authored-by: Lia <lia@librechat.ai>
2026-09-19 09:03:06 -04:00
Joseph Licata
f25242df98
🦘 feat: Share Run-Scoped Files With Subagents (#15848)
* feat: add run-scoped file sharing for subagents

Delegate current-message files through authorized execution catalogs and publish selected immutable code-output versions through the existing file pipeline.

Requires the companion agents SDK context adapter before the dependency can be released and pinned.

* fix: preserve child tool artifacts and verify file-sharing lifecycle

Keep citations and other non-sandbox artifacts on their existing delivery path while retaining private code outputs and inspected images until publication.

Cover native and text delivery, nested and sibling authorization, concurrent runs, cancellation cleanup, and checkpoint recovery through the browser harness.

* test: align file-sharing fixtures and integration cleanup

* test: repair LibreOffice conversion fixtures

* test: verify deployment skill bytes across checkout formats

* fix: align subagent file budgets and refresh boundaries

* chore: update agents SDK to 3.8.6

* refactor: move code output processing into API workspace

* fix: enforce shared file budgets for parents and teams

* fix: declare the shared file encoder interface
2026-09-12 20:14:22 -04:00
JOJO
e49cb798da
🪢 fix: Keep One Code File per Sandbox Destination (#15817)
* 🪢 fix: Keep One Code File per Sandbox Destination

Codeapi mounts a by-ref input at the filename recorded on the stored
object (the file server's Content-Disposition) and only falls back to the
`name` sent in `/exec` when that header is absent. The `<stem>-<hash>`
alias `claimCodeDestination` handed to an older same-name file was
therefore remapped onto the bare name inside the sandbox, and the whole
request was rejected with "Conflicting input destinations" — on every
later turn, because nothing came back to collapse the pair. Any
conversation in which code rewrote an uploaded file in place, or the user
re-uploaded a file under its original name, lost code execution for good.

`primeFiles` now reserves each destination with `reserveCodeDestination`
in newest-content-first order and skips the files that lose, before the
freshness probe so a dropped file is never re-uploaded. This is the same
collapse `seedCodeFilesIntoSessions` and `ToolNode.updateCodeSession`
already apply. The spec stub mirrors the real directory-prefix rule so the
ancestor case is covered at this layer too.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: preserve stored sandbox destinations during priming

* test: remove unused mock transport imports

* fix: recover unnamed legacy inputs and converted images safely

* fix: keep freshness decisions paired with mount destinations

* fix: align recovery destinations with upload and route boundaries

* style: format upload receipt regression fixture

* fix: distinguish stored paths from proposed recovery paths

* fix: preserve lazy provisioning destination priority and recovery names

* fix: reconcile recovery names with live code destinations

* fix: distinguish recoverable destinations from superseded scoped content

* fix: guard optional request user in recovery scope check

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Danny Avila <danny@librechat.ai>
2026-09-10 15:13:38 -04:00
Danny Avila
65bb64bc68
🛟 fix: Recover CodeAPI File Uploads After Throttling (#15784)
* fix: recover CodeAPI uploads after throttling

* fix: cover eager CodeAPI uploads

* fix: scope CodeAPI upload recovery

* test: clarify upload recovery deadlines

* fix: isolate CodeAPI recovery state

* fix: cancel CodeAPI recovery waits

* test: provide CodeAPI registry in handler fixture

* fix: propagate CodeAPI recovery controls

* fix: bound CodeAPI recovery by run

* fix: preserve skill upload cancellation

* test: update deferred loader signal contract

* fix: preserve recovery cancellation

* fix: cancel recovery source reads

* fix: classify Axios upload cancellation

* fix: preserve lazy provisioning cancellation

* fix: close resource recovery boundaries

* fix: preserve provisioning failures compatibly
2026-09-09 15:48:11 -04:00
Danny Avila
ec15075dc8
🚚 feat: Route Attachments and Provision Tools Lazily (#15763) 2026-09-08 23:28:52 -04:00
Danny Avila
ab6cde0c79
⏳ fix: Separate Temporary and General Chat Retention (#15734)
* fix: separate temporary and general chat retention

* fix: inherit retention when branching messages

* fix: harden retention lifecycle boundaries

* fix: preserve subagent retention context

* fix: trust stored chat retention for uploads

* perf: overlap generated file retention lookup

* fix: derive retention from trusted chat state

* fix: preserve retention across the full turn lifecycle

* test: accept the conversation retention read adapter

* fix: carry retention deadlines through durable job saves

* fix: resolve retention at cached and new chat admission

* fix: resolve direct tool retention before execution

* fix: preserve retention across assistant writes and direct tools
2026-09-08 11:22:54 -04:00
Danny Avila
58de77619e
🗝️ refactor: Prefer the Recorded Storage Key for Every Download Stream (#15695)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
S3 and CloudFront records have carried `storageKey` since #12987, yet six
readers still handed `file.filepath` to `getDownloadStream` and re-derived
the key by parsing a presigned or CDN URL, while four others had grown
their own `storageKey || filepath` expression. One resolver now serves all
ten: `resolveDownloadPath` returns the recorded key when present and the
path otherwise, so records without a key (local, Firebase, Azure, code
output) behave exactly as before, and the share route keeps its local-only
query-string strip on top.

`resolveStoredS3Key` reuses the same `StoredFileRef` type. Covered by unit
cases for the resolver and an S3 case proving a record with a key streams
correctly even when its stored URL no longer parses.

Closes #15693
2026-09-07 01:28:35 -04:00
Danny Avila
ebf70826b5
📫 fix: Surface Sandbox Artifact Delivery Failures (#15671)
* fix: Surface sandbox artifact delivery failures

* chore: Sort artifact delivery import
2026-09-06 15:31:19 -04:00
Danny Avila
fe30e3db32
🪫 fix: Back Off Failed Retention Deletes (#15608)
* 🧹 fix: Stop Undeletable Files Starving the Retention Sweep

`getExpiredFiles` returns the oldest `expiredAt` first, capped at `limit`,
and `processDeleteRequest` leaves the record in place when storage deletion
fails. Nothing records the failure, so the same files come back at the head
of the next batch an hour later, forever: no backoff, no cap, and — once
`limit` of them cannot be deleted — no file that expires afterwards is ever
swept again. On the deployment behind #15511 that is ~29k stranded objects
permanently occupying a 100-slot queue, which is why fixing the Code
Interpreter side alone (LibreChat-AI/code-interpreter#85) would not have
resumed deletion there.

Failures are now recorded on the file. `deletionRetryAt` holds it back with
a backoff doubling from one sweep interval to a day, and `deletionAttempts`
retires it from the query entirely once it reaches
`FILE_RETENTION_SWEEP_MAX_ATTEMPTS` (10, so roughly five days of retries).
Both fields are absent on existing records and absence means "never
attempted", so nothing already in the collection changes eligibility.

The record itself is kept rather than deleted — the reference is what an
operator needs to reconcile a bucket the sweep could not clear, and dropping
it would restore the silence that made this leak invisible. Raising
`FILE_RETENTION_SWEEP_MAX_ATTEMPTS` re-admits everything previously given up
on, which is the supported way to resume once the storage-side failure is
fixed; the give-up is logged with that instruction.

The counter is incremented server-side with `$inc` so concurrent sweeps on
separate nodes cannot overwrite each other's progress toward the cap, and a
failure to record a failure is logged and skipped rather than aborting the
rest of the batch.

* 🧭 fix: Delete Code Environment Files Through the Route That Exists

`deleteCodeEnvFile` tried `/sessions/:sid/objects/:fid` before falling back
to `/files/:sid/:fid`. Both have been there since #13424, but only the
second is mounted by any released codeapi — the first gained DELETE in
LibreChat-AI/code-interpreter#85 — so every deletion paid a guaranteed 404
and a wasted round trip, and #15511 read that 404 as the whole bug.

Call `/files/:sid/:fid` directly. It is the safe direction to collapse
toward: codeapi has mounted it since its first release, so this works
against older deployments as well as post-#85 ones, whereas keeping the
other path would not.

Collapsing the loop tightens two behaviours that only existed to serve it.
A 405 now surfaces instead of being swallowed on the way to a second
attempt; there is no second route to try, and a service that refuses the
method should say so. A 404 is still treated as "already gone" — that is
the only thing it can now mean — but it is logged rather than passed over
in silence, because a 404 caused by a misconfigured base URL looks
identical and this branch drops the file's metadata record either way.

* 🔒 fix: Settle sweep retry state from the write, not the read

Two findings from the review of 0e52924.

- `recordFailure` derived the attempt number by adding one to the count the
  batch had queried. Two nodes sweeping the same file read the same value,
  so both believed themselves to be the same attempt: each `$inc` landed,
  the stored count crossed `FILE_RETENTION_SWEEP_MAX_ATTEMPTS`, and neither
  caller ever saw the threshold. The file drops out of the query — the
  starvation guard still holds — but the give-up is never reported, and
  that log line is the operator's only notice, and carries the instruction
  for resuming. Return the count from the increment itself so every caller
  gets a distinct attempt number and exactly one observes the cap.

  The same staleness shortened the backoff, so `deferExpiredFile` now
  writes with `$max`: a deferral can only move later, and a node that
  computed a shorter delay cannot pull the file forward past one another
  node already committed.

- The backoff doubled from a hard-coded hour while claiming to start from
  one sweep interval. At the default they coincide; away from it the
  schedule stops meaning anything — on a six-hour sweep the first three
  attempts all land on consecutive passes, and on a five-minute one the
  first retry skips twelve. Derive the base from
  `FILE_RETENTION_SWEEP_INTERVAL_MS`, floored at a minute so a
  pathologically short interval cannot spend the whole give-up budget on a
  transient outage.

* 🧯 fix: Keep the give-up notice and the retry budget honest

Four findings from the review of ee1bac5.

- The give-up was reported after the deferral write, inside the same catch.
  Once the increment lands the counter is durable, so `getExpiredFiles`
  already excludes the file; a deferral that then failed left only the
  generic recording error and dropped the one line naming the file and
  saying how to resume. Report it as soon as the increment returns, and let
  the deferral fail on its own.

- Retry deadlines were measured from `Date.now()` after the deletion I/O,
  but `startExpiredFileSweep` arms its interval before the sweep runs. A
  one-interval delay therefore expired just *after* the next scheduled
  pass, which skipped the file and pushed its first retry out by a whole
  extra interval — and the same drift applied to every delay that is an
  exact multiple. Anchor deadlines to the sweep's start instead.

- The threshold used `>=`, so once two nodes pushed a file past the cap
  every one of them past it logged the give-up: one error per replica per
  exhausted file rather than one actionable notice. Only the attempt that
  lands exactly on the cap reports it.

- Retry state outlived the content it described. `processCodeOutput` reuses
  a record for a repeated `(filename, conversationId)` — new bytes, new
  storage key, and a fresh `expiredAt` from `getRetentionExpiry` — while
  `createFile` and `updateFile` set only supplied fields. A record carried
  to the cap by its previous content stayed excluded from the sweep
  forever, stranding the new object exactly as this PR set out to prevent;
  a partially failed one started the new object's budget already spent.
  Both write paths now clear the fields: the budget belongs to the storage
  a record currently points at.

The retry-state fixtures in `file.spec.ts` were seeded through `createFile`,
which now clears them, so they drive the real methods the sweep uses.

* 🎯 fix: Clear the retry budget only when a retention lifecycle starts

Both findings from the review of 8d26f9b, and both are consequences of that
commit's reset rather than of the original change.

- The reset was unconditional on every `createFile`/`updateFile`, on the
  reasoning that those paths write content. Two of them do not.
  `prepareImages{Local,Azure,Firebase}` call `updateFile({ file_id })` with
  nothing but the id — a TTL touch — every time an existing image is
  encoded for another chat, and the deferred preview uses the same method
  to transition `status`. Either handed a stranded record a fresh set of
  attempts and another give-up notice, on repeat, defeating the cap for the
  files most likely to be stranded.

  Gate it on the write actually setting `expiredAt`. The budget belongs to
  a retention lifecycle: that is the write which starts a new one, it is
  what `processCodeOutput` supplies when it repurposes a record, and a
  record with no retention deadline is never swept, so the fields are inert
  there anyway.

- Both retry writes bumped `updatedAt`. `processCodeOutput` falls back to
  `updatedAt` as the writer-order stamp for records that predate
  `metadata.sourceDispatchedAt`, so a failed sweep landing mid-harvest read
  as a newer content writer and the harvest dropped its attachment. Mark
  them `timestamps: false`, for the reason `claimCodeFile` already does:
  bookkeeping is not a content write.

* 🅿️ refactor: Park exhausted files instead of excluding them

Three review rounds in a row found defects in how the give-up cap interacts
with record reuse, each in the fix for the last. That is a design error, not
a bug list: a permanent exclusion has to be bound precisely to the content
lifecycle it was recorded against, and File records outlive their content.
`processCodeOutput` repurposes a row for a repeated
`(filename, conversationId)`, `createFile`/`updateFile` set only supplied
fields, and `getRetentionExpiry` returns `{}` on a lookup failure so the row
inherits its old deadline — three separate ways for bookkeeping to survive
into a lifecycle it does not describe, each needing its own guard, and the
two retry writes needing to be lifecycle-conditional on top.

Remove the category instead. `deletionRetryAt` becomes the sweep's only
hold, and reaching `FILE_RETENTION_SWEEP_MAX_ATTEMPTS` parks the file for a
month rather than excluding it. The bound on the batch is the same — a
stranded file costs one slot a month instead of one an hour — but a deadline
that outlives its content can only delay the next object, never lose it, so
nothing outside the sweep has to reason about this state at all.

That deletes more than it adds:

- `createFile` and `updateFile` go back to their original form. No reset, so
  no question of which writes install content, and `prepareImages*` and the
  deferred preview stop mattering here.
- `getExpiredFiles` loses `maxAttempts` and its `$and`; eligibility is one
  `$or` on the deadline.
- The interleaving race between the increment and the deferral degrades from
  a stranded object to a delayed one.

Working through a large backlog is throughput-bound either way: every
attempt costs a slot in the bounded batch, so N stranded files need N ×
`FILE_RETENTION_SWEEP_MAX_ATTEMPTS` passes to settle. Lower that limit when
recovering a deployment that has accumulated many.
2026-09-04 15:38:17 -04:00
Danny Avila
7d9db14ba5
📓 feat: Author Files in Attached Workspaces (#15548)
* feat(code): route file mutations to attached workspaces

* fix(code): Inspect attached edits before commit

* fix(api): Annotate workspace limits

* fix(code): Normalize workspace edit previews

* style(api): Sort workspace imports

* fix(api): Preserve Workspace List Validation
2026-09-04 12:16:23 -04:00
Danny Avila
e23e13e712
🌳 feat: List Attached Workspace Trees (#15527)
* feat(code): list attached workspace files

* test(code): include workspace list history

* fix(code): validate workspace tool scope consistently

* fix(code): Harden workspace listings

* fix(code): Align workspace list validation

* fix(code): Continue complete workspace listings

* fix(code): preserve canonical path protections

* test(code): keep workspace list specs type-safe
2026-09-04 10:52:34 -04:00
Danny Avila
9a64e85b1e
🕯️ feat: Search Text in Attached Workspaces (#15525)
* feat(code): search attached workspaces

* fix(code): remove duplicate workspace type import

* fix(code): abort attached workspace searches

* docs(code): correct workspace abort parameters

* test(code): include workspace search history

* fix(code): keep workspace search foregrounded

* fix(code): preserve attached search routing

* fix(code): Bound workspace search output

* style(code): Format workspace search output
2026-09-04 10:07:06 -04:00
Danny Avila
b7fd4ba2a2
👓 feat: Read Attached Workspace Files (#15524)
* feat(code): read attached workspace files

* style(api): sort workspace handler import

* fix(code): abort attached workspace reads

* test(code): type workspace fetch doubles

* style(code): sort workspace bridge imports

* fix(code): Preserve workspace read boundaries

* style(code): Flatten workspace line validation

* fix(code): Bound attached workspace responses

* fix(code): Require canonical workspace paths

* fix(code): fence workspace bridge operations
2026-09-04 09:58:46 -04:00
Danny Avila
30cd1c2f2e
🔇 fix: Stop logging an absent create_file target as an error (#15591)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
`create_file` reads its target before writing so it can tell a create from
an overwrite, so `readSandboxFile` missing the path is the ordinary case for
a create — yet it was reported at error level, through `logAxiosError`, which
rendered the sandbox's own `cat: …: No such file or directory` as "An error
occurred while setting up the request" plus a stack. Every ordinary file
creation emitted an error-level entry claiming a transport fault about a file
that was written half a second later, and an operator reading it concluded the
file had been lost.

Split the two failures `readSandboxFile` was conflating. A request that never
completed still goes to `logAxiosError`; a request the sandbox answered with
stderr is now classified on that stderr — an absent path logs at debug, and
anything else (permission, a broken interpreter) keeps error level with the
sandbox's own wording. The thrown error is unchanged, so the authoring flow's
create/overwrite decision still reads it the same way.

The classifier moves to `packages/api/src/files/code/errors.ts` and is now
shared: `handlers.ts` had two near-duplicate copies of it, the narrower of
which documented itself as narrower than the broader one without either
deriving from the other.

Also carry the cause on `[preflightCodeOutputBatch] … could not be inspected`.
That warning follows each `create_file` in the same request, and the batch
discarded the error entirely — so a Code API that refused the download, one
routed to the wrong execution profile, and a file too large to inspect all
arrived as the same sentence, and neither this warning nor the one above could
be told apart from the failures around them. It goes through
`getSafeErrorMetadata`, which is what keeps the upstream message and the
artifact's name — both of which can echo submitted content — out of the log.
2026-09-04 09:21:24 -04:00
Danny Avila
51edb2a660
🛫 feat: Enable Self-Service Code Worker Pairing (#15367)
* feat: add self-service code worker pairing

* fix: enforce code worker principal boundaries

* fix: separate pairing control planes from execution

* fix: preserve principal worker routing

* fix: pin artifact traffic to code workers

* fix: authenticate code worker selection

* test: mock worker-aware artifact headers

* fix: harden self-service code worker routing

* fix: fail closed for principal code pairing

* style: format code execution headers

* fix: close self-service pairing gaps

* fix: satisfy code environment type checks
2026-09-01 18:46:33 -04:00
Danny Avila
c8939c9730
🪄 fix: Reuse Downloaded Artifacts for Image Reads (#15454) 2026-09-01 10:55:53 -04:00
Danny Avila
8f6228095f
🖼️ fix: Stop Reporting a Failed Sandbox Image Read as an Unreadable Format (#15453)
* 🖼️ fix: Stop Reporting a Failed Sandbox Image Read as an Unreadable Format

`read_file` on a `/mnt/data` image collapsed every failure — rate limit,
missing path, truncated transfer, transport error — into the binary-file
hint, whose image branch reads as a permanent capability limit ("is an
image file and cannot be read as text"). The model stopped retrying, and
the operator had no way to tell which failure had actually occurred.

The failure that surfaced it: the reader windows the bytes through
`/exec` stdout, one call per window, against the Code API's per-user
execution limiter (20 requests / 30s in the reference deployment). At a
fixed 32KB window a 122KB chart cost 4 calls and a 1MB image cost 33 —
more than a whole limiter window, so anything over ~640KB could never
complete. A turn that generates several charts and reads them back
exhausts the budget partway through and reports the 429s as unreadable
images.

- `handlers.ts`: failed reads name the real cause and the affordance that
  matches it — rate-limited waits and retries, a missing path lists the
  directory, a truncated transfer retries. The "cannot be read as text"
  phrasing is now reserved for bytes that are not a supported image. An
  image past the inline budget says to downscale and read the smaller
  copy rather than pointing at `bash_tool`, which cannot show an image.
- `process.js`: the window is derived from the runner's stdout budget
  (48,960B vs 32,768B, so a 122KB chart is 3 calls and 1MB is 22), halves
  and re-reads the same offset when a runner truncates anyway (remembered
  per Code API, with the env remedy logged), waits out a rate limit it
  can afford instead of discarding the bytes already pulled, and stops at
  a 24-call ceiling so no read stalls a turn across limiter windows.
- `utils/code.ts`: shared `getCodeApiRetryAfterMs` (header, then the
  `retry_after_seconds` body); `skillFiles.ts` drops its private copy.

* ♻️ refactor: Move the Sandbox Image Reader Into TypeScript

Addresses the review round on #15453.

- P1: windowing, window sizing, the in-sandbox reader script, response
  parsing and byte assembly move to `packages/api/src/files/code/image.ts`,
  and the Code API rate-limit policy to `packages/api/src/utils/code.ts`.
  `process.js` keeps only the `/exec` transport it owns. Both new modules
  are tested against their real implementations (19 + 13 cases), including
  the round-trip ceiling, the changed-file guard and the shell-injection
  guard on the reader script.
- P2: a configured stdout budget below the old 8KB window floor produced a
  window the runner could never emit, and narrowing was pinned to the same
  floor, so no retry could succeed. The window now aligns to the base64
  group size instead, and the round-trip ceiling bounds what a small budget
  costs.
- P2: `.env.example` documented a 32,768-byte default that no longer holds
  and never mentioned the stdout budget. It now describes
  `LIBRECHAT_CODE_SANDBOX_OUTPUT_MAX_SIZE` as the setting to tune, with the
  chunk override as the escape hatch.
- P3: `process.spec.js` no longer reimplements the retry parser in its
  module mock. Behaviour is tested where it lives; the mock is a declared
  passthrough and the spec covers what the transport still owns — session
  forwarding, the profile header, parser hand-off and error logging.
- P3: `getCodeApiRetryAfterMs` reads the response body through `in`
  narrowing rather than an `as` assertion.
- P3: the window-sizing JSDoc no longer claims a multiple of 3 in the one
  case that is not — an explicit `LIBRECHAT_CODE_IMAGE_CHUNK_BYTES`.

* 🎯 fix: Size Sandbox Image Reads to What the Limiter Can Deliver

Addresses review round 2 on #15453.

- Narrowing now halves the caller's own failing window instead of the
  shared learned value, and the learned value only moves down to the
  narrowest size any read needed. Reads run concurrently (one per tool
  call), so two reads failing at the same size were compounding into a
  window two steps smaller than the runner would have accepted, and
  persisting it for every later read.
- Narrowing continues until a runner accepts a window or a floor below
  which no runner could serve an image at all. Two halvings stopped at
  12,240 bytes — still ~16KB of base64 — so an 8-10KB stdout cap that was
  not configured up front stayed unreachable.
- The round-trip ceiling is now one limiter window (20), and a read that
  cannot finish within it says so from the FIRST response rather than
  spending the whole budget rediscovering it: the first window reveals
  both the file size and what a call really delivers. A near-limit image
  no longer drains the turn's executions before failing, which is what
  made the wait budget look too small.
- The round-trip message names the size this deployment can actually
  inline, so the model has a downscale target instead of a guess.
- `python3: not found` from the sandbox reader was matching the generic
  "not found" in `isSandboxMissingFileError` and being reported as a
  missing image path, sending the model to `ls /mnt/data` while hiding a
  runner dependency. Image reads now classify only path-specific ENOENT
  forms as a missing file.

* 📐 fix: Size a Truncated Read's Retry From the Cap It Revealed

Addresses review round 3 on #15453.

A ~1KB sandbox stdout cap is a real runner default, and the 1,536-byte
narrowing floor sat above it: discovery stopped one step short and
declared every image unreadable, even though small ones fit comfortably
in the 576-byte window the configured path already derives.

Narrowing now runs until no smaller window exists. It also no longer has
to grope its way down: a truncated response emitted exactly as much as
the runner allows, so its length measures the cap this deployment never
declared, and the retry is derived from it directly — one round-trip
instead of the eight blind halvings it takes to fall from ~49KB to ~1KB.
Blind halving remains the fallback and bounds the result either way, so
every retry is strictly smaller than the window that just failed.
2026-09-01 09:08:01 -04:00
Danny Avila
224512851a
🗂️ fix: Resolve Code Interpreter Destination Collisions (#15449)
* 🗂️ fix: Resolve Code Interpreter Destination Collisions

Codeapi mounts each `/exec` input at a destination derived from its `name`
and rejects the whole request when two entries land on one. LibreChat never
enforced that: `primeFiles` sent `name: file.filename` verbatim, and the
graph seed deduped by `(storage_session_id, id)` — identity, not destination.

Two conversation files sharing a filename therefore killed code execution for
the rest of the conversation. The rejected request never reaches the sandbox,
so no result comes back to collapse the pair, and every later turn re-primes
both and fails identically. Only code-generated outputs are covered by the
`(filename, conversationId, context, tenantId)` partial unique index; uploads
carry `context: message_attachment`, so one conversation can hold several
records sharing a filename — two uploads of `image.png`, or an upload whose
name a later execution wrote back in place.

`packages/api/src/files/code/destinations.ts` now owns the name → destination
mapping for every contributor to an `/exec` file list, mirroring codeapi's
conflict rule including directory-prefix conflicts:

- `primeFiles` claims per pushed file, disambiguating with a `-2`, `-3`
  counter and echoing the resolved path into the tool context, so the model
  is told where each file actually mounts. Claim order comes from immutable
  `createdAt`, not `getFiles`'s `updatedAt` sort, which usage accounting and
  re-upload both bump — order-dependent destinations would silently repoint
  a path an earlier turn had already handed the model. Newest wins the bare
  name, matching `ToolNode.updateCodeSession` and giving an in-place rewrite
  precedence over the original it superseded.
- `seedCodeFilesIntoSessions` drops a later file whose destination is already
  claimed. Renaming there would invent a path nothing told the model about,
  and a conflict at that layer means one logical file arrived twice under
  different storage pointers.

Fixes #15443

* 🧹 chore: Tolerate Holes in the Destination Priority Sort

`primeFiles` sorts the raw `getFiles` result before its own per-entry guard
runs, so the comparator has to survive a null row rather than throw ahead of
the loop that was already defending against one.

Also tightens the tool-context assertion in the upload-vs-output case, which
was passing on a trailing space rather than on the path.

* 🔁 fix: Address Codex Review on Destination Assignment

Four findings from the review of `b0f61d0`.

**Identity-derived suffixes instead of a collision counter.** A counter is
assigned from whatever is in the set, so a third file arriving under the
literal name a displaced file had been given pushed that file along, and code
the model wrote in an earlier turn would silently start reading the newcomer.
The fallback now hashes the file's own `file_id`, so a displaced file keeps
its destination for the life of the conversation and only the winner of the
bare name can change. A trailing counter still guards a hash collision so the
search terminates on any input.

**Rank by last content write, not record creation.** `processCodeOutput`
claims one row per `(filename, conversationId)` and rewrites it in place, so
a repeatedly written output keeps its original `createdAt` while recording the
latest writer in `metadata.sourceDispatchedAt`. Ranking on `createdAt` alone
handed the bare path to an upload the output had since been rewritten over.
Recency is now `max(sourceDispatchedAt, createdAt)` — still never `updatedAt`,
which usage accounting bumps without any content changing.

**Advertise a displaced generated output.** Generated files are left out of
the tool context because the model already knows what it wrote — which stops
being true once a newer same-named file takes the bare path and the output
mounts somewhere the model has never seen. Displaced outputs are now listed
with the name they were written as, and displaced uploads name their original
too.

**Rank conversation files above an agent's private ones.** Each agent claims
destinations over its own candidate set — the conversation's files plus its
own — so a private file taking a bare name from a shared file in one agent but
not in another left two agents advertising different paths for the same file
into one shared mount namespace. Shared files now rank first, so the majority
that every agent sees resolves identically. The private-versus-private case
still drops at the merge and is documented there; closing it needs one
assignment across contributors, made before any agent renders its context.

* 📌 test: Pin Alias Ownership on an In-Place Rewrite

Codex round 2 flagged that the identity alias still falls back to a counter
when the alias itself is literally claimed, so a displaced file can move after
all. The mechanism is real and my JSDoc overstated the guarantee, which this
corrects.

The behaviour stays as it is. The realistic way a file arrives under another
file's alias is the model rewriting the displaced file in place: it was told
`<stem>-<hash>.ext`, wrote back to it, and `processCodeOutput` registered an
output under that literal name. Keeping the alias with the newest holder is
what makes the model's familiar path resolve to its own edit; reserving it
against the newcomer would hand that path back to the bytes it replaced. The
test pins that ordering so a later change cannot invert it silently.
2026-09-01 00:55:54 -04:00
Danny Avila
c1cb591d49
🗿 feat: Add Attached Stateful Code Environments (#15352)
* feat: add attached stateful code environments

* fix: include code environment in lazy agent type

* fix: harden stateful environment routing

* fix: complete code environment route isolation

* fix: declare code environment map type

* fix: preserve configured code execution routes

* fix: harden stateful environment updates

* test: assert route-scoped sandbox readiness

* fix: harden stateful environment lifecycle

* fix: isolate migrated code sessions

* test: assert route-qualified code sessions
2026-08-30 15:38:45 -04:00
Danny Avila
67b7b441b2
🛂 feat: Filter Model-Bound Content by Source (#14425)
* feat: introduce optional content protection seam

* feat: enforce source-aware content filters

* feat: complete source-aware content enforcement

* test: activate skill file-text fail-close fixtures

* fix: harden source-aware content filters

* fix: harden model-bound content filtering

* fix: preserve legacy filters and generated files

* fix: inspect shared scalar metadata

* test: align mocks with current dev dependencies

* feat: add persisted content filter safeguards

* feat: complete source-aware content filter enforcement

* fix: move resume content preflight into TypeScript

* fix: close content inspection edge cases

* fix: harden content protection boundaries

* fix: complete content protection safeguards

* test: align persisted memory filter coverage

* fix: reconcile content protection with current dev

* fix: reconcile content protection with latest dev

* fix: close content protection review gaps

* fix: enforce source-aware provider boundaries

* fix: preserve legacy PII preflight semantics

* test: stabilize stored branch preflight fixture

* fix: defer agent writes until protected model admission

* perf: harden source-aware model-bound filtering

* fix: canonicalize provider lineage before validation

* fix: satisfy model-bound callback type checks

* perf: Bound content protection filtering work

* fix: Bound submission array traversal

* fix: Stabilize bounded content snapshots

* fix: Scope model-bound traversal overflows

* fix: Preserve scoped content inspection

* fix: Accumulate aggregate traversal scopes

* fix: centralize content policy boundaries

* test: align deferred tool policy context

* test: align controller policy mocks

* style: normalize content protection imports

* fix: close content policy review gaps

* fix: narrow active skill policy config

* fix: address content protection review boundaries

* fix: retain exact provenance overflow sentinel

* fix: preserve literal and scoped provenance updates

* fix: narrow persisted edit provenance

* fix: isolate exact overflow attribution

* fix: centralize stored prompt protection

* fix: fail closed on incomplete transcript evidence

* fix: align canonical transcript routing

* refactor: centralize content policy preflights

* fix: isolate upload policy error typing

* style: sort policy preflight imports

* refactor: centralize content policy boundaries
2026-08-21 22:43:32 -04:00
Danny Avila
06bf324cf0
🛤️ feat: Per-Agent Code Execution Routing With Stateful Session Scopes (#14848)
* feat: route code execution per agent profile

* chore: sort execution profile imports

* test: preserve stateful environment literal types

* fix: isolate stateful code environments by user

* fix: preserve per-agent code routing end to end

* fix: route code priming by execution profile

* fix: isolate code profile lifecycle state

* fix: preserve mixed-profile code resources

* fix: complete stateful skill routing
2026-08-16 09:42:15 -04:00
Danny Avila
5ff46d8c67
🛟 fix: Stop Agents When Code Resources Cannot Recover (#14651)
* fix: Block Agents When Code Resources Cannot Recover

* fix: Preserve Resource Recovery Failures Across Agent Paths

* fix: Centralize Fatal Agent Initialization

* chore: sort agent imports
2026-08-07 00:33:28 -04:00
Danny Avila
b11978017d
🧱 fix: Enforce Agent Runtime File Trust Boundaries (#14577)
* fix: secure agent runtime file metadata

* chore: sort agent resource test imports

* fix: Align Agent Tool Resource Types

* fix: Rehydrate Agent Image Resources

* fix: preserve remote agent file authorization
2026-08-02 14:18:28 -04:00
Danny Avila
00c5a747e9
🧵 feat: Native Background Execution for Code Interpreter Tools (#14386)
* 🧵 feat: Native Background Execution for Code Interpreter Tools

* 🩹 fix: Address Codex Round 1 (fallback dedupe, harvest failure, handle parsing)

* 🩹 fix: Live Completion Marker + Unkeyed Attachment Dedupe (Codex Round 2)

* 🎨 chore: Sort Imports + Widen Marker Type Comparison (CI)

* 🩹 fix: Stale-Harvest Guard, Error Marker Status, Faster Anchor Retry (Codex Round 3)

* 🧹 refactor: TS Harvest Module, Claim-Neutral Timestamps, Error Parity (Codex Round 4)

* 🩹 fix: Dispatch-Ordered Stale Guard, Foreground Downgrade, Error Wrapper Parity (Codex Round 5)

* 🩹 fix: Retry Past Unfinished Rows + Per-Call Attachment Dedupe (Codex Round 6)

* 🩹 fix: Writer-Dispatch Ordering, Scoped Live Upserts, Reaped-Task Wrapper (Codex Round 7)

* 🩹 fix: Wildcard toolCallId Matching for Bare Attachment Updates (CI)

* 🩹 fix: Claim-Insert Dispatch Stamp (Schema-Backed) + Scoped Status Markers (Codex Round 8)

* 🩹 fix: Pre-Write Ownership CAS + Agent-Scoped Part Patching (Codex Round 9)

* 🩹 fix: Insert-Path Ownership CAS + Agent-Routed Attachments (Codex Round 10)

* 🩹 fix: Agent-Scoped Marker Ids and Attachment Dedupe (Codex Round 11)

* 🩹 fix: Atomic File Commit and Sibling Preview Fan-Out (Codex Round 12)

- Replace the two-step claim-confirm CAS with an atomic conditional updateFile: the ownership predicate (no sourceDispatchedAt, or <= this write's dispatch order) moves into the update filter, removing confirmCodeFileOwnership and the lost-update window between check and write
- Thread agentId through createDownloadFallback so fallback download rows scope to the emitting agent like primary rows
- Fan terminal preview overlays out to every live attachment sharing the file_id in useAttachmentPreviewSync (sibling tool calls no longer stick on pending)
- Restore background artifacts through toStoredArtifact so the size bound applies on re-anchor
- Apply filterAttachmentsForPart to grouped tool-call attachments in ContentParts so handoff agents with colliding provider call ids do not cross-contaminate groups

* 🩹 fix: Agent-Scoped Live Upserts and Monotonic Dispatch Stamps (Codex Round 13)

- Scope the SSE attachment upsert and the useAttachments DB/live merge by agentId with the same wildcard semantics as toolCallId: distinct non-null agentIds stay separate entries, so handoff agents sharing a claimed file_id and a repeated provider tool id (call_0) no longer merge over each other's cards
- Extend the attachment identity key to fileKey::toolCallId::agentId and register less-specific key variants so bare and agent-less live records still dedupe after overlay
- Stamp background task createdAt from a strictly-increasing per-process dispatch counter: raw Date.now() can tie for same-millisecond dispatches and the stale-output guard accepts equal stamps (needed for idempotent re-commits), which would let an older task overwrite a newer task's committed file
2026-07-22 22:13:15 -04:00
Danny Avila
20cd00c492
🖼️ feat: Return Sandbox Images From read_file as Viewable Artifacts (#14277)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
* 🖼️ feat: Return Sandbox Images From `read_file` as Viewable Artifacts

The code-execution sandbox `read_file` path refused every image
extension because it reads files via `cat` over codeapi's JSON `/exec`
transport, which lossily corrupts non-UTF-8 bytes. The skill-file read
path already surfaced images as artifacts; this brings the sandbox path
to parity so an agent can actually see a chart/screenshot it reads.

- `readSandboxImage` (process.js): a Python base64 reader over `/exec`
  with an in-sandbox size guard so oversize images never cross the wire;
  base64 is ASCII-safe where `cat` corrupts.
- `handleSandboxImageRead` (handlers.ts): byte-integrity check (guards
  against a truncated `/exec` stdout), MIME resolved purely from the
  magic-byte sniff (extension only routes; a mislabeled non-image falls
  back to the bash hint), and graceful degradation on every failure mode.
- Shared `buildImageArtifactResult` used by both read paths; the result's
  `artifact.content` image_url reaches the UI (tool-end callbacks save it
  as an attachment) and the LLM (SDK folds it into the model-visible
  message for Anthropic/OpenAI/Google).

* ✅ test: Sync read_file code-only description assertions with image wording

* 🛡️ fix: Harden sandbox image reads (regular-file guard, completeness check)

Addresses Codex review on PR #14277:

- readSandboxImage now os.stat's the target and rejects non-regular files
  (FIFOs, sockets, /dev/* symlinks) via stat.S_ISREG, and bounds the read at
  limit+1 bytes — a device/FIFO can no longer stream unbounded into memory
  until the request times out.
- handleSandboxImageRead validates completeness (not just the magic header):
  PNG must end with the IEND trailer and WebP's RIFF size must match the byte
  length, so a truncated/interrupted image degrades to the bash hint instead
  of being sent as a corrupt image_url. JPEG/GIF stay header-level (they can
  carry trailing metadata; a strict end-marker would risk false rejections).

* 🩹 fix: Chunk sandbox image reads to fit the runner stdout cap

Inlining any real image failed with "is an image file (.png) and cannot
be read as text". Root cause: readSandboxImage base64-encodes the file to
STDOUT, but the runner caps stdout at SANDBOX_OUTPUT_MAX_SIZE (1024 bytes
by default) and SIGKILLs the job on overflow (status OL), truncating the
JSON mid-base64. The parse then threw and the handler degraded to the
binary hint. The in-sandbox MAX_BINARY_BYTES=5MB guard never fired because
the *transport*, not the file size, is the real ceiling: a 5MB image needs
~6.8MB of stdout. Reproduced against a live MicroVM — a 186KB matplotlib
PNG died with 'stdout length exceeded' at exactly the 65536-byte cap.

Read the file in windows instead: each /exec pulls  raw bytes at an
offset and base64s only that slice, so every response stays under the cap
regardless of how the runner is configured; the chunks are reassembled and
verified against the sandbox-reported total. Verified end-to-end on a real
MicroVM: 25KB and 186KB PNGs both round-trip byte-exact (sha256 match).

Also:
- Detect the truncation explicitly (status OL) and name the fixable cause
  (chunk size / SANDBOX_OUTPUT_MAX_SIZE) instead of "unexpected output".
- Parse the LAST stdout line so a shell banner can't break the read, and
  include a stdout snippet when it genuinely is unparseable.
- LIBRECHAT_CODE_IMAGE_CHUNK_BYTES (default 32KB) tunes the window.
- Tests drive the real reader against a mocked /exec transport rather than
  mocking readSandboxImage, which is why the existing suite stayed green
  through this bug.

* 🎯 fix: Cap sandbox inline images at 1MB, separate from skill-file reads

The sandbox and skill-file image paths shared MAX_BINARY_BYTES (5MB), but
their transports differ: skill files stream from storage, while sandbox
bytes come back base64 over /exec stdout under the runner's output cap, so
the reader windows the file and cost scales in round-trips (~160 at 5MB vs
~32 at 1MB). Nothing is gained by allowing more — vision providers
downsample to ~1.5-2k px regardless, so multi-MB originals buy no fidelity
while grinding through round-trips.

Give the sandbox path its own MAX_SANDBOX_INLINE_IMAGE_BYTES (1MB), used
for both the read cap and the over-limit message (which previously quoted
5MB while the reader enforced something else). Skill-file reads keep 5MB.

Verified against a live MicroVM: a 186KB PNG round-trips byte-exact, and a
1.4MB file returns tooLarge in a single round-trip with zero bytes
transferred, degrading to the existing bash_tool hint.
2026-07-16 07:27:33 -04:00
Danny Avila
53e369fba8
🧪 feat: stateful_code_sessions capability for warm Code API sandbox sessions (experimental) (#14150)
* ✨ feat: stateful_code_sessions capability for warm Code API sandbox sessions

Wire the @librechat/agents stateful sandbox sub-config behind a new,
off-by-default stateful_code_sessions agent capability. createRun sets
toolExecution.sandbox.statefulSessions when code execution is active in the
run AND the capability is enabled; execute_code and bash_tool factories get
the param so their descriptions hedge toward persistence. Rides the existing
variable-not-literal runConfig pattern, so it no-ops until @librechat/agents
is bumped to the version shipping the sandbox sub-config.

* ✨ feat: per-agent stateful code sessions (builder toggle + init gating)

Stateful sessions now require the agent's own opt-in, not just the admin
capability. New agent field stateful_code_sessions (schema + validation +
types) surfaces as a toggle in Agent Builder Advanced settings, gated on
the app capability and disabled without Code Interpreter. initializeAgent
resolves the per-agent truth (admin capability AND builder opt-in AND
code env) once: the registered bash_tool description, the execute_code
factory, and createRun's toolExecution.sandbox gate all read the same
resolved value. statefulSessionsAvailable threads through the same call
sites as codeEnvAvailable, including handoff discovery and added convos.

* 🐛 fix: propagate runtime_session_hint to sandbox executor in event-driven tool path

The event-driven ON_TOOL_EXECUTE handler built config.toolCall without the
resolved runtime_session_hint, so BashExecutor/CodeExecutor never sent
runtime_session_hint to the Code API. Every conversation then collapsed onto
the server-derived default session (no per-conversation isolation). Copy
tc.runtimeSessionHint onto toolCallConfig._runtime_session_hint, mirroring the
SDK direct-execution path.

* 🐛 fix: address Codex review findings for stateful code sessions

- OpenAI-compatible service (packages/api/src/agents/openai/service.ts) now
  derives and passes statefulSessionsAvailable alongside codeEnvAvailable, so
  the feature activates on that route (previously statefulCodeSessions resolved
  false there and createRun never sent toolExecution.sandbox).
- Thread runtime_session_hint through the host file-authoring tools
  (create_file/edit_file/read_file): those host branches return before the
  generic tool path, so readSandboxFile/writeSandboxFile now forward the
  per-conversation hint instead of falling back to the Code API default session.
- StatefulSessions builder toggle clears its form value when Code Interpreter is
  disabled, so a saved agent matches the disabled UI and re-enabling code doesn't
  silently reactivate stateful sessions.

* 🐛 fix: normalize stateful_code_sessions on save when Code Interpreter disabled

Addresses Codex review (round 2): a stale `stateful_code_sessions` opt-in
could persist when Code Interpreter (`execute_code`) is disabled from the
main agent builder without opening Advanced settings, silently reactivating
warm sessions if code was later re-enabled.

- AgentPanel: normalize in `composeAgentUpdatePayload` (the always-run save
  path) so `stateful_code_sessions` is forced to `false` whenever
  `execute_code !== true`, regardless of whether Advanced was opened.
- StatefulSessions: revert the mount-scoped useEffect (round-1 approach) —
  it only fired while the Advanced panel was mounted, missing this path.
- Add spec coverage for both branches of the normalization.
2026-07-12 08:12:04 -04:00
Danny Avila
1da789bac0
🗂️ feat: Add Agent File Authoring Tools (#13435)
* feat: add agent file authoring tools

* style: format file authoring changes

* style: satisfy file authoring prettier

* test: fix file authoring initialization expectations

* fix: complete skill file authoring flow

* fix: pass skill authoring state on edit

* test: mock missing bundled skill file

* fix: harden agent file authoring gates

* fix: preserve file authoring runtime context

* test: fix authoring context mock typing

* fix: preserve subagent skill primes

* test: avoid array at in handler spec

* refactor: deepen skill authoring runtime wiring

* fix: address codex authoring review findings

* test: fix authoring collision fixture type

* test: add skill file authoring mock e2e

* fix: Improve skill file authoring recovery

* fix: Show file authoring args while running

* fix: Clarify skill rename authoring errors

* fix: Keep code-only file authoring schemas sandbox scoped

* fix: Address skill authoring review findings

* fix: Gate skill authoring on write access
2026-06-03 23:58:12 -04:00
Danny Avila
e3cc2a9c62
🗄️ refactor: Honor All-Data Retention for Agent Files (#13424)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
* 🛡️ fix: Honor All-Data Retention for Agent Files

* 🧹 fix: Delete Agent Tool Storage on Retention Sweep

* 🧯 fix: Clean Tool Storage After Missing Primary Files

* 🪢 fix: Defer Tool Deletes Until Primary File Resolves

* 🧭 fix: Prefer Session Object Delete for Code Files
2026-05-30 22:32:22 -04:00
Danny Avila
9dd062e42e
🧯 fix: Harden Data Retention Semantics (#13049)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
* feat: support data retention for normal chats

Add retentionMode config variable supporting "all" and "temporary" values.
When "all" is set, data retention applies to all chats, not just temporary ones.
Adds isTemporary field to conversations for proper filtering.

Adapted to new TS method files in packages/data-schemas since upstream
moved models out of api/models/.

Based on danny-avila/LibreChat#10532

Co-Authored-By: WhammyLeaf <233105313+WhammyLeaf@users.noreply.github.com>
(cherry picked from commit 30109e90b0)

* feat: extend data retention to files, tool calls, and shared links

Add expiredAt field and TTL indexes to file, toolCall, and share schemas.
Set expiredAt on tool calls, shared links, and file uploads when
retentionMode is "all" or chat is temporary.

(cherry picked from commit 48973752d3)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: lint/test

(cherry picked from commit 310c514e6a)

* fix: address code review feedback for data retention PR

Critical:
- Fix BookmarkMenu crash: restore optional chaining on conversation
- Fix migration hazard: backward-compatible sidebar filter that also
  checks expiredAt for documents without isTemporary field

Major:
- Add logging to getRetentionExpiry error path, align with tools.js
- Add tests for retentionMode: ALL in saveConvo and saveMessage
- Fix share route: apply expiredAt for temporary chats too by
  querying the conversation's isTemporary flag server-side
- Add assertions for getRetentionExpiry mocks in process tests

Minor:
- Fix ChatRoute isTemporaryChat to be strictly boolean via Boolean()
- Fix stale test description (expired -> temporary)
- Comment out retentionMode default in example yaml
- Simplify verbose if/else to isTemporary === true
- Add compound index on { user: 1, isTemporary: 1 }
- Remove narrating comment from process.spec.js

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
(cherry picked from commit 6bad535f90)

* chore: fix typescript

(cherry picked from commit 826527a46b)

* fix: lint

(cherry picked from commit 77817e80ea)

* fix: use mockSanitizeArtifactPath in retention test

The 'getRetentionExpiry is called with the request object' test
referenced an undefined `mockSanitizeFilename` identifier, breaking
both lint (no-undef) and the test suite. Use the existing
`mockSanitizeArtifactPath` mock that the surrounding tests already
use, since `processCodeOutput` calls `sanitizeArtifactPath` (not
`sanitizeFilename`) before invoking `getRetentionExpiry`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit 52ea2da66d)

* fix: forward isTemporary from client for retention on file uploads and tool calls

Server-side `getRetentionExpiry` (file uploads) and the tool-call
controller both read `req.body.isTemporary`, but the file upload
multipart form and the tool-call payload did not include that field.
In `retentionMode: temporary` (default), files uploaded and tool
calls created from temporary chats were therefore retained
indefinitely.

Forward the Recoil `isTemporary` flag in both client paths so the
existing server checks can fire correctly. `ToolParams` gains an
optional `isTemporary` field.

Addresses Codex P1 review feedback on PR #29.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit 7e937df05a)

* test: stub store.isTemporary in useFileHandling test mocks

Previous commit added `useRecoilValue(store.isTemporary)` to the
hook. The test file mocks `~/store` with only `ephemeralAgentByConvoId`
and does not stub `useRecoilValue`, so all 7 cases threw
"Invalid argument to useRecoilValue: expected an atom or selector but
got undefined". Add a stub default export with `isTemporary` and a
`useRecoilValue` mock returning `false`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit eb1609537d)

* fix: harden data retention semantics

* fix: provide sweep request context for expired files

* fix: preserve temporary flags in all-retention updates

* fix: honor assistant versions in retention sweeps

* fix: retain non-temporary flags in all mode

* fix: hide expired retained records

* fix: propagate retained conversation expiry

* fix: refresh meili retention cutoff

* fix: prevent overlapping file sweeps

* fix: show legacy retained conversations

* fix: index legacy retained records

* fix: harden retention cleanup edge cases

* fix: count failed file storage sweeps

* fix: preserve legacy temporary retention

* fix: assign retention sweep worker deterministically

* fix: hide expired shared links on reads

* fix: prevent retention refresh after parent expiry

* fix: break code output retention import cycle

* fix: harden retention review findings

* fix: ignore expired share duplicates

* fix: reject expired retained share creation

* fix: harden retention review edge cases

* fix: address retention audit findings

* fix: enforce expired conversation shares in all retention

* fix: scope temporary upload flag to chat files

* fix: address retention review findings

* fix: address codex retention review findings

* fix: tighten missing storage detection

* test: remove unused file process spec bindings

---------

Co-authored-by: WhammyLeaf <233105313+WhammyLeaf@users.noreply.github.com>
Co-authored-by: Aron Gates <aron@muonspace.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-19 21:58:42 -04:00
Danny Avila
7129b1b1e4
📜 refactor: Improve Skill Handling Logs (#13057)
* refactor: Streamline batch upload error handling in `uploadCodeEnvFile`
* refactor: Enhance session info error logging in `getSessionInfo`
* refactor: Update error logging to use `logAxiosError` in various agent handlers and skill file processing functions
* refactor: Consolidate missing resource checks in `createToolExecuteHandler` for better clarity
2026-05-11 02:15:51 -04:00
Danny Avila
c67e2b54dc
🔐 feat: Mint Code API Auth Tokens (#13028)
* feat: Mint CodeAPI auth tokens

* style: Format CodeAPI download route

* fix: Prune CodeAPI token cache

* fix: Propagate CodeAPI managed auth

* test: Mock CodeAPI auth in traversal suite

* fix: Pass auth context to invoked skill cache

* feat: Mint CodeAPI plan context

* chore: Refresh CodeAPI auth guidance

* fix: Guard OpenID JWT fallback

* fix: Default CodeAPI JWT tenant in single-tenant mode

* chore: Update @librechat/agents to version 3.1.84 in package-lock.json and package.json files

* chore: Standardize references to Code API in comments and tests
2026-05-09 16:09:10 -04:00
Danny Avila
ac3600cdd7
🗂️ fix: Remove Generated Code Files From Prompt Context (#13037) 2026-05-09 11:38:53 -04:00
Danny Avila
93c4ef4ba8
🧱 refactor: typed CodeEnvRef + kind discriminator + principal-aware sandbox cache (#12960)
* 🧱 refactor: typed CodeEnvRef + kind discriminator + tenant-aware sandbox cache

Final cutover for the LibreChat ↔ codeapi sandbox file identity. Replaces
the magic string `${session_id}/${file_id}?entity_id=...` with a typed,
discriminated `CodeEnvRef`. Pre-release lockstep deploy with codeapi
#1455 and agents #148; no legacy aliases retained.

## Final shape

```ts
type CodeEnvRef =
  | { kind: 'skill'; id: string; storage_session_id: string; file_id: string; version: number }
  | { kind: 'agent'; id: string; storage_session_id: string; file_id: string }
  | { kind: 'user';  id: string; storage_session_id: string; file_id: string };
```

`kind` drives codeapi's sessionKey: `<tenant>:<kind>:<id>[✌️<version>]`
for shared kinds, `<tenant>:user:<userId>` for user-private (auth context
provides `userId`). `version` is statically required for `kind: 'skill'`
and forbidden otherwise via discriminated union — constraint holds at
compile time on every consumer, not just codeapi's runtime validator.

`id` is sessionKey-meaningful for `'skill'` / `'agent'`; informational
only for `'user'` (codeapi resolves user identity from auth context).

## What changed

- `packages/data-provider/src/codeEnvRef.ts` — discriminated union +
  `CODE_ENV_KINDS` const-tuple keeps the runtime list and TS union
  locked together.
- Schemas: `metadata.codeEnvRef` and `SkillFile.codeEnvRef` enums
  tightened to `['skill', 'agent', 'user']`.
- `primeSkillFiles` writes `kind: 'skill'`, `id: skill._id`,
  `version: skill.version`. Cache-hit path reads `codeEnvRef`
  directly. Bumping `skill.version` on edit naturally invalidates
  the prior cache entry under the new sessionKey.
- `processCodeOutput` writes `kind: 'user'`, `id: req.user.id`. Output
  bucket is always user-scoped, regardless of which skill the
  execution invoked. New regression test pins the asymmetry.
- `primeFiles` reupload preserves `kind`/`id`/`version?` from the
  existing ref so a skill-cache-miss reupload doesn't silently demote
  to user bucket.
- `crud.js` upload functions (`uploadCodeEnvFile` /
  `batchUploadCodeEnvFiles`) thread `kind`/`id`/`version?` to the
  multipart form (codeapi #1455 option α). Without these on the wire,
  codeapi falls back to user bucketing and skill-cache invalidation
  never fires. Client-side validation mirrors codeapi's validator.
- `Files/process.js` — chat attachments use `kind: 'user'`; agent
  setup files use `kind: 'agent'`.
- Drops `entity_id` everywhere (struct, schema sub-docs, write paths,
  upload form fields). Drops `'system'` from the kind enum (no emitter
  ever existed).

## Test plan

- [x] `cd packages/data-provider && npx jest src/codeEnvRef.spec` — 4 / 4
- [x] `cd packages/data-schemas && npx jest` — 1447 / 1447
- [x] `cd packages/api && npx jest src/agents` — 81 / 81 in skillFiles +
  handlers + resources
- [x] `cd api && npx jest server/services/Files server/controllers/agents` —
  436 / 436
- [x] `cd api && npx jest server/services/Files/Code` — 98 / 98 (incl.
  new "outputs are user-scoped regardless of which skill the execution
  invoked" regression and "reupload forwards kind/id/version from
  existing ref")
- [x] `npx tsc --noEmit -p packages/data-{provider,schemas}/tsconfig.json
  && npx tsc --noEmit -p packages/api/tsconfig.json` — clean (only
  pre-existing unrelated dev errors in storage/balance, untouched here)

## Deploy notes

- **24h cache-miss burst** on first deploy. Inputs (skill caches re-prime
  under new sessionKey shape) and outputs (any pre-Phase C skill-output
  cached files become unreadable). Bounded by codeapi's 24h TTL.
- **Lockstep with codeapi #1455 and agents #148.** Either repo can land
  first since no aliases to drain, but the three deploys must overlap
  within the same maintenance window.
- **`@librechat/agents` bump to `3.1.79-dev.0`** required after agents
  #148 lands and is published.

## What this enables

Auth bridge work (JWT-based tenant/user identity between LC and codeapi)
— codeapi now derives sessionKey purely from `req.codeApiAuthContext.{
tenantId, userId}`, so the next chapter is replacing the header-asserted
user identity with a verified-claim path.

* 🩹 fix: persist execute_code uploads under codeEnvRef metadata key

Codex review P1 (chatgpt-codex-connector). `Files/process.js` was
storing the upload result under `metadata.fileIdentifier` even though:
- `uploadCodeEnvFile` now returns `{ storage_session_id, file_id }`,
  not the legacy magic string.
- The post-cutover schema (`File.metadata.codeEnvRef`) only declares
  `codeEnvRef` — mongoose strict mode silently strips unknown keys.
- All readers (`primeFiles`, `getCodeFilesByIds`,
  `categorizeFileForToolResources`, controller filtering) check
  `metadata.codeEnvRef`.

Net effect of the bug: chat-attached and agent-setup execute_code files
would lose their sandbox reference on save, and primeFiles would skip
them on subsequent code-execution turns — the file blob would still be
available locally but never re-mounted in the sandbox.

Fix: construct the full `CodeEnvRef` (`{ kind, id, storage_session_id,
file_id }`) at the write site and persist under `metadata.codeEnvRef`.
`BaseClient`'s "is this a code-env file" presence check accepts the new
shape alongside the legacy `fileIdentifier` for back-compat with any
pre-cutover records still in the database. Mirrors the same change in
`processAttachments.spec.ts` (which re-implements the BaseClient logic
for testability).

New regression tests in `process.spec.js` cover three cases:
- chat attachments (`messageAttachment=true`) → `kind: 'user'`
- agent setup (`messageAttachment=false`) → `kind: 'agent'`
- legacy `fileIdentifier` key is NOT persisted (would be schema-stripped)

* 🩹 fix: read storage_session_id on primed file refs (Codex P1)

Codex review (chatgpt-codex-connector). After Phase B's per-file
`session_id` → `storage_session_id` rename, `primeFiles` emits the
new field — but `seedCodeFilesIntoSessions` was still reading
`files[0].session_id` for the representative session and `f.session_id`
for the dedupe key. In runs with only primed attachments (no skill
seed), `representativeSessionId` was `undefined`, the function
returned the unchanged map, and `seedCodeFilesIntoSessions` silently
dropped the entire batch. The first `execute_code` call then started
without `_injected_files` and the agent couldn't see prior-turn
artifacts.

Fix:
- `codeFilesSession.ts`: read `f.storage_session_id` for both the
  dedupe key and the representative session id. JSDoc updated to
  match the new field name.
- `callbacks.js`: the two output-file persistence paths read
  `file.session_id` to pass to `processCodeOutput` — switch to
  `file.storage_session_id`. The original comment explicitly says
  this should be the STORAGE session, which is exactly the field
  Phase B renamed.
- `codeFilesSession.spec.ts`: fixture builder uses `storage_session_id`
  and `kind: 'user'` to match the post-cutover `CodeEnvFile` shape.

Lockstep coordination: this matches the post-bump shape of
`@librechat/agents` 3.1.79+. CI tsc errors against the currently-pinned
3.1.78 are expected and resolve when the dep bumps in this PR before
merge.

* 📦 chore: Bump `@librechat/agents` to version 3.1.80-dev.0 in package-lock and package.json files

* 🪪 fix: thread kind/id/version through codeapi /download URLs (Phase C α)

Symmetric fix for the upload-side wire change in 537725a. Codeapi's
`sessionAuth` middleware now requires `kind`/`id`/`version?` on every
download/freshness URL — without them it 400s with "kind must be one
of: skill, agent, user" before serving the file.

Three sites construct codeapi-side URLs that go through `sessionAuth`:

- `processCodeOutput` (`Files/Code/process.js`): `/download/<sess>/<id>`
  for freshly-generated sandbox outputs. Always `kind: 'user'` +
  `id: req.user.id` — code-output files are always user-private,
  regardless of which skill the run invoked.
- `getSessionInfo` (`Files/Code/process.js`): `/sessions/<sess>/objects/<id>`
  for the 23h freshness check. Pulls kind/id/version straight off the
  `codeEnvRef` already in scope — skill files stay skill-bucketed,
  user files stay user-bucketed.
- `/code/download/:session_id/:fileId` LC route (`routes/files/files.js`):
  proxies to codeapi for manual downloads. Code-output files only on
  this route, so `kind: 'user'` + `id: req.user.id`.

The `getCodeOutputDownloadStream` helper in `crud.js` now takes an
`identity` param, validated by a `buildCodeEnvDownloadQuery` helper
that mirrors `appendCodeEnvFileIdentity`'s shape rules: kind required
from the closed `{skill, agent, user}` set, version required for
'skill' and forbidden otherwise. Bad callers fail fast on the client
instead of round-tripping a 400.

Also cleans up two log-noise sources reported alongside the 400:

- `logAxiosError` in `packages/api/src/utils/axios.ts` was dumping
  `error.response.data` raw. With `responseType: 'arraybuffer'` that's
  a `Buffer` (~4 chars per byte after JSON-serialization); with
  `responseType: 'stream'` it's a `Readable` whose internal state
  serializes the entire ring buffer + socket. New `renderResponseData`
  decodes small buffers as UTF-8 (truncated past 2KB) and stubs streams
  as `'[stream]'`. Diagnostics stay useful, log lines stop being
  megabytes.
- `/code/download` route's catch was bare `logger.error('...', error)`,
  bypassing the redactor. Switched to `logAxiosError` so it benefits
  from the same buffer/stream handling.

Tests updated to match the new contract:
- crud.spec: `getCodeOutputDownloadStream` fixtures pass `userIdentity`;
  new cases cover skill identity (with version), bad kind rejection,
  skill-without-version rejection.
- process.spec: `getSessionInfo` test passes a full `codeEnvRef` object.

* ♻️ refactor: extract codeEnv identity helpers into packages/api

Per the project convention that new backend code lives in TypeScript
under `packages/api`, moves `appendCodeEnvFileIdentity` and
`buildCodeEnvDownloadQuery` from `api/server/services/Files/Code/crud.js`
into a new `packages/api/src/files/code/identity.ts` module.

Both helpers are pure validators that mirror codeapi's
`parseUploadSessionKeyInput` server-side rules (closed kind set,
`version` required for `'skill'` and forbidden otherwise) — they
deserve TS support and a dedicated spec rather than living as
JSDoc-typed helpers in the legacy `/api` workspace. The new module:

- Exports a `CodeEnvIdentity` interface using the
  `librechat-data-provider` `CodeEnvKind` discriminated union.
- Adds 13 unit tests in `identity.spec.ts` covering the validation
  matrix (skill+version, agent, user, and every rejection path) plus
  URL encoding for the download query.
- Re-exported from `packages/api/src/files/code/index.ts` alongside
  `classify`, `extract`, and `form`.

Consumer updates:
- `api/server/services/Files/Code/crud.js`: drops the local helpers
  and imports them from `@librechat/api`. Net -64 lines.
- `api/server/services/Files/Code/process.js`: same.
- Test mocks for `@librechat/api` in three spec files now stub the
  helpers' validation behavior locally rather than pulling them
  through `requireActual` (which would drag in provider-config
  init-time side effects). The package's `exports` field only
  surfaces the root barrel, so leaf imports aren't reachable from
  legacy `/api` test setup.

No runtime behavior change. Identity validation rules and emitted
form/query shapes are byte-for-byte identical pre/post.

* 🪪 fix: emit resource_id alongside id on _injected_files (skill 403 fix)

Companion to codeapi #1455 fix and agents 3.1.80-dev.1 — the wire
shape for shared-kind files now requires `resource_id` distinct from
the storage `id`. Without this LC change, codeapi's sessionKey
re-derivation on every shared-kind /exec rejects with 403
session_key_mismatch:

    cached:  legacy:skill:69dcf561...✌️59  (signed at upload, skill _id)
    derived: legacy:skill:ysPwEURuPk-...✌️59  (storage nanoid)

Emit sites updated:

- `primeInvokedSkills` cache-hit path: `resource_id: ref.id` (the
  persisted skill `_id` from `codeEnvRef.id`); `id: ref.file_id`
  unchanged (storage uuid).
- `primeInvokedSkills` fresh-upload path: `resource_id: skill._id.toString()`
  on every primed file (the `allPrimedFiles` builder type now carries
  the field).
- `processCodeOutput`'s `pushFile` (Code/process.js): `resource_id: ref.id`
  — for `kind: 'user'` this is informational (codeapi derives
  sessionKey from auth context) but emitted for shape uniformity
  with shared kinds.

Bumps `@librechat/agents` to `^3.1.80-dev.1` (the version that
ships the matching `CodeEnvFile.resource_id` field).

## Test plan

- [x] `cd packages/api && npx jest src/agents` — 67 / 67 pass
  (skillFiles fixtures updated to assert `resource_id` on the
  emitted CodeSessionContext.files).
- [x] `cd api && npx jest server/services/Files server/controllers/agents` —
  445 / 445 pass (process.spec fixtures updated for the reupload
  + cache-hit emission).
- [x] `npx tsc --noEmit -p packages/api/tsconfig.json` — clean.

* fix(skill-tool-call): carry resource_id through primeSkillFiles → artifact

Codeapi was 400ing every /exec following a `handle_skill` tool call
with `resource_id is invalid` (`type: 'undefined'`). Both code paths
in `primeSkillFiles` (cache-hit + fresh-upload) returned files
without `resource_id`/`kind`/`version`, and the artifact in
`handlers.ts` forwarded the stripped shape into
`tc.codeSessionContext.files` → `_injected_files`.

`primeInvokedSkills` (the NL-detected loader) had already been fixed
end-to-end; this commit aligns the tool-invoked path with the same
contract: `resource_id` = `skill._id.toString()`, `kind: 'skill'`,
`version` = the skill's monotonic counter.

Tests added to `skillFiles.spec.ts` lock the contract on
`primeSkillFiles` directly so future refactors can't silently drop
the resource identity again.

* fix(handlers.spec): align session_id → storage_session_id rename + kind discriminator

Pre-existing TS errors against the post-rename `CodeEnvFile` shape:
the test file still used `session_id` on per-file objects (renamed to
`storage_session_id` in agents Phase B/C) and was missing the `kind`
discriminator the discriminated union requires. Both inputs and the
matching `expect.toEqual(...)` mirrors updated together so the
runtime equality check still holds.

Lines 723-732 stay as-is — they sit behind `as unknown as
ToolCallRequest` and TS already skipped them.

* chore: fix `@librechat/agents`, correct version to 3.1.80-dev.0 in package.json files

* chore: bump `@librechat/agents` to version 3.1.80-dev.1 in package.json and package-lock.json

* chore: bump `@librechat/agents` to version 3.1.80-dev.2

* feat(observability): trace file priming chain from primeCodeFiles to _injected_files

Diagnosing the user-upload "files=[] on first /exec" bug requires
seeing where in the LC chain a file ref disappears. Prior to this
patch the chain (primeCodeFiles → primedCodeFiles → initialSessions
→ CodeSessionContext → _injected_files) was opaque end-to-end:
  - primeCodeFiles silently dropped files without `metadata.codeEnvRef`
  - reuploadFile catches all errors and continues with no signal
  - the handlers.ts handoff to codeapi never logged what it was sending

After this patch, a single grep on `[primeCodeFiles]` plus
`[code-env:inject]` shows the full per-file path:

  [primeCodeFiles] in: file_ids=N resourceFiles=M
  [primeCodeFiles] file=<id> path=skip reason=no-codeenvref filename=...
  [primeCodeFiles] file=<id> path=cache-hit-by-session storage_session_id=...
  [primeCodeFiles] file=<id> path=reupload reason=no-uploadtime ...
  [primeCodeFiles] file=<id> path=reupload reason=stale ...
  [primeCodeFiles] file=<id> path=reupload-success oldSession=... newSession=... newFileId=...
  [primeCodeFiles] file=<id> path=reupload-failed session=...
  [primeCodeFiles] file=<id> path=fresh-active storage_session_id=...
  [primeCodeFiles] out: returned=N skippedNoRef=M reuploadFailures=K

  [code-env:inject] tool=<name> files=N missingResourceId=K     (debug)
  [code-env:inject] M/N files missing resource_id ...           (warn)
  [code-env:inject] tool=<name> _injected_files=0 ...           (warn)

The boundary log warns when LC sends zero injected files on a
code-execution tool call — that's the user's actual symptom showing
up at the LC side instead of having to correlate against codeapi's
`Request received { files: [] }`.

Tag chosen as `[code-env:inject]` rather than `[handoff:exec]` to
avoid collision with the app-level "handoff" semantic (subagent
handoff workflow).

Structural cleanup in primeFiles: replaced the `if (ref) { ... }`
nesting with an early `if (!ref) continue` so the per-path
instrumentation hooks land at top-level scope instead of indented
inside a conditional. Behavior unchanged; pushFile / reuploadFile
identical.

Spec fixtures (handlers.spec.ts, codeFilesSession.spec.ts) updated
to include `resource_id` on `CodeEnvFile` literals — required by
the post-3.1.80-dev.2 type now installed.

## Test plan

- [x] `cd packages/api && npx jest src/agents/handlers.spec.ts src/agents/codeFilesSession.spec.ts src/agents/skillFiles.spec.ts` — 69/69 pass
- [x] `cd api && npx jest server/services/Files/Code/process.spec.js` — 84/84 pass
- [x] `npx tsc --noEmit -p packages/api` — clean
- [x] `npx eslint` on all four touched files — clean

* chore: add CONSOLE_JSON_STRING_LENGTH to .env.example for JSON log string length configuration

* fix(files): align codeapi upload filename with LC's sanitized DB filename

User-attached files for code execution were uploading to codeapi
under `file.originalname` (raw upload filename, may contain spaces /
special chars) while LC's DB record stored the sanitized form
(`sanitizeFilename(file.originalname)`, underscores). Codeapi
preserves whatever filename the upload sent, so the sandbox saw
`/mnt/data/<originalname>` while LC's `primeFiles` toolContext text
+ `_injected_files.name` referenced `file.filename` (sanitized).

Visible failure: agent gets system prompt saying

    /mnt/data/librechat_code_api_-_active_customer_-_2025-11-05.xlsx

…tries that path, hits `FileNotFoundError`, then notices the
sandbox's actual `Available files` line says

    /mnt/data/librechat code api - active customer - 2025-11-05.xlsx

…retries with spaces, succeeds. Wastes a tool call per upload and
leaks raw filenames into model context.

Fix: sanitize once and use the sanitized form in both the codeapi
upload AND the LC DB record. Sandbox path = LC toolContext text =
in-memory ref name. No drift.

Reupload path (`Code/process.js` line 867 `filename: file.filename`)
already uses the sanitized DB name, so it stays consistent with the
fresh-upload path after this change.

## Test plan

- [x] `cd api && npx jest server/services/Files/process` — 32/32 pass
- [x] `npx eslint` on the touched file — clean

* chore: bump `@librechat/agents` to version 3.1.80-dev.3 in package.json and package-lock.json
2026-05-08 12:29:43 -04:00
Danny Avila
8f92ec012c
🧭 fix: Navigate Signed CDN Downloads (#12998)
* fix(files): navigate signed CDN downloads

* fix(files): avoid popup target for signed downloads

* test(files): restore download URL mock
2026-05-07 13:36:57 -04:00
Danny Avila
1bc2692a15
🌥️ feat: Add Optional Region-aware S3/CloudFront Storage Keys (#12987)
* feat(files): add optional region-aware storage keys

* test(files): fix region storage CI fixtures

* feat(files): finalize inline CloudFront asset namespaces

* fix(files): allow wildcard region CloudFront cookies

* fix(files): preserve legacy storage key compatibility

* fix(files): align CloudFront clear cookie cleanup

* fix(files): clear legacy CloudFront cookie scopes

* chore(files): clean up storage review nits

* fix(files): keep inline namespaces CloudFront-only
2026-05-06 23:16:56 -04:00
Danny Avila
9c81792d25
🔐 feat: Add Signed CloudFront File Downloads (#12970)
* feat: add signed CloudFront downloads

* fix: preserve local IdP avatar paths

* fix: address signed download review findings

* fix: harden CloudFront cookie scope validation

* fix: preserve URL save API compatibility

* fix: store CDN SSO avatars under shared prefix

* fix: Harden CloudFront tenant file access

* fix: Preserve CloudFront download compatibility

* fix: Address CloudFront review follow-ups

* fix: Preserve file URL fallback user paths

* fix: Address download review hardening

* fix: Use file owner for S3 RAG cleanup

* fix: Address final download review nits

* fix: Clear stale avatar CloudFront cookies

* fix: Align download filename helpers with dev

* fix: Address final CloudFront review follow-ups

* fix: Stream S3 URL uploads

* fix: Set S3 stream upload length

* fix: Preserve download metadata filepath

* fix: Avoid remote content length for stream uploads

* fix: Use bounded multipart URL uploads

* fix: Harden S3 filename boundaries
2026-05-06 19:48:30 -04:00
Danny Avila
6c6c72def7
🚀 feat: Decouple File Attachment Persistence from Preview Rendering (#12957)
* 🗂️ feat: add `status` lifecycle to file records for two-phase previews

Schema and model foundation for decoupling the agent's final response
from CPU-heavy office-format HTML extraction.

- `MongoFile.status: 'pending' | 'ready' | 'failed'` (indexed) and
  `previewError?: string` mirror the lifecycle: phase-1 emits the file
  record at `pending` so the response is unblocked; phase-2 transitions
  to `ready` (with text/textFormat) or `failed` (with previewError) in
  the background. Absent for legacy records — clients treat that as
  `ready` for back-compat.
- Mirror types added to `TFile` in data-provider so frontend cache
  consumers see the new fields.
- New `sweepOrphanedPreviews(maxAgeMs)` method on the file model
  recovers stale `pending` records left behind by a process restart
  mid-extraction; transitions them to `failed` with
  `previewError: 'orphaned'`. Cheap because `status` is indexed.

* ⚡ feat: two-phase code-execution preview flow (unblocks final response)

The agent's final response no longer waits on CPU-heavy office HTML
extraction. Phase-1 (download + storage save + DB record at
`status: 'pending'`) is awaited as before; phase-2 (extract +
`updateFile`) runs in the background with a hard 60s ceiling.

Three flows, all funneling through `processCodeOutput` and updated to
the new `{ file, finalize? }` return shape:

- `callbacks.js` (chat-completions + Open Responses streaming): emit
  the phase-1 attachment immediately (carries `status: 'pending'` for
  office buckets so the UI shows "preparing preview…"), then
  fire-and-forget `finalize()`. If the SSE stream is still open when
  phase-2 lands, push an `attachment` update event with the same
  `file_id` so the client merges over the placeholder in place.

- `tools.js` direct endpoint: same split — return the phase-1
  metadata immediately, run extraction in the background. Client
  polls for the resolved record.

`finalize()` wraps the existing 12s per-render timeout in a 60s outer
`withTimeout`. The HTML-or-null contract from #12934 is preserved:
office types that fail extraction transition to `status: 'failed'`
with `previewError: 'parser-error' | 'timeout'` rather than falling
back to plain text (would be an XSS vector).

Promises continue running after the HTTP response closes (Node
doesn't kill them). The boot-time orphan sweep covers the only case
that loses progress — actual process restart mid-extraction.

`primeFiles` annotates the agent's `toolContext` line for prior-turn
files: `(preview not yet generated)` for pending, `(preview
unavailable: <reason>)` for failed. The model can volunteer "you can
still download it" instead of pretending the preview is fine.

`hasOfficeHtmlPath` exported from `@librechat/api` so `processCodeOutput`
can decide whether a file expects a preview at all.

* 🔍 feat: `GET /api/files/:file_id/preview` endpoint and boot orphan sweep

- New `GET /api/files/:file_id/preview` route returns
  `{ status, text?, textFormat?, previewError? }`. The frontend's
  `useFilePreview` React Query hook polls this while phase-2 is in
  flight, then auto-stops on terminal status. ACL identical to the
  download route (reuses `fileAccess` middleware). Defaults `status`
  to `'ready'` for legacy records so back-compat is implicit.
  `text` only included when `status === 'ready'` and non-null —
  preserves the HTML-or-null security contract from #12934.

- `sweepOrphanedPreviews()` invoked on boot in both `server/index.js`
  and `server/experimental.js`. Recovers any `pending` records left
  behind by a process restart mid-extraction (the only case the
  in-process two-phase flow can't handle on its own). Fire-and-forget
  so a transient sweep failure doesn't block startup.

* 🖥️ feat: frontend two-phase preview consumer (polling + UI states)

Wires the React side to the new lifecycle so the user sees what's
happening with their file while phase-2 extraction runs in the
background and after the response stream closes.

- `useAttachmentHandler` upserts by `file_id` (was append-only) so
  the phase-2 SSE update event merges over the pending placeholder
  in place. Lightweight attachments without a `file_id`
  (web_search / file_search citations) keep the legacy append path.

- `useFilePreview(file_id)` React Query hook with
  `refetchInterval: (data) => data?.status === 'pending' ? 2500 : false`
  so polling auto-stops on the first terminal response without the
  caller having to flip `enabled`.

- `useAttachmentPreviewSync(attachment)` bridges polled data into
  `messageAttachmentsMap`. Polling enabled iff
  `status === 'pending' && isAnySubmitting` — per the design ask:
  active polling while the LLM is still generating, then quiet.
  Process-restart and post-stream cases are covered by polling on
  the next interaction.

- `Attachment.tsx` renders a small `PreviewStatusIndicator` (spinner +
  "Preparing preview…" for pending, alert icon + "Preview unavailable"
  for failed) inside `FileAttachment`. Download button stays fully
  functional in both states. Two new English locale keys.

- Data-provider scaffolding: `TFilePreview` type, `endpoints.filePreview`,
  `dataService.getFilePreview`, `QueryKeys.filePreview`.

* 🧪 fix: stub `useAttachmentPreviewSync` in pre-existing Attachment test mocks

The new `useAttachmentPreviewSync` hook is called unconditionally inside
`FileAttachment` (added in the prior commit). Two pre-existing test
files mock `~/hooks` to provide `useLocalize` only — the un-mocked
preview hook reference resolved to undefined and crashed render with
`(0 , _hooks.useAttachmentPreviewSync) is not a function` on the
Ubuntu/Windows CI runners.

Fix is local to the test mocks: add a no-op stub that returns
`{ status: 'ready' }` so the component renders the legacy chip path.
The two-phase preview behavior itself has its own dedicated suites
(`useAttachmentHandler.spec.tsx`, `useAttachmentPreviewSync.spec.tsx`).

* 🐛 fix: route phase-2 attachment update to current-run messageId

Codex P1 review on PR #12957. `processCodeOutput` intentionally
preserves the original DB `messageId` across cross-turn filename reuse
so `getCodeGeneratedFiles` can still trace a file back to the
assistant message that originally produced it. The phase-1 SSE emit
already routes by the current run's messageId — `processCodeOutput`
runtime-overlays it via `Object.assign(file, { messageId, toolCallId })`
and the callback writes `result.file` directly.

Phase-2 was passing the raw `updateFile` return through
`attachmentFromFileMetadata`, which read `messageId` straight off the
DB record. On a turn-N run that re-emitted a filename from turn-1
(e.g. agent writes `output.csv` again), the phase-2 SSE update
routed to `turn-1-msg` instead of `turn-N-msg`. Frontend's
`useAttachmentHandler` upserts under the wrong messageAttachmentsMap
slot — turn-N's pending chip stays stuck at "preparing preview…"
while turn-1's already-resolved attachment gets re-merged.

Fix: thread `runtimeMessageId` through `attachmentFromFileMetadata`
and pass `metadata.run_id` from the phase-2 emit site. Mirrors how
phase-1 sources its messageId. Tests cover the cross-turn reuse case
plus the writableEnded / null-finalize / no-finalize paths to lock
in the broader phase-2 emit contract.

* 🛠️ refactor: address codex audit findings (wire-shape parity, DRY, defensive catch)

Comprehensive audit on PR #12957. Resolves all valid findings:

- **MAJOR #1 — Wire-shape parity**: phase-1 ships the full `fileMetadata`
  record over SSE; phase-2 was using a tight `attachmentFromFileMetadata`
  projection. Drop the projection and have phase-2 spread `{...updated,
  messageId, toolCallId}` so both events match the long-standing
  legacy phase-1 shape clients depend on.

- **MAJOR #2 — DRY**: extract `runPhase2Finalize({ finalize, fileId,
  onResolved })` into `process.js` (alongside `processCodeOutput` whose
  contract it pairs with). Both `callbacks.js` paths and `tools.js`
  now flow through it. Single catch path eliminates divergence
  surface — the fix landed in 01704d4f0 (cross-turn messageId routing)
  was a symptom of this duplication risk.

- **MINOR #3 — JSDoc accuracy**: `finalizePreview`'s buffer is bounded
  by `fileSizeLimit`, not the 1MB extractor cap. Updated and added a
  note about peak heap from queued buffers.

- **MINOR #4 — Defensive catch**: `runPhase2Finalize`'s catch attempts
  a best-effort `updateFile({ status: 'failed', previewError:
  'unexpected' })` for the file_id, so a programming bug in
  `finalizePreview` doesn't leave the record stuck `'pending'` until
  the next boot-time orphan sweep.

- **NIT #6 — Stale PR refs**: 12952 → 12957 in 3 places.

- **NIT #7 — Schema bound**: `previewError` capped at `maxlength: 200`
  to prevent a future codepath from accidentally persisting a stack
  trace.

Skipped per audit verdict (non-blocking):
- #5 (memory pressure): documented in JSDoc; impl change was reviewer's
  "consider", not actionable.
- #8 (double DB query per poll): low cost, indexed by_id, polling is
  gated narrow.
- #9 (TAttachment cast): the union type is intentional; the casts are
  safe widening, refactoring TAttachment is invasive and out of scope.

Tests: 11 new (7 `runPhase2Finalize` unit tests covering happy path,
null-finalize, throws, double-fail, no-fileId, no-onResolved; +4
wire-shape parity assertions in the existing cross-turn test). 328
backend tests pass; 528 frontend tests pass; lint and typecheck clean.

* 🛡️ refactor: address codex P1+P2 + rename to drop phase-1/2 jargon

Codex round 2 review on PR #12957 caught two race conditions and one
recovery gap, all triggered by cross-turn filename reuse (`claimCodeFile`
intentionally returns the same `file_id` for the same
`(filename, conversationId)` across turns). Plus naming cleanup the
user requested — internal "phase 1 / phase 2" vocabulary leaks across
sprints, replace it everywhere with terms describing what's actually
happening.

P1 — stale render overwrites newer revision (process.js)
  Two turns reusing `output.csv` share a `file_id`. If turn-1's
  background render resolves AFTER turn-2's persist step, the
  unconditional `updateFile` writes turn-1's stale text/status over
  turn-2's pending placeholder. Fix: stamp a fresh `previewRevision`
  UUID on every emit, thread it through `finalizePreview`, and make
  the commit conditional via a new optional `extraFilter` argument
  on `updateFile` (`{ previewRevision: <expected> }`). The defensive
  `updateFile` in `runPreviewFinalize`'s catch uses the same guard
  so a programming error from an older render also can't override a
  newer turn.

P1 — stale React Query cache on pending remount (queries.ts)
  Same root cause from the frontend side. Cache key
  `[QueryKeys.filePreview, file_id]` may hold a prior turn's `'ready'`
  payload; with `refetchOnMount: false` and the polling gate on
  `pending`, polling never starts for the new placeholder. Fix:
  `useAttachmentHandler` invalidates that query whenever an attachment
  with a `file_id` arrives. Both initial-emit and update events
  trigger invalidation — uniform gate.

P2 — quick-restart orphans skipped by boot sweep (files.js)
  Boot `sweepOrphanedPreviews` uses a 5-min cutoff for multi-instance
  safety. A crash + restart inside the cutoff leaves `pending` records
  that never get touched again. Fix: lazy sweep inside the preview
  endpoint — if a polled record is `pending` and `updatedAt` is older
  than 5 min, mark it `failed:orphaned` on the spot before responding.
  Conditional on the same `updatedAt` we observed so a concurrent
  legitimate update wins. Cheap, bounded by user activity.

Naming cleanup
  - `runPhase2Finalize` → `runPreviewFinalize`
  - `PHASE_TWO_TIMEOUT_MS` → `PREVIEW_FINALIZE_TIMEOUT_MS`
  - All `phase-1` / `phase-2` / `two-phase` prose replaced with
    "the immediate emit", "the deferred render", "the persist step",
    "the deferred preview", etc. Skill-feature `phase 1/2` references
    (different feature) left alone.

Tests: 10 new (4 lazy-sweep × preview endpoint, 3 cache-invalidation ×
useAttachmentHandler, 3 extraFilter × updateFile data-schemas).
Backend 332/332, frontend 531/531, data-schemas 37/37, lint clean.

* 🛠️ refactor: address comprehensive review (round 3) — stale-cache MAJOR + 3 minors

Comprehensive review on PR #12957 caught a P1 follow-on bug from the
prior `invalidateQueries` fix, plus 3 maintainability findings.

MAJOR: stale React Query cache not actually fixed by `invalidateQueries`
  The previous fix called `invalidateQueries` to flush stale cached
  preview data on cross-turn filename reuse. But `useFilePreview` had
  `refetchOnMount: false`, which made the new observer read the
  stale-marked 'ready' data without refetching. The polling
  `refetchInterval` then evaluated against stale 'ready' → returned
  `false` → polling never started → user stuck on stale content.

  Fix (belt-and-suspenders):
    a) `useAttachmentHandler` switched to `removeQueries` — drops the
       cache entry entirely so the next mount has nothing to read and
       must fetch.
    b) `useFilePreview` no longer sets `refetchOnMount: false`, so the
       React Query default (`true`) kicks in — second line of defense
       if any future codepath observes stale data before the handler
       has a chance to evict.

MINOR: `finalizePreview` JSDoc missing `previewRevision` param
  Added with explanation of the conditional update guard.

MINOR: asymmetric stream-writable guard between SSE protocols
  Chat-completions delegated the gate to `writeAttachmentUpdate`;
  Open Responses inlined `!res.writableEnded && res.headersSent`.
  Extracted `isStreamWritable(res, streamId)` predicate; both paths
  + `writeAttachmentUpdate` now share the single source of truth.

NIT: `(data as Partial<TFile>).file_id` cast repeated 4 times
  Extracted to a `fileId` local at the top of the handler.

Tests: existing 9 invalidate-tests rewritten as remove-tests; +1 new
lock-in test asserts removeQueries is called and invalidateQueries
is NOT (regression guard against round-3 finding). 332 backend pass,
532 frontend pass, lint clean.

Skipped findings (deferred / acceptable):
- MINOR: post-submission pending state has no auto-recovery — the
  `isAnySubmitting` polling gate was the user's explicit design;
  LLM context surfaces failed/pending so the model can volunteer.
  Worth a follow-up if real users hit it.
- NIT: double DB query per preview poll — reviewer marked acceptable;
  changing `fileAccess` middleware is out of scope.

* 🛡️ test: address comprehensive review NITs (initial-emit guard + isStreamWritable coverage)

NIT — chat-completions initial emit skips writableEnded check
  The Open Responses initial emit was switched to use the new
  `isStreamWritable` predicate in the round-3 commit, but the
  chat-completions initial emit kept the older narrower check
  (`streamId || res.headersSent`). On a client disconnect mid-stream
  (`writableEnded === true`) it would still hit `res.write` and
  raise `ERR_STREAM_WRITE_AFTER_END` — caught by the outer IIFE
  catch but logged as noise. Switch this site to `isStreamWritable`
  too so both initial-emit paths share the same gate as the
  deferred update emits.

NIT — `isStreamWritable` not directly unit-tested
  The predicate was only covered indirectly via the deferred-preview
  SSE tests (writableEnded skip, headersSent check). Export from
  `callbacks.js` and add 5 parametric tests pinning down each branch
  (streamId truthy, res null, !headersSent, writableEnded, happy
  path) so a future condition addition can't silently regress.

* 🐛 fix: stuck "Preparing preview…" + inline the chip subtitle

Two related fixes for a stuck-spinner bug a user reported in manual
testing of PR #12957.

**Stuck spinner (the bug)**
The deferred preview render can complete a few seconds AFTER the SSE
stream closes (typical case: PPTX render finishes ~3s after the LLM
emits FINAL). When that happens, the SSE update is silently dropped
(`isStreamWritable` returns false on a closed stream) and polling is
the only recovery path.

The earlier polling gate was `status === 'pending' && isAnySubmitting`,
which mirrored the original design intent ("only query while the LLM
is still generating"). But `isAnySubmitting` flips false the moment
the model emits FINAL — milliseconds before the deferred render
commits. Polling never runs, the chip stays "Preparing preview…"
forever even though the DB has `status: 'ready'` with valid HTML.

Drop the `isAnySubmitting` part of the gate. `useFilePreview`'s
`refetchInterval` is already a function-form that returns `false` on
the first terminal response, so polling auto-stops within one tick of
resolution. The server-side render ceiling (60s) plus the lazy sweep
in the preview endpoint cap the worst case to ~24 polls per pending
attachment. Polling itself never blocks UX — the gate's purpose was
"don't waste cycles", and capping by terminal status is the correct
expression of that.

**Inline the chip subtitle (the visual)**
The previous design rendered "Preparing preview…" as a loose-feeling
spinner+text BELOW the file chip. The chip itself looked done while a
floating annotation said it wasn't.

`FileContainer` gains an optional `subtitle?: ReactNode` prop that
overrides the default file-type label. `Attachment.tsx` passes a
`PreviewStatusSubtitle` (spinner + "Preparing preview…" / alert +
"Preview unavailable") into that slot when the file's preview is
pending or failed. The chip footprint stays identical to its `'ready'`
form — just the second row swaps from "PowerPoint Presentation" to
the status indicator. No floating element, no layout shift.

Tests: regression test pinning down "polling stays enabled after the
LLM finishes" so a future revert can't reintroduce the stuck-spinner
bug. Existing FileContainer tests pass unchanged (subtitle override
is opt-in). 522 frontend tests pass; lint clean.

* 🐛 fix: deferred-preview survives reload + matches artifact card chrome

Fixes the remaining stuck-pending case after the polling gate fix: on
a reloaded conversation, message.attachments come from the DB frozen at
the immediate-persist `status: 'pending'`, but `messageAttachmentsMap`
is empty because no SSE handler ever fired for that messageId. Polling
now INSERTS a new live entry when no record matches the file_id, and
`useAttachments` merges live entries onto DB entries by file_id so the
resolved text/textFormat reach `artifactTypeForAttachment` and the
chip routes through the proper PanelArtifact card.

Also replaces the small file chip used during the pending state with
a PreviewPlaceholderCard that mirrors ToolArtifactCard chrome, so the
transition to the resolved PanelArtifact no longer reshapes the UI.

* ✨ feat: auto-open panel when deferred preview resolves pending→ready

The legacy auto-open path is gated only on `isSubmitting`, so an
office-file preview that resolves *after* the SSE stream closes would
render in place but never auto-open the panel — even though that's
exactly the moment the result becomes meaningful to the user. Adds a
per-file_id one-shot signal that `useAttachmentPreviewSync` flips on
the pending→ready edge; `ToolArtifactCard` consumes it on mount and
auto-opens regardless of submission state. The signal is *only* set on
the actual transition (history loads of pre-resolved files don't
trigger it) and is consumed once (panel close + reopen on the same
card stays user-controlled).

* 🐛 fix: drop placeholder Terminal overlay + scope auto-open to fresh resolutions

Two fixes for issues spotted in manual testing of the deferred-preview
auto-open feature:

1. PreviewPlaceholderCard was passing `file={attachment}` to FilePreview,
   which triggered SourceIcon's Terminal overlay (`metadata.fileIdentifier`
   is set on every code-execution file). The artifact card itself doesn't
   show that overlay; the placeholder shouldn't either, so the
   pending→resolved transition is visually seamless.

2. The `previewJustResolved` flag flipped on every pending→ready
   transition observed by the polling hook — including stale-pending
   DB records that resolve via the first poll on a *history load*.
   Conversations whose immediate-persist snapshot left attachments at
   `status: 'pending'` would yank the panel open every revisit.
   Adds `mountedDuringStreamRef` to the hook (mirroring ToolArtifactCard)
   so the flag fires only when the hook itself was mounted during an
   active turn — preserving the pre-PR contract that the panel only
   auto-opens for results the user is actively waiting on, never for
   history.

* 🐛 fix: don't downgrade preview to failed when only the SSE emit throws

Codex P2 finding on PR #12957: the original chain placed `.catch` after
`.then(onResolved)`, so a throw inside `onResolved` (transport-side
errors — SSE write race after stream close, an emitter listener
throwing) would propagate into the finalize catch and persist
`status: 'failed'` / `previewError: 'unexpected'`. That surfaced
"preview unavailable" in the UI for a perfectly valid file, and
degraded next-turn LLM context to reflect a non-existent failure.

Wraps `onResolved` in its own try/catch so emit errors are logged but
do not affect the file's persisted status. Extraction success and
emit success are now independent: if extraction succeeds and
`finalizePreview` writes the terminal status, the polling layer / next
page load surfaces the resolved preview even if this turn's SSE emit
didn't land.

* 🛡️ fix: run boot-time orphan sweep under system tenant context

Codex P2 finding on PR #12957: `File` is tenant-isolated, so under
`TENANT_ISOLATION_STRICT=true` the boot-time `sweepOrphanedPreviews`
threw `[TenantIsolation] Query attempted without tenant context in
strict mode` and the recovery path silently failed every restart.
Stale `status: 'pending'` records would be stuck until a user happened
to poll the preview endpoint and trigger the lazy sweep — which only
covers the file the user is currently looking at, not the bulk
candidate set the boot sweep is designed to recover.

Wraps the sweep in `runAsSystem(...)` in both boot paths
(`api/server/index.js` and `api/server/experimental.js`) and pins the
contract with regression tests in `file.spec.ts` — one test asserts
the bare call throws under strict mode, the other asserts the
`runAsSystem`-wrapped call succeeds.

* 🧹 chore: trim verbose comments from previous commit

* 🧹 chore: address review findings (dead branch, lazy-sweep cutoff, stale JSDoc)

- finalizePreview: drop unreachable !isOfficeBucket branch (caller
  already gates on hasOfficeHtmlPath, so this path is always office)
- preview endpoint: drop lazy-sweep cutoff from 5min to 2min — anything
  past the 60s render ceiling is definitively orphaned, and per-request
  sweep can be tighter than the per-instance boot sweep
- strip stale `isSubmitting` references from JSDoc in 3 spots (the
  client-side gate was removed in 9a65840)

Skipped: function-length (#3) and client-side polling cap (#4) —
refactors without correctness/perf wins; remaining NITs.

* 🧹 fix: trim 1 query off pending polls + clear stale lifecycle on cross-shape updates

- Preview endpoint: reuse fileAccess middleware's record for the
  lifecycle check; only re-fetch with text on the terminal ready
  response. Cuts the typical poll lifecycle from 2(N+1) to N+1
  queries, since the vast majority of polls hit while pending and
  don't need text at all.
- processCodeOutput non-office branch: explicitly null out status,
  previewError, previewRevision (codex P2). Without this, an update at
  the same (filename, conversationId) where the prior emit was an
  office file leaves stale lifecycle fields and the client renders
  the wrong state for the now non-office artifact.
- Tests: rewire preview.spec mocks for the new shape, add boundary
  test pinning the 2min cutoff, add regression test for the
  cross-shape update.

* 🐛 fix: keep polling on transient errors but cap permanently-broken endpoint

Codex P2: the previous `data?.status === 'pending' ? 2500 : false` gate
killed polling on the first transient error. With `retry: false`, a 500
left `data` undefined, the callback returned false, and the chip was
stuck "Preparing preview…" forever — exactly the bug the polling layer
was supposed to recover from.

Inverts the gate: stop on terminal success (`ready`/`failed`) or after
5 consecutive errors. Transient errors keep retrying; a permanently
broken endpoint caps at ~12.5s instead of polling forever. Predicate
extracted as `previewRefetchInterval` for direct unit testing without
fighting React Query's timer machinery.

* ✨ feat: render pending-preview files in their own row

Pending deferred-preview chips now bucket into a separate row above
the resolved attachments — reads as "this is still happening" rather
than mixing with completed downloads. Once status flips to ready, the
chip re-buckets into panelArtifacts; failed re-buckets into the file
row alongside other downloads.

* 🎨 fix: render pending-preview chips in the panel-artifact row, not the file row

Previous bucketing put pending chips in the file row (since
`artifactTypeForAttachment` returns null for empty-text records). The
pending placeholder is a future panel artifact — sharing the row keeps
the chip in place when it resolves instead of jumping rows.

Plain files still get their own row.

* 🐛 fix: phase-1 SSE replay must not regress a resolved attachment

Codex P1: useEventHandlers.finalHandler iterates
responseMessage.attachments at stream end and dispatches each through
the attachment handler. Those records are the immediate-persist
snapshot (status:pending, text:null) — if a deferred update has
already moved the same file_id to ready/failed, the existing merge
let the pending fields win and downgraded the resolved record. Result:
chip flickers back to pending and polling restarts until the lazy
sweep corrects.

Pin the terminal lifecycle fields (status, text, textFormat,
previewError) when existing is ready/failed and incoming is pending.
Other field updates still go through.

* 🐛 fix: track preview-poll error cap outside React Query state

Codex P2: the previous cap relied on `query.state.fetchFailureCount`,
but React Query v4's reducer resets that to 0 on every fetch dispatch
(the `'fetch'` action). With `retry: false`, each failed poll left
count at 1 and the next dispatch reset it back to 0, so the `>= 5`
branch never fired and a permanently-broken endpoint polled forever.

Track consecutive errors in a module-level Map keyed by file_id,
incremented in a thin `fetchFilePreview` wrapper around the data
service call. The Map is cleared on success and on cap-stop, so
memory is bounded by in-flight pending file_ids per session.
2026-05-06 03:04:19 -04:00