* 📡 fix: Explain Model Streams That Die Mid-Response
* 📡 fix: Bound Model Stream Failures to Response Bodies
---------
Co-authored-by: Lia <lia@librechat.ai>
* 🛬 fix: Settle Background Tool Results Before the Server Shuts Down
* 🛬 fix: Preserve Background Shutdown Results Through Fallback
* 🧪 test: Match Formatted Cluster Shutdown Wiring
---------
Co-authored-by: Lia <lia@librechat.ai>
* 📬 fix: Show Background Results Still Waiting for the Agent
The chat header listed a background task as completed as soon as its tool
finished, while its result could still be queued behind a busy turn and
arrive minutes later as a new agent turn. The header also read only the
process-local registry, so tasks on another replica were invisible.
The task index now reads the durable delivery store the agent's
check_background_task already uses: each finished task carries
delivery pending, delivered or failed, and finished results known only to
the store are listed too. The header marks results still on their way,
keeps them ahead of delivered ones, and polls them every 10 s.
* fix: Retain And Index Background Result Delivery States
* fix: Pass Background Delivery CI Checks
---------
Co-authored-by: Lia <lia@librechat.ai>
* feat: Attach or leave a workspace on a saved chat
A saved chat that recorded running without an attached environment could
never take one: `without_attached` is sealed for the life of the chat, and
the only transition was a move between attached environments. Switching an
existing chat to a coding agent therefore ended in a composer that could
send but never reach files or commands. A chat sealed to a machine that went
offline had the opposite dead end: Send stayed disabled with nothing on
offer but a reconnect hint, and the "Move the chat to continue without it"
copy promised a request the server rejected as `invalid`.
One transition mechanism now covers all three directions. `from` still
repeats the persisted decision, so a stale view cannot replace a decision it
has not seen, and only a decision the chat actually recorded can move:
- attach, from a recorded `without_attached` onto the environments its
agents use, with each target workspace revalidated on its worker;
- detach, an empty `to`, which clears the selections without polling the
machine the chat is leaving;
- move, unchanged, including its refusal to switch the workspace of an
environment the decision already covers.
The composer keeps the control visible while a chat runs without a
workspace, so that state is named and reversible, and offers "Continue
without workspace" whenever a sealed machine stops being usable. History is
untouched and no file is copied.
A send now also records the decision it was submitted under on its own
conversation. The server seals that pair for the rest of the chat, so a chat
that became saved mid-run held none while the stored one was already sealed:
the composer re-derived an agent default the seal refuses and reported
"Choose workspace" with Send disabled until the page was reloaded.
* 🎨 style: Format the Workspace Transition Hook Tests
* fix: Never seal a partial workspace decision, and gate the new directions
Two review findings on the transition mechanism.
A transition replaces the sealed decision whole, and the next turn resolves
every environment or none. The move path could not truncate before, because it
was offered only in `choose` state, which `aggregateState` reaches only when no
environment is unavailable, unsupported or missing: every environment was then
either carried over or selectable. Widening the gate to `!canSubmit` so an
unreachable machine could be left behind broke that invariant, and an
environment nothing can cover would have been dropped from a decision the
composer then sealed, blocking the next send and needing a second transition to
escape. A nonempty transition now requires every environment to be retained or
selectable; when one cannot be, only leaving attached execution is offered,
which is the escape.
The move protocol version becomes 2. Attach and detach travel on the same
owner-authorized endpoint as the move, so a client built against this change
would otherwise offer both to a deployment that only implements moves, where an
attach comes back `locked` and an empty target set `invalid`. The client already
compares the advertised number against the constant, so the bump is the gate.
`conversationMoves.enabled` still decides whether an owner may replace a sealed
decision at all.
* fix: Keep v1 moves working, and take the conversation updater from its owner
Two review findings on the previous head.
Bumping the move protocol wholesale made the client's exact-version check treat
a replica still advertising 1 as supporting nothing, so an attached chat whose
agent moved to another machine lost the relocation that replica already served
-- the previous head's own compatibility test pinned that regression. The
protocol is now two constants: version 1 is the move between attached
environments, version 2 adds attaching an environment to a chat that recorded
running without one and leaving attached execution behind. The server
advertises the highest it implements and the client keeps every lower
capability, so a v1 replica keeps its move while attach and detach wait for the
version that serves them. Where an environment cannot be covered and leaving is
not served either, no transition is offered and the composer reports the state.
Recording the submitted decision had made this hook a second Recoil writer of
the conversation atom, widening the surface that migration has to convert. The
host that owns the atom already holds its setter, so the hook receives it
instead of reaching through `~/store`.
* fix: Advertise attach and detach beside the move instead of above it
Raising the move protocol to 2 kept a client and server of the same build in
agreement, but not the two builds a rollout actually runs at once. The deployed
client compares the advertised number for equality against its own constant of
1, so a server advertising 2 tells every client already loaded in a browser that
moves are unsupported, and an attached chat whose agent changed machines loses
its relocation until that tab reloads.
The move version therefore stays 1, exactly as before this branch, and the two
new directions are advertised on their own field:
codeEnvironmentMoveVersion: 1 // move between attached environments
codeEnvironmentTransitionVersion: 2 // attach an environment, or leave one
Both come from `conversationMoves.enabled`, since all three are one owner
replacing one sealed decision. A client that predates this capability reads only
the move version and keeps working; a client that has it offers attach and
detach only where the second field appears, so it never sends a replica an
attach it refuses as `locked` or an empty target set it calls `invalid`.
* fix: Hold a send while a workspace decision is being replaced
Confirming an attach left Send enabled. The composer disables only the
workspace menu while the request is in flight, so a turn submitted right after
the confirmation started under the decision being replaced: it ran without the
workspace its owner had just chosen, and the replacement still landed, because
the stored decision the compare-and-swap expects was unchanged. The chat then
reported an environment its last turn never used, and nothing said so.
The server's own check has the same shape. It reads the conversation's
generation state, then revalidates each target workspace over the network, then
swaps; a turn that starts during that round trip is invisible to the check that
already ran. It now re-reads the generation state immediately before the swap,
leaving only the instant the compare-and-swap itself covers, which closes the
window for every client rather than just this one.
On the client the submission resolver refuses while a replacement is pending,
so the same signal disables the Send button and makes a programmatic send
return false. The workspace control stays reachable, so the state a chat is in
remains visible while it settles.
* fix: Fence Workspace Transitions and Reconcile Rejected Choices
Advance a private conversation revision after publishing an active generation and before reading its authoritative workspace decision. Transition CAS writes compare that revision, covering admission during the final idle-check window without expiring locks. Preserve tenant middleware and exclude private counter writes from chat activity and indexing.
Default existing undecided chats to no workspace, reconcile failed submissions from the server, scope pending guards to the affected chat, compare selection tuples without delimiter collisions, and opt attach/detach into deployment policy independently of existing moves. Retain legacy selection-only choices through first-send events.
* fix: Retain Failed Workspace Recovery and Skip Disabled Fences
Keep unconfirmed code-environment decisions in conversation-scoped Jotai state after authoritative reads fail, mutation observers reset, or navigation unmounts them. Expose a retry action and release submission only after a matching successful read. Ignore stale completions and reject mismatched conversation responses.
Reuse the owner-scoped conversation already loaded by each ingress when effective conversation moves are disabled. Retain the admitted revision fence for enabled moves, including move-only deployments. Cover failure/retry lifecycle and both policy branches at the helper and ingress levels.
* fix: Reuse the Atomic Admitted Snapshot Through Recovery
Return the admission revision and persisted decision from one Mongo findOneAndUpdate. Skip content indexing per internal query while retaining tenant middleware, private projection, and timestamps. Carry the returned snapshot through web and API initialization instead of validating against a stale pre-admission selection.
Keep first-send workspace choices in both the recovery preset and conversation template, and reconcile using the generated conversation ID from error or message data rather than requiring a saved submission snapshot. Cover single-query storage, indexing isolation, moved-machine validation, and first-send error recovery.
* fix: Reconcile Cancelled Turns and Keep Detach Reachable
Reconcile the submitted code decision before the local cancellation path finalizes streamed content or recovers a minimal submission. A cancel that beats the server rejection must not leave the attempted choice sealed locally; navigation must not update a different conversation.
Offer the authorized detach-only transition when a sealed conversation no longer has any attached agents, while ordinary chat stays sendable. Keep the selector visible for that transition and retain existing policy gates. Cover cancellation branches, no-attached-agent states, managed-only agents and the detach action.
* fix: Recover Workspace Transitions Across Navigation
* test: Verify Saved Workspace Transitions Against Current Dev
* test: Assert Explicit No-Workspace State in Native Journey
* test: Await Browser Settlement Before Switching Native Agents
* test: Settle URL Agent Selection Before Native Chat
* test: Wait for Worker Lease Expiry Before Offline Detach
---------
Co-authored-by: Lia <lia@librechat.ai>
Co-authored-by: Danny Avila <danny@librechat.ai>
* 📮 fix: Count Undelivered Background Results as Outstanding
* fix: Reconcile delivered tasks, discard exactly, and count pending subagent results
* fix: Settle manually claimed remote deliveries and keep pending controls while wake-ups drain
* fix: Report dead-lettered deliveries as failed and read subagent wake-ups from the durable store
* fix: List restored dead letters and retire deliveries on local durable claims
* 🐢 fix: Back Off Waiting Completion Wake-ups and Deliver Them When Ready
* fix: Signal every readiness path and make the completion wait cap configurable
* 🧷 fix: Wait for Long Dispatch Turns Before Giving Up Background Code Results
* fix: Mark held completion deliveries instead of pipeline expedite, and announce store-won approval expiry
* fix: Keep code tasks protected and wait without a fixed cutoff
* fix: Scope settle expedites to the resumed conversation and close the remaining signal gaps
* style: Format the settlement waiter
* fix: Announce a won approval expiry once and release subagent wake-up registrations
* fix: Outlast terminal persistence before giving up an unanchored code result
* fix: Anchor code results the moment their dispatch turn settles
* fix: Scope harvest epoch pinning to the owning stream
---------
Co-authored-by: Lia <lia@librechat.ai>
* 🐢 fix: Back Off Waiting Completion Wake-ups and Deliver Them When Ready
* fix: Signal every readiness path and make the completion wait cap configurable
* fix: Mark held completion deliveries instead of pipeline expedite, and announce store-won approval expiry
* fix: Scope settle expedites to the resumed conversation and close the remaining signal gaps
* fix: Announce a won approval expiry once and release subagent wake-up registrations
* 🧪 ci: Count System Instructions in Mock Model Usage (#16350)
The e2e fake model reported input tokens for the chat messages alone, because the SDK hands a test override model the pruned messages without the systemRunnable pipe. Once #16301 gave every memory-enabled chat a system prompt, the calibrated context snapshot put used tokens below the instructions and the gauge dropped its Messages row. Count input over the complete prompt, as a real provider bills it.
---------
Co-authored-by: Lia <lia@librechat.ai>
* 🔕 perf: Back Off Idle Durable Agent Recovery Polls
* 🎛️ style: Format Queued-Turn Recovery Test
* fix: Preserve Durable Recovery Wakeups and Discovery Invariants
---------
Co-authored-by: Lia <lia@librechat.ai>
* fix: Surface Persistent Memory Before First Save
* fix: Distinguish Unreadable Memory From Empty Memory
---------
Co-authored-by: Lia <lia@librechat.ai>
Update package repository and bugs URLs, GitHub links in docs, templates and comments, and image references (compose files, deployed-update, Helm values and renovate hints) from danny-avila to LibreChat-AI / librechat-ai. Covers the rag-api repository and its images, which moved to the organization. Sponsor links and the sealed star-history chart are unchanged.
* 🛝 refactor: Route Packaged Chat Completion Frames Through a Writer
* style: Match Chat Completion Contract Test Import Order
* fix: Avoid ES2022 Array Methods in Protocol Contract Test
---------
Co-authored-by: Lia <lia@librechat.ai>
* Add client side tools
* 🚦 fix: Ask the Model to Call Caller-Executed Tools Alone
* Add client tool continuation replay tests
* Fix clientTools docstring
* fix: return only tools that have been actually applied
* 🔁 fix: Continue a Caller-Executed Tool Exchange Across Requests
* fix: handle null values for tool descriptions and parameters in client tools
* 🧩 fix: Keep the Arguments of Every Tool Call in a Streamed Message
* Fix PR comments
* fix: Preserve client tool handoff across SDK event ordering
---------
Co-authored-by: Lia <lia@librechat.ai>
* fix: preserve Insights with guarded ACL role updates
* test: settle ACL index builds before opening the transaction
* fix: replace ACL mask partitions with single-operator bit writes
* style: drop the now-unused RoleBits import
* fix: make ACL role replacement one atomic guarded write
* fix: export the permission write retry ceiling
* fix: retry ACL mutations per document and preserve atomic permissions
---------
Co-authored-by: Lia <lia@librechat.ai>
* ✒️ fix: Persist a Generated Title Before the Turn Ends
* 🧷 fix: Keep the Early Title Save From Rewriting Message References
* 🩹 fix: Carry the Recovered User Message Reference
* 🧵 fix: Reference Every Recovered Message Row
* 🪢 fix: Reference Paused and Resumed Response Rows
---------
Co-authored-by: Lia <lia@librechat.ai>
* ⚡ perf: Exclude Agent Version History From Default `getAgent` Reads
getAgent now defaults its projection to { versions: 0 }; loadAgent/loadAddedAgent
and canAccessAgentFromBody use getAgentWithVersionCount so the version count stays
exact without transferring the unbounded versions array on every chat request.
Callers that need history (v1 update/revert handlers) request it explicitly.
* fix: tighten agent version projection types
* fix: type projected agent version counts
* fix: type default agent projection
* style: format agent projection overload
* fix: expose projected agent return type
* 🔑 fix: Stringify Ban Cache Keys When Redis Is Off
Keyv/Mongo throws when checkBan passes a raw ObjectId as the cache key
with Redis disabled, which 500s every social OAuth login.
Fixes#16025
Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com>
* 🔑 fix: Stringify Violation Log Keys and Cover Bans With Real Keyv
logViolation keyed its violation and ban logs by the raw
`req.user.id ?? req.user._id`, the same shape checkBan used, so a lean
user document would hit the same keyv 5.6 `key.startsWith` TypeError on
the namespaced violation stores and record its ban under an ObjectId.
The new spec drives checkBan and logViolation through real namespaced
Keyv instances (Mongo-backed ban store, in-memory violation logs) with
an ObjectId-only OAuth user: an unbanned user passes, and a ban logged
from one address is enforced for the same user from another.
* 🎯 refactor: Stringify Only the ObjectId Fallback for Ban Keys
The id is converted once, where a non-string can enter: the lean
document's `_id` fallback. `req.user.id` is already a string on every
path that sets it (the JWT strategies, Mongoose's `id` virtual, the 2FA
limiter's temp-token id), and the email lookup already stringifies, so
`getBanCacheKey` and that line go back to their dev form.
---------
Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com>
Co-authored-by: Danny Avila <danny@librechat.ai>
* 🗂️ fix: Keep a Running Attachment Chat in the Sidebar
Since #15694, a turn with model-bound attachments defers its user-message
write until the model admits the payload. That write was also what created
a new conversation's row, so for the whole first model call, which with a
long generated script or extended thinking can run for minutes, the
conversation did not exist. Any conversation-list refetch in that window
(starting another chat, deleting one, reloading) dropped the running chat
from the sidebar until its run finished, and GET /api/convos/:id answered
404 for it.
When attachments are the only reason to defer, the new conversation's row
is now written as the run starts, carrying the same fields the message
write would set but no message. The message itself stays deferred until
admission, and its write waits for a seed still in flight so the two
upserts never race. A content policy still holds back every write, the
row included, as #14425 intended.
The conversation half of saveMessageToDatabase moves into
saveConversationToDatabase so the seed and the message write share one
path.
* 🧱 refactor: Own Turn Conversation Writes in packages/api
The conversation half of a turn's writes (the row a message save
upserts, the retention context it stamps, and the new seed ahead of a
deferred first message) moves into `conversations/save.ts`, which takes
`getConvo`/`saveConvo` from the caller. BaseClient keeps only the
wiring: the retention lookup that must stay synchronous for Stop, the
message save, and the calls into the module.
The seed passes an empty `appendMessageIds`, which tells `saveConvo`
the row holds no messages yet, so it no longer reads the message list
before the upsert. It also resolves the conversation once and leaves it
on the request, so the deferred message save reuses it instead of
looking the conversation up again.
save.spec.ts drives the real data-schemas methods against an in-memory
MongoDB: the seed creates an empty row without reading messages, the
message save appends to it without a second lookup, an existing chat and
a subagent thread are left alone, a temporary chat keeps its retention,
and a failed lookup settles instead of rejecting.
* fix: Stop Sealing a Code Environment Decision a Chat Never Made
Switching an existing chat to a coding agent left the composer unable to
send. The chat had no workspace to select, Send stayed disabled, and the
workspace chip was a read-only status with a recovery hint.
Both layers treated "no decision recorded" as "decided to run without an
attached environment". `readPersistedDecision` documented that a stored
conversation always carries a decision and inferred `without_attached`
whenever `codeEnvironmentMode` and `codeWorkspaces` were both absent,
which is the ordinary state of a chat whose turns never involved a
code-capable agent. `resolveConversationCodeEnvironmentDecision` then
sealed that inference, so the first workspace its owner picked came back
as `locked`. The client mirrored the same premise: `locked` was any saved
chat, so `useCodeWorkspace` withheld agent defaults, a remembered
selection, and a sole workspace, and `CodeWorkspaceMenu` rendered its
non-interactive locked branch.
Only a recorded decision is sealed now. A conversation holding neither
field is undecided, so its first coding turn establishes the decision,
and `resolvePersistableCodeEnvironmentDecision` records that decision
with its selections; writing the mode alone would leave `attached`
without the selections the next turn validates. On the client the same
predicate drives `locked`, and the selection defaults that already
applied to a new chat now apply to any chat that has not decided.
An explicit `without_attached` stays sealed, a legacy row still infers
`attached` from the selections it stores, and a sealed decision is still
replaced only by its owner's move.
* fix: Record the first code environment decision on an existing chat
A saved chat that holds no decision may record the one a run establishes,
but every caller persists through `saveConvo`, which carries those fields
only in `$setOnInsert`. On a row that already exists the write was dropped,
so the choice never survived a reload and a later turn could resolve a
different workspace.
`saveConvo` now fills a whole decision onto a row that holds none, by
compare-and-set on that absence: a writer that decided first keeps its
decision, a chat that already holds one is untouched, and bare selections
never seed a row.
* fix: Record the decision a failed turn of a saved chat ran under
`saveErrorTurn` built its conversation fields only when it was seeding a new
row, so an existing chat kept the stored error turn without the decision that
turn ran under: a retry could then pick a different workspace than the failure
already recorded. The validated decision now travels on both branches, where
the conditional write in `saveConvo` keeps it from replacing a stored one.
* refactor: Resolve the error turn's decision fields in `packages/api`
The error-turn save derived its own decision fields, putting persistence
policy in a CJS controller where `/api` is meant to hold wiring only. It now
calls `resolvePersistableCodeEnvironmentDecision`, the resolver the streaming
saves already use, so one rule decides what a turn may record: a chat that
holds a decision keeps it and receives no decision fields at all.
* fix: Keep the legacy code environment lock until the protocol is advertised
The composer unlocked a saved chat that holds no decision without consulting
`codeEnvironmentDecisionVersion`, so during the rollout window it could attach
an agent default and submit it to a replica that still reads a field-less row
as a sealed `without_attached` and rejects the turn as `locked`.
The unlock now waits for the deployment to advertise the protocol, which makes
the hook identical to its previous behavior while the flag is unset. Nothing is
lost by waiting: the unmade decision is only reported once that flag is on.
---------
Co-authored-by: Lia <lia@librechat.ai>
* 🫗 fix: Roll Back Incomplete Skill Archive Imports
A .zip or .skill import created the skill first and persisted its bundled
files one by one, keeping the skill and answering 201 when a file failed
validation, a size limit, storage, or its database row. The client never
read the internal `_importSummary`, so it showed "Skill created" and
navigated to a skill that silently lacked the resources SKILL.md names.
Import is now atomic: any failed bundled file rolls back the skill, its
file rows, the ACL grant, and the blobs already written to storage, and
the request answers 422 `skill_import_incomplete` with the failed paths
and reasons. The upload dialog stays open and lists them.
* fix: Localize Import Failure Reasons and Report Every Lost File
Addresses the first review round on the archive import rollback.
Failure reasons are now stable codes (`invalid_path`, `file_too_large`,
`archive_too_large`, `archive_entry_changed`, `persistence_failed`) that
the client localizes, so a non-English locale no longer sees English text
and storage or driver messages stay in the server log.
The decompression budget stops the scan mid-archive, and the entries it
never reached were missing from the response, telling the user they had
seen the whole failure set. Those paths are now reported too.
Rollback deletes stored blobs only once `deleteSkill` confirms the skill
is gone; otherwise the blobs stay and the request answers 500
`skill_import_rollback_failed`, because a surviving skill pointing at
deleted files is worse than unreferenced storage.
The failure list scrolls instead of overflowing the dialog, and a
response that resolves after the dialog was dismissed no longer
repopulates it for the next session.
* fix: Ignore Late Skill Import Successes
* fix: Make Skill Import Rollback Retryable
* fix: Propagate Incomplete Skill Cleanup
* fix: Settle Rollback Blob Cleanup
* test: Type Partial Cleanup Fixture
* fix: Preserve Settled Skill Cleanup Steps
* fix: Export Skill Cleanup Step Type
* fix: Reconcile Partially Deleted Skills
* fix: Reconcile Skill Rollback Caches
* style: format cleanup reconciliation changes
* fix: complete skill rollback cleanup
* chore: leave generated locale catalogs unchanged
* fix: preserve synced skill blob cleanup
* test: type synced blob cleanup mock
* fix: Validate Archive Paths and Bound Rollback Cleanup
---------
Co-authored-by: Lia <lia@librechat.ai>
Co-authored-by: Danny Avila <danny@librechat.ai>
* ⏳ fix: Keep BYOM tools queued across capacity windows
* ⏳ fix: Configure the admission budget and refresh queued credentials
Addresses the Codex review on #16060:
- Add `limits.maxQueueWaitMs` to the code environment config schema so a
deployment can shorten or disable the five-minute admission budget from
`librechat.yaml`; omission keeps the merged behavior, `0` surfaces the
first capacity expiry to the model. Resolved through
`resolveAttachedWorkspaceQueueWaitMs` and passed by ToolService, matching
how `maxCommandTimeoutMs` already flows.
- Accept an auth-header supplier in `executeWorkspaceTool` and mint it per
admission attempt, so a call queued past the Code API token TTL presents
fresh credentials instead of failing permanently with 401. The attached
Bash tool and the workspace file operations now pass suppliers.
- Recheck the deadline after a clamped wait and throw the saved rejection
rather than opening an admission window that could still be admitted
once the budget is spent.
* fix: Complete workspace admission wiring and caller regressions
* fix: Share protected edit retry budgets and schema limits
---------
Co-authored-by: Lia <lia@librechat.ai>
The agent tool-resource branch of DELETE /files removed a file's association
from the agent and answered 200 without reaching processDeleteRequest, so the
bytes on disk and the RAG chunks of a file the caller owned survived the delete.
A replaced File Search document kept answering from its old chunks (#15930).
An attached file now goes through the full delete pass, but only when nothing
else holds it: a file the caller does not own, and a file any other
(agent, tool_resource) pair still references, are unlinked and left whole.
Nothing is unlinked until the destroy it depends on has succeeded.
- packages/api/src/files/deletion.ts: deleteAgentResourceFiles owns the
operation with its db methods and delete pass injected.
- packages/data-schemas: getSharedResourceFileIds answers the last-reference
question, excluding the pair being removed by the agent's _id.
- Local/crud.js: unlinkFile propagates non-ENOENT, so a file whose bytes
survive is no longer reported as deleted.
Co-authored-by: Mohammed Alshyakh <mohmedmm@users.noreply.github.com>
* fix: Resolve live OpenID bearer before the MCP early domain gate
* fix: Drop unread credentials from the MCP early domain gate
The early domain gate resolves the whole server config through
`processMCPEnv`, but decides from the URL alone. A credential placeholder in
any other field therefore raised `OpenIDReauthRequiredError` from a stale
request-time OpenID snapshot, before the connection path could refresh that
bearer, and the tool was dropped from the agent's toolset.
`buildMCPDomainValidationConfig` narrows the config to what the decision reads,
so the gate needs no live credential. A URL placeholder still fails closed, and
the argument is never mutated, so direct-bearer recovery keeps the placeholder
it knows how to refresh.
Replaces the per-tool `upstreamTokenProvider` call the gate would otherwise
make (1 + N per request per server) and covers every credential-bearing field,
not only `Authorization`.
Co-authored-by: Artyom Bogachenko <SpectralOne@users.noreply.github.com>
---------
Co-authored-by: Lia <lia@librechat.ai>
Co-authored-by: Artyom Bogachenko <SpectralOne@users.noreply.github.com>
* fix: Serialize MCP OAuth token refresh across replicas
`MCPTokenStorage.inflightRefreshes` coalesces refresh-token redemptions
inside one Node process. Behind a load balancer without session affinity,
one user's concurrent requests land on different replicas, each reads the
same not-yet-rotated refresh token and redeems it. RFC 9700 servers treat
the second redemption as replay and revoke the whole grant family, so the
user is asked to authorize the MCP server again.
`forceRefreshTokens` now takes a cross-replica refresh flight before
redeeming, using the same `FlowStateManager.acquireLease` primitive the
OAuth teardown fence uses under a distinct key. A replica that waited for
the flight adopts the tokens the holder rotated instead of redeeming
again; when the flight is still held after the wait window it falls back
to the unfenced redemption every earlier release performed.
* fix: Redeem unfenced when the refresh flight lease store fails
* fix: Fence MCP refresh waiters instead of redeeming unfenced
A waiter that could not take the cross-replica refresh flight fell through to
an unfenced redemption after 10s, while the holder's own stale abort does not
fire until 60s. A refresh taking 11 to 60 seconds therefore still let two
replicas redeem one refresh token, the replay this fence exists to prevent.
The waiter now polls until it either adopts the tokens the holder rotated or
acquires the flight, and fails as MCPTokenRefreshUnavailableError rather than
redeeming beside a live holder. `getTokens` callers defer connection recovery
on that error, and the stored credential is left intact for a later attempt.
The flight is keyed by the stored credential (tenant, user, server name)
rather than the caller's OAuth binding digest, so a rolling config change
cannot hand two replicas different locks over one stored token. The wait is
an operator lever, `oauthRefreshWaitTimeout`, clamped to half the stale
window. An adoption read that fails no longer gives up the held flight.
* fix: Annotate derived refresh-flight constants for isolatedDeclarations
* fix: Recapture the credential generation a peer published on adoption
* fix: Hold the MCP refresh flight until redemption settles
Four corrections to the cross-replica refresh flight.
The flight lease equalled the window that aborts a stalled redemption, on
the claim that an expired flight could never belong to a redemption still
able to reach the token endpoint. Aborting proves no such thing: the
request may have been processed with its response lost, and a stalled
event loop can delay the abort past its own deadline. The lease now
outlives the abort, so a peer cannot redeem a credential the provider has
already rotated. A live replica still releases on settle, so the margin is
paid only by one that died holding the flight.
The credential snapshot was taken after the first failed acquisition, so a
holder that stored and released in that gap was snapshotted post-rotation:
the next attempt saw an unchanged record and redeemed the credential it
should have adopted. It is taken before the first attempt now.
`oauthRefreshWaitTimeout` accepted zero while the runtime mapped every
non-positive value to the default, so the config validated and then
behaved contrary to its value. Zero is rejected at load.
Adoption announced a second credential change through
`handleOAuthRefreshSuccess`, whose `onOAuthCredentialsChanged` advances
authorization state after persistence this replica did not perform. That
retired the generation recaptured beside it and fenced the build against
its own tool publication. Adoption now updates the local token-flow cache
and recaptures, without announcing.
* fix: Check every held MCP refresh flight for a peer's rotation
Four corrections, two of them consequences of moving the credential
observation ahead of the first lease attempt.
That move created a storage read inside `beginRefreshFlight`, and its
failure reached a handler written for a lease-store outage, so a transient
read error became an unfenced redemption beside a live peer. The flight
now handles its own reads: losing the observation costs adoption and
nothing else, and only the lease store failing may redeem unfenced.
The observation was also never compared when the first acquisition
succeeded. A peer that rotated and released before this replica contended
left an acquisition that looked uncontended, and its fresh credential was
redeemed a second time. Every acquisition now runs one rotation check,
which also collapses two code paths into one.
`getTokens` already loads the refresh record to decide a refresh is
needed, so it is passed on as the observation baseline instead of read
again, and the read taken under the flight is reused as the credential
redeemed. Two reads on a latency-counted path where there were three.
`runSilentRefresh` collapsed contention into null, sending the 401 path to
interactive OAuth, so a slow peer prompted the user to authorize a server
whose credential was about to be valid. The retryable outcome now travels
through the silent-refresh layers and the connection defers, matched by
name as well as identity because these errors cross the package boundary.
* chore: Regenerate the Agents OpenAPI spec for repositoryInstructions
The drift check added by #15928 fails on dev: #16008 added
`repositoryInstructions` to the agent schema without regenerating the
committed spec, so every branch merging dev inherits the failure. This is
the remedy the check itself prescribes and carries no hand-written change.
Drop this commit if dev regenerates first.
* fix: preserve MCP refresh outcomes across replica boundaries
* fix: keep MCP adoption fenced through publication
* fix: anchor MCP refresh adoption to rejected credentials
* fix: Complete OAuth Adoption Adapters and Isolate Legacy Flow Readers
* fix: Preserve OAuth Request Identity Through Recovery and Discovery
* fix: Fence OAuth Adoption Against All Credential Writers
* fix: Gate Coordinated OAuth Rollout and Preserve Unauthenticated Identity
* fix: Preserve OAuth Coalescing Across the Staged Rollout
* style: Sort OAuth Timeout Constant Import
* fix: Invalidate Both OAuth Token Flow Protocols on Rotation
* test: Complete the Rollback Token Flow Fixture
---------
Co-authored-by: Lia <lia@librechat.ai>
Co-authored-by: Danny Avila <danny@librechat.ai>
* 🗣️ fix: Keep Provider Error Text on Unclassified Agent Failures
An upstream failure LangChain does not classify discarded the provider's own
message and answered with the generic upstream sentence plus a status, so a
gateway or privacy-proxy rejection lost the only account of what happened.
The provider text now rides along in the persisted payload as `message`,
withheld only where a content policy inspects the traffic — the same condition
`getUserFacingProviderError` and `getUserFacingRequestError` already decide by.
* fix: Harden Provider Error Detail Retention
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
* 📭 fix: Deliver Attachment Text Until a File Tool Holds It
A `none`-routed upload had its extracted text withheld whenever a file tool was
enabled, on the assumption the tool would serve the file. A plain chat with the
File Search toggle runs an ephemeral agent, so the upload is filed under no tool
resource and never embedded: the text was withheld for a vector store that never
received the file, and the attachment reached nothing.
Withholding now requires the record to show the tool holds the file, which is the
same evidence deferred provisioning reads before queueing it. `resources.ts`
delegates to that shared predicate so the two cannot disagree again.
* test: Give the Code-Running Primary Agent a Sandbox Copy
---------
Co-authored-by: Lia <lia@librechat.ai>
* 🔌 feat: OpenAPI spec and docs for the public Agents API
* 🔧 fix: address review on Agents API OpenAPI spec and docs:
- adapter: mark JSON request bodies as required
- openapi.js: move gate, config read and spec serving into packages/api
as an injected router factory; leave /api as wiring only
- docs: compute URLs in-browser and use a relative servers url so docs
work under a base path (e.g. /chat)
- build: copy the generated spec into dist so the multi-stage image ships it
- agents: reuse the enforced list envelope for the docs so the AgentList
schema cannot drift from the runtime cursor and strictness
* 🔧 fix: align Agents API OpenAPI contract with enforced runtime
Second review round on the public Agents API spec. Three cases where the
published contract described something the server does not do, so generated
clients would fail against the live API:
- Upload form: document the `purpose` field (enum: file_search, execute_code,
context) that POST /agents/{id}/files actually requires, instead of the
internal-only `tool_resource`; requests following the old form were rejected
with 400 because `purpose` was absent.
- Auth: drop the `apiKeyBearer` scheme from the management and skill endpoints.
They gate on OIDC/M2M via requireAgentManagementAuth and deliberately reject
the API-key fallback; the API key belongs to the out-of-scope
OpenAI-compatible endpoints and returns when those are documented.
- Skill update: document the 409 conflict PATCH /skills/{id} returns on a stale
expectedVersion. The agent update path collapses its internal 409 to 400, so
it correctly keeps no 409.
* 🔧 fix: serve docs on experimental server and document the 401
Third review round on the public Agents API spec.
- Experimental server: mount routes.openapi in api/server/experimental.js before its /api 404 handler. Without it, npm run backend:experimental returns 404 for /api/docs and /api/openapi.json even when openapi.enabled is true, because that server never mounted the router.
- Auth failure: document the 401 that requireAgentManagementAuth returns on a missing, expired, malformed, or unbound token. Its body is { "error": "Unauthorized" }, a flatter shape than the management error schema, so add a dedicated UnauthorizedError schema and a shared 401 response to the agent and skill contracts rather than reusing the standard error schema.
* 🔧 fix: document skill-update, auth-conflict, upload-limit, and watch-copy gaps
* 🔧 fix: added error shapes for permission denied and internal server errors in API responses docs
* fix: cover the auth-layer 500 shape and pin the account-deletion code
* fix: update OpenApiRouterDeps documentation and simplify request handling in isEnabled function
* 🔧 fix: gate the Swagger asset mount behind the OpenAPI toggle
* 🔧 fix: accept the malformed-JSON 400 envelope in the contract
* fix: accept the tenant-isolation 403 envelope in the contract
* fix: add description for content property in SkillFileUpdateRequest to clarify byte limit