* chore: add @langchain/langgraph-checkpoint-mongodb for HITL durable resume
* feat: HITL tool approval runtime — backend (Slice B)
- endpoints.agents.checkpointer config + durable Mongo checkpointer (seam over the app
connection; SDK MemorySaver fallback) with a TTL index + deleteThread pruning
- HITL run wiring (PreToolUse policy hook + humanInTheLoop) attached in createRun, fully
inert when toolApproval.enabled is off
- interrupt gate (pause job -> requires_action + emit on_pending_action) and a resume
route that rebuilds the run from the durable checkpoint and run.resume()s it
- atomic single-winner resolve; agent-consistency guard; expireStaleApprovals terminal
event; checkpoint pruned on every non-paused completion (thread_id == conversationId)
* feat: HITL tool approval UI — frontend (Slice B)
approve/reject/edit/respond + ask-user controls in the tool card (OAuth-button precedent),
batch-aware single submit, live + reconnect (resumeState.pendingAction) wiring, and resume
mutations posting to /agents/chat/resume.
* fix(hitl): decouple ApprovalProvider from chat context
ApprovalProvider is now pure state (safe to mount in provider-less / shared / test
renders); the context-dependent submit moved to a useResumeSubmit hook the cards call.
Part imports getAskUserQuestionPart from ~/utils/approval directly so suites that
partial-mock ~/utils render Part without throwing.
* fix(hitl): address Codex review — backend
- P1: enforce per-tool allowed_decisions on resume (reject a crafted decision the
policy disallows) via findDisallowedDecisions
- prune the durable checkpoint on user-abort of a paused run, and before a fresh
HITL turn, so a new turn cannot rehydrate an expired/aborted interrupt (thread_id
is the stable conversationId)
- persist + use isTemporary and the original parentMessageId on resume (temporary
chats stay temporary; initializeAgent scopes thread files off the right parent)
- generate a deferred first-turn title BEFORE completeJob so its event reaches the
client and the final event carries the real title
- moderateText: skip when there is no text (tool-approval resume) and moderate the
ask-user answer, instead of denying on an empty input
* fix(hitl): address Codex review — frontend
- render ToolApproval for ANY paused agent tool card (bash/code/file/etc.), not just
the generic ToolCall, by wrapping the tool-card branch in Part (moved the rendering
out of ToolCall)
- findPendingActionMessageIndex only matches an assistant message, never the user
message (the underscore-strip could target the user bubble before the assistant
placeholder exists)
* fix(hitl): address Codex re-review
- title eligibility checks the user message’s parent (first turn), not the response’s
parent — the previous check could never be true and skipped title generation
- use client.buildResponseMetadata() for the resumed message so contextUsage /
thoughtSignatures survive (the abort-only helper dropped them)
- moderate decisions[].responseText (the respond action’s user text)
- give /chat/abort req.config (configMiddleware) so the HITL checkpoint prune on abort
actually runs
- read resume state BEFORE setContentParts so the in-memory store does not lose the
pre-pause seed content
- count resumes against LIMIT_CONCURRENT_MESSAGES (increment/decrement) so paused-then-
resumed turns cannot bypass the limit
- require actionId on resume so a body without it cannot resolve the current action
* fix(hitl): address Codex re-review (round 3) — resume fidelity
Bring the lean resume path to parity with sendMessage for things it bypassed:
- carry userMCPAuthMap into the rebuilt run so approved MCP tools keep the user's creds
- seed initialSessions (buildInitialToolSessions) so approved code/file/skill tools have
the pre-pause uploaded-file context (esp. cross-replica / after restart)
- await client.artifactPromises and persist them as response attachments (else tool
artifacts created after the pause vanish on reload / for late subscribers)
- merge metadata: cumulative usage (+ summary marker) from the job, contextUsage /
thoughtSignatures from the client — fixes the round-2 regression that underreported
post-resume cost
* fix(hitl): address Codex re-review (round 4) — resume hardening
- resume: require an EXACT paused agent_id match (reject omitted/ephemeral
agent_id, not just a different one) and reject an endpoint mismatch, so a
request can't rebuild the claimed checkpoint on a different graph
- moderateText: also moderate a tool-approval decision's reject `reason` and
stringified `editedArguments`, not just `responseText`
- request: re-mark the paused response `unfinished:true` after BaseClient saves
it as completed, so an expired / never-resumed approval doesn't leave a
"finished" response in history; the resume path overwrites it on success
* test(hitl): route-level integration test for the resume controller
Adds api/server/controllers/agents/__tests__/resume.spec.js, a supertest
integration test that drives the real ResumeAgentController over the full
pause -> approve -> resume -> finalize lifecycle with the SDK run, durable
checkpointer, Mongo, and concurrency cache mocked. The pure decision/liveness
helpers run for real via requireActual, so the guard ladder is exercised end to
end rather than stubbed.
25 cases covering:
- the authorization / staleness / agent-and-endpoint / actionId guard ladder
- tool_approval validation (undecided tool call, policy-disallowed decision)
- ask_user_question answer requirement
- the concurrency gate (429) and the atomic single-winner claim (409)
- the happy path: ACK, run reconstruction, decision->SDK mapping, finalize
(save the now-finished response, emit done, complete job, prune checkpoint)
- first-turn title generation before stream completion
- re-pause (no double finalize), abort-during-resume (no double finalize),
and the resume-failure terminal path (emitError + completeJob + prune)
* test(hitl): strengthen resume coverage + add approval util tests
Acts on a self-audit of the new resume integration test.
resume.spec.js (25 -> 32 cases):
- replace the tautological emitDone assertion (it only checked the hardcoded
`final: true`) with a structural check of the finalEvent payload —
responseMessage content/id/unfinished, requestMessage identity, title
- cover the previously-unwalked finalize branches: tool-artifact attachments
(null-filtered), the aggregatedContent fallback when live content is empty,
and client response-metadata attachment
- add guard cases: unsupported pending-action type (400) and the
pre-multi-tenancy null-tenantId pass-through (must not 403)
- add error-path cases: first-turn title generation throwing must still
finalize, and a completeJob failure during a resume error must force a
terminal job state via the last-resort updateJob
client/src/utils/approval.spec.ts (new, 15 cases):
- applyPendingAction tool_approval: join by tool_call_id not position,
skip completed calls, default allowed_decisions to [], referential
stability when nothing changes
- applyPendingAction ask_user_question: append, idempotent replace on replay,
non-array content coercion
- getAskUserQuestionPart type guard; findPendingActionMessageIndex
assistant-only resolution (never resolves to the user bubble)
* fix(hitl): address Codex re-review (round 5)
Five findings verified against the code before fixing:
- resume: require an EXACT endpoint match (like agent_id) — a resume that OMITS
endpoint must not fall through, since the shared chat middleware treats a
missing/non-agents endpoint as the ephemeral agent and could rebuild the
claimed checkpoint on a different graph
- resume: filter malformed content parts before saving the finished response,
matching the normal AgentClient path (a resumed turn could otherwise persist
an empty/invalid tool_call part that breaks reload/rendering)
- resume: accumulate tool artifacts across pause segments — persist them on
re-pause and MERGE (not overwrite) at finalize, so artifacts produced before
a second approval pause aren't dropped by the next rebuilt client
- approval (client): findPendingActionMessageIndex returns -1 when a provided
responseMessageId isn't found, so the caller retries instead of attaching the
prompt/approval to a prior assistant reply; fall back to the last assistant
only when no responseMessageId is given
- RedisJobStore: make appendChunk extend-only (XADD + EXPIRE-if-shorter via a
single eval) so the on_pending_action chunk emitted after a pause can't reset
the chunk-stream TTL back to the running window and evict pre-pause content
before the approval is resolved
Tests: +endpoint-omitted/unsupported-type/malformed-filter/attachment-merge/
re-pause-persist cases in resume.spec.js (36); ask-retry -1 semantics in
approval.spec.ts (16); extend-only TTL assertion in the RedisJobStore Redis
integration spec.
* test(hitl): mongodb-memory-server integration test for the checkpointer seam
The checkpointer unit spec covers config/selection with no DB connection; this
exercises the durable Mongo seam against a real (in-memory) MongoDB — the part
correctness actually depends on:
- getAgentCheckpointer builds a real MongoDBSaver when Mongo is connected and
setup() creates the TTL index (expireAfterSeconds) on the checkpoint collection
- memory type returns undefined (SDK MemorySaver fallback) even when connected
- saver is memoized per resolved config
- deleteAgentCheckpoint prunes a thread's persisted checkpoint (the cross-turn
isolation guarantee: turn N+1 on the same conversationId can't rehydrate it)
- pruning is thread-scoped — deleting one conversation leaves others intact
- undefined threadId is a no-op
* fix(hitl): address Codex re-review (round 6)
Four findings verified against the code before fixing:
- messageFilterPii: scan the resume payload's user-authored text (ask-user
`answer`, and a tool-approval decision's `respond` text, `reject` reason, and
edited tool arguments) — the shared /resume route ran through the PII filter
but it only inspected req.body.text, so a blocked token rode the resume
payload back into the model/tool (mirrors the earlier moderateText fix)
- resume: re-prime skill files invoked in the pre-pause segment before rebuilding
the run, so an approved code/file-backed tool keeps the injected skill-file
session refs instead of running without them (mirrors the normal path's
primeInvokedSkills; the pre-pause content stands in for the message payload)
- hitl: pin the graph identity. Persist a fingerprint of the graph-determining
request fields (endpoint, agent_id, model, spec, ephemeralAgent — normalized)
on the pending action at pause, and reject a resume whose recomputed
fingerprint differs. This closes the ephemeral-agent gap, where agent_id is
undefined so the id guard can't tell two ephemeral configs apart
- resume: reject incomplete edit/respond decisions (findIncompleteDecisions) —
an `edit` without an object editedArguments or a `respond` without non-empty
responseText is 400'd before mapping, rather than defaulting to {} / '' and
resuming with behavior the user never approved
Tests: incomplete-decision + fingerprint match/mismatch cases in resume.spec.js
(41); findIncompleteDecisions + computeAgentRequestFingerprint unit tests; and
resume-field PII cases in messageFilterPii.spec.ts.
* fix(hitl): address Codex re-review (round 7)
Four findings verified against the code before fixing:
- RedisJobStore: clear `agent_id` on createJob (add it to staleHitlFields). The
job hash is keyed by conversationId and reused across turns; updateMetadata
only writes agent_id when truthy, so a conversation that switched from a saved
agent to an ephemeral/no-agent turn kept the old id and the resume guard
rejected the valid pause as a different agent. (real correctness bug)
- fingerprint: include `promptPrefix` in computeAgentRequestFingerprint, and
re-send it on resume (ResumeAgentFields + buildResumeFields). Ephemeral agents
derive their system instructions from promptPrefix, so a resume changing it
previously passed the pin and rebuilt different instructions. (completes the
round-6 fingerprint)
- resume: the re-pause branch now persists the segment's accumulated CONTENT
(filtered), not just artifacts, so an approval that expires/reaps without a
final resume no longer loses everything streamed during the resumed segment.
- request: carry `manualSkills`/`alwaysAppliedSkills` on the persisted user
message so a resumed turn's reconstructed requestMessage keeps its skill pills
instead of dropping them until a full reload.
Deferred (narrow, no safe contained fix yet — see PR thread replies):
- resume rebuild without `addedConvo` for a multi-conversation/added-agent pane
- cross-replica re-prime of manually-selected (not model-invoked) skill files
Tests: stale-agent createJob clearing (Redis integration), promptPrefix
fingerprint match/mismatch (resume.spec.js + policy.spec.ts), re-pause content
persistence (resume.spec.js).
* fix(hitl): address Codex re-review (round 8)
Five findings verified against the code before fixing; the headline is a durable-
resume correctness fix (the fingerprint had surfaced it as a 403):
- resume durability (the important one): persist the graph-determining request
fields (endpoint, agent_id, model, spec, promptPrefix, ephemeralAgent) on the
pending action as `resumeContext`, and REPLAY them onto the resume request via
a router-level middleware that runs before buildEndpointOption. The client
can't reconstruct the ephemeral-agent config after a reload/cross-session, so
the round-6/7 fingerprint would 403 a valid durable resume — and even without
it the rebuilt agent would lose its tools. Replaying server-side rebuilds the
SAME graph regardless of client state (and a crafted resume can't swap it; the
fingerprint still matches because the body is restored first).
- RedisJobStore: also clear `isTemporary` on createJob (same class as agent_id):
a prior temporary turn's flag would otherwise survive a reused conversation
hash and a later non-temporary resume would save its response as temporary.
- resume: persist `contextMeta` (context-window calibration) onto the saved
response like BaseClient does, so the next turn can seed its pruner.
- request: carry manualSkills/alwaysAppliedSkills into the onStart metadata
update (not just the preliminary one it overwrites), so a resumed turn's
requestMessage keeps its skill pills.
Deferred (narrow — see thread reply):
- saved-agent edited WHILE a run is paused: agent_id matches but the definition
changed; needs an agent version/config hash, which is a larger change for a
narrow window.
Tests: resumeContext pick/apply + round-trip (policy.spec.ts), contextMeta +
manualSkills-on-requestMessage (resume.spec.js), isTemporary clearing (Redis
integration).
* style(hitl): prettier line-wrap in policy.spec.ts (R8 lint fix)
* fix(hitl): address Codex re-review (round 9)
Five findings, all fixed (addedConvo — deferred in rounds 7/8 — is now trivial
thanks to the round-8 replay):
- replay addedConvo: add it to RESUME_CONTEXT_KEYS so the resume middleware
restores the parallel/secondary-agent config from the paused request; the
client can't reconstruct it, and it determines the rebuilt graph.
- skill pills (the real fix this time): the round-8 onStart metadata write was
overwritten by trackUserMessage (the authoritative userMessage writer). Carry
manualSkills/alwaysAppliedSkills in the emitted `created` message and persist
them in trackUserMessage; widen UserMessageMeta + SerializableJobData.userMessage.
- execute-code files on resume: seed the paused user message's own files onto
req.body.files before initializeClient — they're excluded from the
parent-walk code-session rebuild, so an approved code/read-file tool would
otherwise resume without them.
- in-memory pending-action UI: route ApprovalEvents.ON_PENDING_ACTION in the
resume replay/pending-event loops to applyPendingActionToMessages (mirror the
live handler), so a pause that lands in the snapshot window still renders its
approval controls instead of sitting paused with no UI.
- abort isTemporary: the /chat/abort partial-save now sources isTemporary from
the job metadata, not req.body (the stop button posts only conversationId), so
aborting a paused temporary chat no longer persists an orphaned partial.
Tests: addedConvo in pickResumeContext (policy.spec.ts), file-restore on resume
(resume.spec.js), abort-from-job-isTemporary (abort.spec.js).
* fix(hitl): address Codex re-review (round 10) — resume/expiry races
Three concurrency/coherence findings, verified against the code before fixing:
- expiry-sweep CAS scope: both stale-approval sweeps (GenerationJobManager
expireStaleApprovals and the RedisJobStore requires_action cleanup) called
expire()/transitionStatus WITHOUT the observed pendingAction.actionId, so the
CAS only checked status===requires_action. Between the read and the CAS a user
could resolve the observed action and the run re-pause on a FRESH action; the
stale sweep would then abort that valid new pause. Now both pass the observed
actionId as expectActionId, so the CAS only fires for the action read as stale
(a re-paused action has a different id → no-op).
- resume graph cache: resumeCompletion cached the rebuilt graph (created with
messages:[]) via setGraph; RedisJobStore.getContentParts prefers a cached
graph over reconstructing from the chunk log, so a same-replica reload/status
poll mid-resume returned aggregatedContent missing the pre-pause content. Skip
setGraph on resume so introspection falls back to the complete chunk
reconstruction (setContentParts still seeds the in-memory store).
- pending-action UI: applyPendingActionToMessages scheduled a SINGLE
animation-frame retry then dropped the pending action; Recoil/React updates can
take several frames under load, leaving a valid requires_action run with no
approval controls. Retry across frames (bounded at 120) until the target
message commits.
Test: expire() with a mismatched expectedActionId no-ops while the matching id
expires (pendingAction.spec.ts).
* chore(deps): update @librechat/agents to version 3.2.53 and @langchain/langgraph to version 1.4.7 in package-lock.json and related package.json files
* refactor(hitl): add resolveToolApprovalPolicy seam for layered policy
Extract the single point where tool-approval policy is resolved for a turn
(`resolveToolApprovalPolicy`) and route the run call site through it instead
of reading `endpoints.agents.toolApproval` inline.
Behaviour-preserving: only the `endpoint` layer is wired today, so the result
is identical to reading the app policy directly. The `agent` and `skills`
layers are reserved seams with documented precedence (endpoint owns the
`enabled` kill switch; agent overrides mode/allow/deny/ask/reason; skills may
only tighten), so future per-agent and per-skill policy plumbing lands in one
function rather than at the `createRun` site. Adds focused unit tests.
* fix(hitl): address Codex re-review (round 11) — resume hardening
F1 (P2, security) — applyResumeContext now DELETES any RESUME_CONTEXT_KEY
absent from the persisted context, so the resume body carries exactly the
graph-determining fields the pause had. Previously only defined keys were
overwritten, leaving a client-supplied `addedConvo` (which the request
fingerprint does not cover) in place — a crafted resume could rebuild a
single-agent checkpoint as a different multi-agent graph/tool set.
F3 (P2) — the resume route ACKs (res.json) before initializeClient, so a
post-ACK getMCPRequestContext(req, res) saw the response as finished and
returned undefined, leaving the resumed run without its run-scoped MCP
connection store (approved MCP / OAuth-overlay tools then ran without their
request-scoped connections). Pre-seed the store with a null res +
cleanupOnResponse:false before the ACK and tear it down in the finally,
mirroring the normal stream path (request.js). userMCPAuthMap was already
preserved separately, so credentials were not lost — only the connection store.
Declined: the ApprovalContext NEW_CONVO guard (P2) is a false positive — the
`created` SSE event updates the conversation atom before any pause renders, so
the id is concrete by click time (details in the PR thread).
Tests: policy.spec (absent-key delete) + resume.spec (MCP context pre-seed/cleanup order).
* fix(hitl): address Codex re-review (round 12) — resume fidelity + multi-tool UI
F4 (P2) — temporal prompt vars: resume rebuilt the agent without restoring
req.conversationCreatedAt or req.body.timezone, so {{current_datetime}}-style
vars compiled a different system prompt than the paused graph (resume wall-clock,
unzoned). Add 'timezone' to RESUME_CONTEXT_KEYS (persisted at pause, replayed by
the resume middleware) and restore conversationCreatedAt from the convo before
initializeClient — mirroring the normal path's resolveConversationCreatedAt.
F5 (P2) — multi-tool approval: applyPendingActionToMessages stopped retrying once
ANY tool-call part was tagged, so siblings that rendered on later frames never got
approval controls and the resume route 400'd the partial batch. Add
countTaggedApprovalParts and keep the bounded RAF retry going until every
action_request is tagged (ask_user_question unchanged — one synthetic part).
F6 (P3) — Edit accepted `null`/`[]` (valid JSON, non-object), enabling Submit for
a value the resume route rejects via findIncompleteDecisions. Mirror the server's
plain-object check in the client (store + editIsValid) so Submit only enables for
an accepted value.
Tests: policy.spec (timezone round-trip), resume.spec (conversationCreatedAt
restore), approval.spec (countTaggedApprovalParts).
* fix(hitl): address Codex re-review (round 13) — recurse into subagent approvals
F9 (P2) — a tool paused INSIDE a subagent has its tool_call_id in the parent
subagent tool_call's nested `subagent_content`, not as a top-level message part.
applyToolApproval and countTaggedApprovalParts only scanned top-level content, so
the approval never attached and the round-12 retry loop counted 0 tagged parts and
spun to its frame cap with no controls. Both now recurse into `subagent_content`
(immutably, so React refs update): the nested call gets tagged and is counted, so
the retry terminates. Added approval.spec cases for the nested tag + count.
Note: surfacing the interactive approve/reject controls inside the subagent view is
a deliberate follow-up — ToolApproval -> useResumeSubmit -> useChatContext crashes
when rendered in the portaled subagent dialog (outside the chat/approval providers),
so that needs the controls scoped to the in-provider inline render (or the dialog
wrapped with the providers). This commit fixes the data/traversal layer only.
F7 (discovered-tool history on resume) and F8 (redis chunk TTL pause race) were
verified false positives — see the PR threads.
* fix(hitl): address Codex re-review (round 14) — resume fidelity + expiry relay
F13 (P2) — manualSkills are graph-determining (skill allowed-tools union into the
tool set before tools load) but weren't replayed, so a reload lost the skill tools
and a crafted resume could inject a different skill past the fingerprint. Add
'manualSkills' to RESUME_CONTEXT_KEYS (same replay-only pattern as timezone/
addedConvo; the delete-absent half blocks injection). Not alwaysAppliedSkills —
that's resolved server-side from the DB, not req.body.
F12 (P2) — the resume final SSE built requestMessage from job.metadata.userMessage
(persisted without files), so attachments vanished from the user bubble on resume.
Spread the already-restored req.body.files onto it, matching the normal path.
F11 (P2) — multi-replica approval expiry: RedisJobStore.cleanupRequiresActionIndex
on another replica can win the requires_action->aborted CAS (it sets the hash error
but has no event transport), and the local sweep then skips because the job is no
longer requires_action, so a client subscribed here never gets the terminal error
until the reap path. expireStaleApprovals now relays APPROVAL_EXPIRED_ERROR for a
locally-subscribed job already aborted FOR approval expiry (error-string gated,
idempotent via the errorEvent flag). emitError already publishes cross-replica.
Tests: policy.spec (manualSkills round-trip + inject-drop), resume.spec (final
requestMessage carries restored files).
* fix(hitl): render approval controls for subagent-nested tool pauses (F10)
Round-13 made applyToolApproval/countTaggedApprovalParts recurse into
subagent_content (data), but SubagentDialogPart rendered nested TOOL_CALL parts
with <ToolCall> only and never mounted <ToolApproval>, so a tool paused inside a
subagent showed no controls and the run was unresolvable.
Render <ToolApproval> in SubagentDialogPart's TOOL_CALL branch when the nested
tool_call carries an approval and isn't yet resolved, mirroring the top-level
Part.tsx render. The subagent dialog portals (OGDialog → ReactDOM.createPortal),
but React context flows through the React tree, not the DOM tree, so ToolApproval
resolves ApprovalProvider/ChatContext and the controls work + submit.
Also harden useResumeSubmit: read ChatContext via useContext (non-throwing)
instead of the throwing useChatContext wrapper, so the cards never crash when
rendered outside a ChatContext.Provider (e.g. a search/citation render that passes
chat context as a prop) — they degrade to inert (buildResumeFields returns null).
* style(hitl): re-sort run.ts imports after dev rebase
* fix(hitl): address Codex re-review (round 15) — resume content fidelity
F14 (P2) — hide_sequential_outputs was applied in chatCompletion before
saving/emitting content but not on resume, so a sequential-agent chain that
pauses for HITL and resumes persisted/emitted intermediate outputs the setting
is meant to hide. Extracted the filter into applyHideSequentialOutputsFilter()
and call it from both chatCompletion and resumeCompletion (after handleRunInterrupt,
covering the finalize + re-pause reads of client.contentParts).
F16 (P2) — on a reloaded HITL pause, the DB already holds the paused user row +
partial assistant row; useResumeOnLoad fed those as submission.messages, then
finalHandler/createdHandler appended the same pair via requestMessage/responseMessage,
duplicating the turn (buildTree doesn't dedupe children by messageId). buildSubmission-
FromResumeState now strips the paused user/response rows (by messageId, incl. the
padded/unpadded response id) from submission.messages — they're re-supplied by the
placeholders + final event. Frontend-only; live (non-reload) pause path untouched.
Deferred: F15 (collapsed-card subagent approval registration/visibility) — see thread.
Tests: client.test (filter keeps last + tool_call parts / no-op when off),
useResumeOnLoad.spec (paused pair stripped from submission.messages).
* fix(hitl): address Codex re-review (round 16) — chunk TTL, slot, job replacement
F17 (P2) — chunk-stream TTL on pause-before-chunk. CHUNK_APPEND_LUA derived its
ceiling only from the chunk key's current TTL, so when the chunks key didn't exist
at pause (fire-and-forget append in flight, or an ask-user pause before any chunk),
the on_pending_action append created the stream with only the 20m running TTL while
the approval window is 24h — content evicted before resume. The Lua now also reads
the job key (KEYS[2]); when status == requires_action it takes max(running, TTL(jobKey))
(the approval window transitionStatus set), else the running TTL. Extend-only preserved;
gated on paused status so normal runs never inflate. Both keys share {streamId} (cluster-safe).
F19 (P2) — with LIMIT_CONCURRENT_MESSAGES, the approval prompt was emitted before the
original request released its slot, so a fast Approve got /resume 429'd. handleRunInterrupt
now releases the slot (idempotent via pendingRequestReleased) right after the pause, before
the prompt; the request.js pause branch and resume.js finally only release if it didn't
(no double-release).
F20 (P2) — finalizeResumedTurn never checked the job wasn't replaced before emitDone/
completeJob/saveMessage, so a stale resume could clobber a newer turn that reused the
conversationId. Added the createdAt guard the normal request path uses (skip finalization
when the live job's createdAt != the paused job's).
Deferred: F18 (subagent_content not reconstructed on Redis resume) — joins the subagent
cluster (F15). See thread.
Tests: RedisJobStore integration (pause-before-chunk gets approval TTL; running stays short),
resume.spec (skip finalization on replacement; no double slot release on re-pause).
* 🛡️ fix: Guard HITL terminal side-effects against job replacement
Jobs are keyed by streamId == conversationId, so a new request REPLACES the
running one on the same conversation. The replaced generation's tail must not
clobber the live generation's state. Each path now re-reads the live job and
compares createdAt against the generation's captured identity before acting.
- Thread the generation's createdAt onto the client (request.js + resume.js)
as client.jobCreatedAt — the identity every guard compares against.
- handleRunInterrupt: skip approvals.pause when this run is no longer the live
job, so a stale interrupt can't flip the NEWER job to requires_action.
- chatCompletion finally: skip the checkpoint prune when replaced, so an older
run's late finally can't delete the newer run's resume checkpoint.
- resume catch-path: gate emitError/completeJob/prune behind a stillLive check
(fail-open if the read throws), mirroring finalizeResumedTurn's success guard.
- Persist the turn's uploaded files on job.metadata.userMessage (authoritative
trackUserMessage writer) and prefer them on resume over the user DB row, whose
save can still be racing a fast /resume.
Tests: 13 guard-predicate cases in jobReplacement.spec.js.
* 🔁 fix: Harden HITL resume — ownership re-check, file seeding, deferred-tool replay
Three follow-ups to the round-17 job-replacement guards (Codex review 4594099963):
- G1 (resume.js): the success-path ownership guard runs at the START of
finalizeResumedTurn, but saveMessage + first-turn title generation await long
enough for a new request to replace the job on the same conversationId. Re-read
the live job immediately before emitDone/completeJob/prune so the terminal writes
can't tear down the REPLACEMENT job — mirrors the catch-path guard.
- G2 (request.js): onStart's metadata/chunk writes that persist the turn's files
are fire-and-forget, so a fast approval could read job.metadata.userMessage before
files landed. Seed files into getPreliminaryUserMessage instead — that write is
AWAITED before the run starts, so files are durable before any interrupt can emit.
- G3 (run.ts + client.js + resume.js + IJobStore.ts): the resumed graph is rebuilt
with messages: [], so createRun's tool_search-discovery scan finds nothing. A
deferred tool discovered earlier in the turn (and targeted by the paused call) was
therefore absent from the rebuilt schema-only toolMap — resume would throw "unknown
tool" (no loadRuntimeTools fallback is wired). Capture discovered tool names at
pause via extractDiscoveredToolsFromHistory(run.getRunMessages()), persist them on
job.metadata.discoveredTools, and replay them into createRun's new discoveredToolNames
input (merged with message-extracted names, gated on hasAnyDeferredTools — inert
otherwise). A new createRun test proves the deferred tool is promoted with the replay
and absent without it (reproducing the bug).
Tests: real createRun deferred-replay suite (run-summarization.test.ts) + G1/G2/G3
guard predicates (jobReplacement.spec.js). Full suite green.
* 🔒 fix: Close HITL resume metadata + file-substitution + pause-race gaps
Four findings on the round-18 commit (Codex review 4594430222):
- H1 (P1, regression in round-18 G3): the discoveredTools captured at pause never
reached resume — three metadata allowlists dropped it: GenerationJobManager
.updateMetadata, RedisJobStore.deserializeJob, and buildJobFacade (plus the
GenerationJobMetadata type). Added discoveredTools to all four, so the deferred-tool
replay actually works end-to-end (in-memory store already kept it via Object.assign).
- H2 (P2, security): /resume honored a client-supplied `files` array, letting a crafted
client resume an approved code/read-file tool against a DIFFERENT file set than the one
approved (files aren't in the resume fingerprint/context). Resume now ALWAYS sources
files from the paused job (metadata → DB row), clearing any client-supplied set.
- H3 (P2, ephemeral fidelity): non-default model parameters (temperature, max tokens,
custom endpoint params) were lost on resume — ephemeral agents derive them from the
request body, which the resume payload omits. Capture the resolved model_parameters in
resumeContext at pause and replay them onto the body on resume (excluding `model`, which
is replayed via the fingerprinted RESUME_CONTEXT_KEYS path). Saved agents already source
these from the DB.
- H4 (P2, Redis race): a pause landing between the resume snapshot and the Pub/Sub
subscription reached neither resumeState.pendingAction nor (Redis) pendingEvents, and
approval events aren't persisted to replayEvents — the client attached to a paused job
with no approval UI. subscribeWithResume now re-reads the live job AFTER subscribing and
surfaces the pending action if the snapshot missed it (live read, no staleness).
Tests: discoveredTools metadata round-trip + subscribeWithResume re-read (pendingAction
.spec.ts); client-file substitution rejection (resume.spec.js); model-parameter replay
predicate (jobReplacement.spec.js).
* 🧹 fix: Clear stale discovered tools, release slot on claim error, extend run-step TTL
Three follow-ups on the round-19 commit (Codex review 4594783691):
- I1 (P2): the round-19 discoveredTools field wasn't cleared on Redis streamId reuse.
HSET only overwrites listed fields and handleRunInterrupt only writes discoveredTools
when THIS turn discovers a deferred tool — so a replacement turn that pauses without its
own discovery inherited the prior run's tool names and force-loaded undiscovered deferred
tools on resume. Added discoveredTools to createJob's staleHitlFields HDEL list (the
in-memory store already builds a fresh object, so it was Redis-only).
- I2 (P2): with LIMIT_CONCURRENT_MESSAGES, approvals.resolve runs after the slot increment
but before the run's try/finally, so a store/Redis error there leaked the slot until the
counter TTL expired (spurious 429s on retry of the still-paused approval). Wrapped the
claim in try/catch that decrements the slot and returns 500.
- I3 (P3): saveRunSteps did SET ... EX running unconditionally, resetting the run-steps key
to the 20-min running TTL even while the job is paused for the longer approval window —
a reload after that window lost the tool timeline. Now uses a paused-window TTL script
mirroring the chunk-stream no-shrink behavior (extends to the approval window when the
job hash is requires_action).
Also fixes a latent strict-tsc cast error in the round-19 pendingAction test.
Tests: claim-throws-releases-slot (resume.spec.js); discoveredTools cleared on reuse +
saveRunSteps preserves the paused TTL (RedisJobStore integration, USE_REDIS).
* 🛡️ fix: Guard fast-resume save race, gate HITL to resumable routes, expire on stale submit
Three findings on the round-20 commit (Codex review 4595045652):
- J2 (P1): a fast /resume can claim + finalize the COMPLETED response while the original
request's pause branch is still awaiting `response.databasePromise`; the later
unfinished-save then overwrites the completed content. Re-check the job is still paused on
THIS generation's action (a claim leaves requires_action; a replacement bumps createdAt)
before marking the row unfinished; fail open on a read error.
- J3 (P1): the tool-approval wiring (humanInTheLoop + PreToolUse hook + checkpointer) was
applied to EVERY createRun caller when toolApproval.enabled, but the OpenAI-compatible and
Responses controllers never inspect run.getInterrupt() or persist a pending action — an
approval-gated tool would pause there with no approval surface or resume endpoint and the
route would emit a normal final response / [DONE] with the tool call dangling. Gate the
wiring on a new createRun `hitlCapable` flag, set only by AgentClient (chat + resume).
- J4 (P2): a stale-action 409 on submit returned without driving expiry, leaving the job
requires_action with a dead action until the periodic sweeper ran — any attached SSE client
got no terminal event and the stream appeared to hang. Extracted GenerationJobManager
.expireApproval(streamId, actionId) (expire CAS + terminal SSE, shared with the sweeper) and
call it from the resume route when the observed action is stale.
J1 (nested subagent approval controls not mounting while the details dialog is closed) is a
valid frontend issue in the deferred subagent-HITL path — tracked separately (replied on the
thread) since the fix touches the shared dialog primitive and needs UI verification.
Tests: HITL-gate both directions (run-summarization.test.ts); expire-on-stale-submit
(resume.spec.js); fast-resume unfinished-save guard predicate (jobReplacement.spec.js).
* 💄 style: Wrap captureAgents signature to satisfy prettier (CI lint)
* 🧠 feat: Configurable Reasoning Replay for Custom Endpoints
Adds customParams.includeReasoningContent so OpenAI-compatible custom endpoints (e.g. Xiaomi MiMo, Kimi) can replay reasoning_content on tool-call turns natively, without impersonating the moonshot provider.
* 🔁 feat: Replay reasoning_content across turns for opted-in custom endpoints
Extends the DeepSeek reasoning-content format spoof to honor customParams.includeReasoningContent, so custom OpenAI-compatible endpoints (Xiaomi MiMo, Kimi) reconstruct reasoning_content from persisted history on later turns, matching DeepSeek thinking-mode parity. Adds shouldReplayReasoningContent predicate (tested) and surfaces the flag on the initialized agent.
* 🪢 refactor: Split within-run vs cross-turn reasoning replay flags
moonshot only replays reasoning_content within a run's tool calls, not across turns. Decouples the two: includeReasoningContent = within-run replay (exact moonshot parity), new includeReasoningHistory = cross-turn reconstruction from persisted history (implies includeReasoningContent, since reconstruction is a no-op without the within-run replay flag).
* 🩹 fix: Apply reasoning replay across all param-format branches
Move the within-run includeReasoningContent application out of the OpenAI-only branch in getOpenAIConfig to after the branch dispatch, so custom endpoints using anthropic/google defaultParamsEndpoint gateway modes also honor includeReasoningContent/includeReasoningHistory. Addresses Codex finding.
* chore: Update @librechat/agents to v3.2.46
* 🧽 refactor: De-spoof reasoning replay via explicit preserveReasoningContent
Now that @librechat/agents 3.2.46 exposes an explicit preserveReasoningContent option on formatAgentMessages, pass it directly instead of impersonating provider: deepseek. Behavior is unchanged (shouldReplayReasoningContent still gates DeepSeek + the custom includeReasoningHistory flag); also corrects the comment to reference includeReasoningHistory.
* 🌳 fix: Walk subagents in the reasoning-history replay gate
The gate only checked the primary agent and top-level handoff/parallel configs, so an opted-in custom endpoint used solely as a nested subagent had its persisted reasoning dropped on later turns. New exported anyAgentReplaysReasoningContent walks subagentAgentConfigs (cycle-safe, mirrors anyAgentHasCodeEnv); client.js uses it. Addresses Codex finding.
* 🔧 chore: Update dependencies in package-lock.json and package.json
Bump `form-data` to version 4.0.6 and update `hasown` and `mime-types` dependencies in package-lock.json. Add an `overrides` section in package.json to ensure compatibility with the new `form-data` version.
* 📦 chore: Bump `@librechat/agents` to v3.2.42
* fix(agents): use `path` for read/write/edit/create file tools
Pairs with @librechat/agents renaming the read_file/write_file/edit_file tool
parameter from `file_path` to `path` (models — esp. Kimi K2 — emit `path` far
more reliably, and it matches grep/glob/list_directory which already use `path`).
- tools.ts: LibreChat's own code/skill file-tool schemas use `path`
(the skill read_file tool inherits the SDK definition, which is already renamed)
- handlers.ts: read `args.path` for the model-facing tool arg + error messages
- the internal host `readSandboxFile`/`writeSandboxFile` contract is unchanged
- tests updated
Requires @librechat/agents with the param rename (danny-avila/agents#250). All
agents unit suites green (175).
* chore: update @librechat/agents to v3.2.41 and bump related dependencies in package-lock.json and package.json files
* fix(api): Refactor header merging in MCPConnection to use Object.assign for clarity
* test(e2e): mock emits `path` for create/edit file-authoring tools
The mock LLM still sent `file_path` for the create_file/edit_file calls, which the
renamed handlers no longer read -> the skill-file-authoring e2e failed with
'Expected skill to be persisted'. Switch the fixture to `path` to match the tools.
(The internal readSandboxFile/writeSandboxFile contract stays on `file_path`, so
api/server/services/Files/Code/process.js and its spec are unchanged.)
* 🔧 chore: Update `@librechat/agents` to v3.2.38 and bump related dependencies in package-lock.json and package.json files
* 🔧 chore: Upgrade `multer` dependency to version 2.2.0 in package-lock.json and package.json
* 🔧 chore: Upgrade `nodemailer` dependency to version 9.0.1 in package-lock.json and package.json
* 🔧 chore: Upgrade `@aws-sdk/client-bedrock-agent-runtime` and `@aws-sdk/client-bedrock-runtime` to versions 3.1071.0, update related dependencies in package-lock.json and package.json
* 🔧 chore: Upgrade `form-data` to version 4.0.6 and `hono` to version 4.12.25, update related dependencies in package-lock.json and package.json
* 🔧 chore: npm audit fix
* 🔧 chore: Remove unused Babel dependencies from package-lock.json and package.json
* 🔧 chore: Add '@mistralai/mistralai' to esModules in Jest configuration files
* 🛡️ fix: Bound object-traverse against DAG fan-out and shared refs
Detect cycles via the ancestor chain (so shared, non-circular references in sibling branches / DAGs are traversed correctly) and add defensive maxNodes (100k) / maxDepth (100) caps. The removed global visited set was implicitly bounding work at O(distinct nodes); ancestor-chain-only detection is O(root-to-node paths), exponential on DAGs (a depth-24 diamond went from 26 to 50M visits / 1.6s of synchronous work). The caps bound it to ~9ms while leaving normal traversal untouched. Adds a spec covering shared refs, cycles, DAGs, and both bounds. The lone consumer, debugTraverse, inherits the defaults with no change.
* 🪵 refactor: Remove legacy api/config logger duplicate
The api/config winston logger was a stale parallel implementation of the canonical @librechat/data-schemas logger, with unbounded redaction (regex-only redactFormat, npm traverse-based debugTraverse). Its winston instance and the logger export from api/config/index.js had zero consumers — every ~/config importer uses the MCP/flow-manager exports. The only live tie was ToolService's use of redactMessage.
Re-export redactMessage from @librechat/data-schemas (behaviorally identical, a superset of the regex set), point ToolService at it, delete api/config/winston.js and api/config/parsers.js, drop the dead logger export, and remove the orphaned ~/config/parsers mock from the global test setup.
* 🧹 chore: Drop orphaned traverse dep and stale legacy logger tests
Deleting api/config/{winston,parsers}.js left the npm 'traverse' package unused in api/package.json (flagged by the detect-unused-packages CI check) and orphaned two tests that imported the deleted modules. Remove the traverse dependency (sync package-lock), and delete api/config/__tests__/{parsers,logToFile}.spec.js — the canonical logger's behavior is covered by packages/data-schemas/src/config/parsers.spec.ts.
* 🩹 fix: Make object-traverse caps bound work and survive update()
Address Codex review: (1) break the child loops as soon as the node budget is spent and iterate objects via for...in instead of materializing Object.entries/Object.keys, so maxNodes actually bounds work for wide arrays/objects; (2) detect ancestor cycles against an immutable original-node stack rather than context.node, which a callback's update() can reassign (the debug formatter rewrites array nodes in place). Adds tests for the wide-array bound and the update()-cycle case.
* 🎚️ fix: Tighten object-traverse defaults to a ~1ms log budget
Lower maxNodes 100000 -> 2500 and maxDepth 100 -> 5. Measured cost is ~140ns/node with the debug formatter callback, so 2500 nodes keeps a single log under ~1ms even on slower prod hardware; real log objects are ~25-30 nodes at depth 3-4, leaving ample headroom. maxNodes is the fan-out/cost lever; maxDepth bounds recursion and output readability (depth-5 covers typical logs, deeper renders compactly).
* 🪙 feat: Context-usage projection — data-provider + client wiring
Consumer side of the SDK-aligned context projection (agents
`projectAgentContextUsage`). Adds the `/api/endpoints/context-projection`
data-provider plumbing (endpoint, service, query key, `TContextProjectionRequest`)
and a `useContextProjectionQuery` gated to fire only when no fresh snapshot
covers the viewed branch.
Wires `useTokenUsage` precedence to: live snapshot → fresh persisted snapshot
(window matches the resolved one) → server projection → per-message estimate.
A model/window switch marks the baked snapshot stale (its `maxContextTokens`
no longer matches) and falls to the projection — closing the gauge's
window-switch (G1) and snapshot-less-branch (G2) gaps. Snapshot and projection
share the render-relevant fields, so they render uniformly.
Backend endpoint + agents version bump land in follow-up commits. Includes the
design spec (CONTEXT_PROJECTION_SPEC.md).
* 🪙 feat: Context-projection backend endpoint
POST /api/endpoints/context-projection → resolveContextProjection (packages/api):
reconstructs the viewed branch (parent-chain walk from messageId), resolves the
agent config (instructions/provider/model/maxContextTokens), reuses LibreChat's
stored per-message tokenCounts as the index map (no re-tokenizing), and calls
the agents SDK projectAgentContextUsage — no model call. Thin controller injects
db.getMessages/db.getAgent; route mirrors /token-config.
First cut targets message-windowing accuracy; tool-schema tokens are deferred to
a follow-up that reuses the full initializeAgent path.
* 🩹 fix: Codex review on context projection (G1 guard, IDOR, recount, summary)
- Guard `currentActive` against a stale window: a model/window switch on the
current branch left the live snapshot outranking the projection (G1 didn't
fire). Now defers to the projection unless streaming or the window matches.
- Scope branch lookups to the authenticated user (`getMessages` filter +
injected `userId`) — was loading any conversation by id (IDOR).
- Recount messages with no stored `tokenCount` via the tokenizer instead of
charging 0, so snapshot-less/imported histories don't under-report.
- Fall back (null) for already-summarized branches rather than projecting from
the full raw parent chain (the next call would send summary + tail); the
client's summary-baseline-aware estimate handles them until a follow-up
replays the summary boundary.
* 🩹 fix: Codex round 2 — drop agent load, summary marker, edit-invalidation
- Stop loading agent/model-spec config server-side (closes the agent-access
IDOR and the spec-prompt special-casing). Provider/model/window now come from
the client-resolved request (`limits.endpoint`/model — the agent's real
provider, not the `agents` endpoint, so the tokenizer is right). Agent/spec/
promptPrefix instructions are uniformly deferred to the full-fidelity follow-up.
- Detect summarized branches via the live path's `metadata.summaryUsedTokens`
marker (was the wrong `summaryTokenCount` field) and fall back to the
summary-aware estimate.
- Invalidate the projection query on in-place message edits via a branch
content `revision` in the cache key (the tail id is unchanged on edit).
Deferred (valid, not a regression): same-window endpoint/model switch keeps a
window-matched snapshot — needs endpoint/model persisted on the snapshot, which
lands with the fidelity follow-up. Smoke-tested: fits / prunes / summarized→null
/ no-window→null.
* 🛡️ fix: make context projection strictly additive (no-regression)
Revert the G1 window-match guard on the live/branch snapshot. When no explicit
maxContextTokens is set (the common default), the SDK's snapshot window is
reserve-derived (~0.9·(modelContext − maxOutputTokens)) while useTokenLimits
resolves the raw model context — so `snapshot.maxContextTokens === resolvedMax`
is false for the SAME model, and the guard would wrongly drop a valid
current-branch snapshot to projection/estimate post-stream (a regression in the
default case, per initialize.ts:1240-1243).
The projection now activates ONLY for snapshot-less branches (G2): the
precedence is live snapshot → persisted branch snapshot → projection → estimate,
where the first two are byte-for-byte the prior behavior and the projection just
slots ahead of the estimate. Window/model-switch (G1) detection needs the
snapshot to carry its model/window and defers to the fidelity follow-up.
* 🩹 fix: surface projections as estimates, not authoritative snapshots
A first-cut projection carries the SDK's windowing but omits instruction/tool
overhead, so rendering it as `isEstimate: false` showed a confident under-count
for snapshot-less branches. Mark projection-sourced views `isEstimate: true` +
`snapshotActive: false` (and drop the snapshot field) so they present as a
better estimate than sumBranch — improved used/window number, estimate framing,
no misleading granular breakdown with ~0 tools. Real snapshots stay
authoritative. (Codex round 3, projection.ts:139.)
* 🧹 chore: drop CONTEXT_PROJECTION_SPEC.md from the PR
* 🎨 style: fix import-sort order in projection.ts (CI sort-imports check)
* 🔧 chore: update @librechat/agents dependency to version 3.2.36 in package-lock.json and related package.json files
* chore: npm audit fix
* 🎨 style: fix import-sort order in data-service.ts (CI sort-imports check)
* 🩹 fix: drop dead calibrationRatio in projectionParams (tsc never error)
Inside the ternary, branchSnapshot is narrowed to null (the gate is
), so accessed a
property on (frontend typecheck failure). It was also dead — there is
never a snapshot to seed from in this branch — so just remove it.
* Revert "chore: npm audit fix"
This reverts commit 4cdb862d0c.
* 🧾 fix: Bill Subagent Child-Run Model Usage in Parent Transactions
* 🩹 fix: Type Subagent Usage Sink Structurally Until SDK Release
* 🔧 chore: Update @librechat/agents dependency to version 3.2.35 in package-lock.json and related package.json files
* 📬 feat: Report Tool Results Per Call via onResult Channel
Tool batches already execute in parallel here, but results were only
delivered to the agent graph through the single resolve(results[])
call — so a fast tool's completion event waited on the slowest call
in the batch. Report each result through the optional onResult channel
(agents SDK > 3.2.33) as it settles, letting the graph emit that
call's completion immediately. resolve remains the authoritative batch
outcome; the callback is optional-chained, so this is a no-op until
the SDK release lands and remains backward compatible after.
* 🧹 chore: Prettier Formatting in onResult Spec
* 🧹 chore: Sort Imports in handlers.ts
* 🔧 chore: Update @librechat/agents dependency to version 3.2.34 in package-lock.json and related package.json files
Keep the Google Gen AI SDK aligned with the latest 2.x release. Updates the
declared range in both backend manifests (api, packages/api) and regenerates
the lockfile to resolve @google/genai to 2.8.0.
No application code changes: the sole consumer
(api/app/clients/tools/structured/GeminiImageGen.js) uses the stable
`GoogleGenAI` constructor and `models.generateContent` API, and the upstream
changelog records no breaking changes to those between 2.0 and 2.8.
Closes#13551
The tsdown migration (#13595) externalizes all third-party imports
(Rollup inlined them), so several modules the api source imports must be
present at runtime. Six were not, causing production (`npm ci --omit=dev`)
to crash on boot with `Cannot find module 'get-stream'` (then the next).
Fixed following the package's existing convention — packages/api declares
runtime libs as `peerDependencies`, and the `/api` app provides them as
real `dependencies` (how express/mongoose/sharp already resolve):
- `api/package.json` (the prod app, the provider): add the 3 that were
missing — `get-stream`, `jszip`, `mongodb`. (`dedent`/`lodash`/`nanoid`
were already provided by /api.)
- `packages/api/package.json`: add all 6 to `peerDependencies` (the
contract) and to `devDependencies` (workspace build/tests), matching
the existing `mammoth`/`pdfjs-dist`/`sanitize-html` dev+peer pattern.
`jszip`/`mongodb` move out of dev-only (were pruned in production).
Pinned to CJS-compatible majors (get-stream@6, nanoid@3). Verified the
built bundle has zero undeclared externals and the 3 newly-provided deps
are production (non-dev) in the lockfile, so they survive `--omit=dev`.
* 🔧 chore: Update ESLint config, add import sorting script, Test Sharding, Bump `@librechat/agents`
* Change 'no-nested-ternary' rule from 'warn' to 'error' in ESLint config
* Add new scripts for sorting imports in the project
* Update lint-staged configuration to include import sorting
* Modify GitHub Actions workflows to support sharding for unit tests
* chore: remove nested ternary expressions
* refactor: Extract scale multiplier logic into a separate function in CircleRender component
* refactor: Simplify auto-refill rendering logic in Balance component for better readability
* refactor: Improve width style handling in DataTable components for clarity and maintainability
* chore: remove CircleRender component
* delete: Remove CircleRender component as it is no longer needed in the project
* chore: Bump @librechat/agents to version 3.2.31 and update Node.js engine requirement
* Update @librechat/agents dependency from 3.2.2 to 3.2.31 in package-lock.json, api/package.json, and packages/api/package.json
* Change Node.js engine requirement from >=20.0.0 to >=24.0.0 in @librechat/agents
* chore: Add import sorting check to ESLint CI workflow
* Implement a new job in the GitHub Actions workflow to verify import ordering on changed files.
* The job checks for changes in specific file types and reports any import order drift, providing instructions for local fixes.
* 🎭 test: Run Mock E2E Suite Through createRun With In-Process Fake Model
Replace the standalone HTTP mock LLM server with an in-process fake model
injected into the real createRun -> Run.create pipeline via
run.Graph.overrideTestModel, so the mock suite exercises the agents
integration end-to-end without a live provider or a separate server.
- Bump @librechat/agents to 3.2.2 for the FakeChatModel/createFakeStreamingLLM exports
- Add an env-gated applyTestRunHook seam in packages/api createRun (no /api changes)
- Add e2e/setup/fake-model.js to drive default replies + the skill-authoring tool-call flow
- Drop the mock-llm webServer from playwright.config.mock.ts and set LIBRECHAT_TEST_RUN_HOOK
* 🧹 test: Retire Standalone Mock LLM Server From E2E Recorder
Migrate the `--profile=mock` recorder onto the same in-process fake model
as the Playwright mock suite, then delete the now-unused HTTP mock server
so the fake-LLM logic lives in a single place.
- Point record.js mock profile at the fake model via LIBRECHAT_TEST_RUN_HOOK
- Remove the mock-llm-server spawn/wait and MOCK_LLM_PORT plumbing from record.js
- Delete e2e/setup/mock-llm-server.js (e2e/setup/fake-model.js is now the only source)
- Update e2e/README.md to describe the in-process fake LLM
* 🏷️ ci: Rename Playwright Mock E2E Check to Playwright E2E Tests
* chore: Update @librechat/agents to version 3.1.93 and @langfuse packages to version 5.3.0 in package-lock.json and package.json files
* chore: Update browserify-sign to version 4.2.6 and qs to version 6.15.2 in package-lock.json
* 📦 chore: npm audit fix 2026-05-18
- Added @js-sdsl/ordered-map version 4.4.2
- Updated @librechat/agents to version 3.1.87
- Upgraded @opentelemetry/sdk-node to version 0.218.0
- Added new dependencies for gRPC and OpenTelemetry exporters
* 🔧 chore: Update @librechat/agents to version 3.1.87 in package-lock.json and package.json files
* 🔧 chore: Upgrade @opentelemetry/sdk-node to version 0.218.0 in package.json and package-lock.json
* 📦 chore: Bump `@librechat/agents` to v3.1.86 in package-lock.json and package.json files
* 📦 chore: Update dependencies in package-lock.json to latest versions, including @protobufjs/codegen, @protobufjs/inquire, @protobufjs/utf8, and protobufjs
* 📦 chore: Add `librechat-data-provider` dependency in package.json and package-lock.json, and update build dependencies in turbo.json
* 📦 chore: Update @librechat/agents to version 3.1.85 in package-lock.json and package.json files
* 📦 chore: Update mermaid to version 11.15.0 in package.json and package-lock.json
* 🧱 refactor: typed CodeEnvRef + kind discriminator + tenant-aware sandbox cache
Final cutover for the LibreChat ↔ codeapi sandbox file identity. Replaces
the magic string `${session_id}/${file_id}?entity_id=...` with a typed,
discriminated `CodeEnvRef`. Pre-release lockstep deploy with codeapi
#1455 and agents #148; no legacy aliases retained.
## Final shape
```ts
type CodeEnvRef =
| { kind: 'skill'; id: string; storage_session_id: string; file_id: string; version: number }
| { kind: 'agent'; id: string; storage_session_id: string; file_id: string }
| { kind: 'user'; id: string; storage_session_id: string; file_id: string };
```
`kind` drives codeapi's sessionKey: `<tenant>:<kind>:<id>[✌️<version>]`
for shared kinds, `<tenant>:user:<userId>` for user-private (auth context
provides `userId`). `version` is statically required for `kind: 'skill'`
and forbidden otherwise via discriminated union — constraint holds at
compile time on every consumer, not just codeapi's runtime validator.
`id` is sessionKey-meaningful for `'skill'` / `'agent'`; informational
only for `'user'` (codeapi resolves user identity from auth context).
## What changed
- `packages/data-provider/src/codeEnvRef.ts` — discriminated union +
`CODE_ENV_KINDS` const-tuple keeps the runtime list and TS union
locked together.
- Schemas: `metadata.codeEnvRef` and `SkillFile.codeEnvRef` enums
tightened to `['skill', 'agent', 'user']`.
- `primeSkillFiles` writes `kind: 'skill'`, `id: skill._id`,
`version: skill.version`. Cache-hit path reads `codeEnvRef`
directly. Bumping `skill.version` on edit naturally invalidates
the prior cache entry under the new sessionKey.
- `processCodeOutput` writes `kind: 'user'`, `id: req.user.id`. Output
bucket is always user-scoped, regardless of which skill the
execution invoked. New regression test pins the asymmetry.
- `primeFiles` reupload preserves `kind`/`id`/`version?` from the
existing ref so a skill-cache-miss reupload doesn't silently demote
to user bucket.
- `crud.js` upload functions (`uploadCodeEnvFile` /
`batchUploadCodeEnvFiles`) thread `kind`/`id`/`version?` to the
multipart form (codeapi #1455 option α). Without these on the wire,
codeapi falls back to user bucketing and skill-cache invalidation
never fires. Client-side validation mirrors codeapi's validator.
- `Files/process.js` — chat attachments use `kind: 'user'`; agent
setup files use `kind: 'agent'`.
- Drops `entity_id` everywhere (struct, schema sub-docs, write paths,
upload form fields). Drops `'system'` from the kind enum (no emitter
ever existed).
## Test plan
- [x] `cd packages/data-provider && npx jest src/codeEnvRef.spec` — 4 / 4
- [x] `cd packages/data-schemas && npx jest` — 1447 / 1447
- [x] `cd packages/api && npx jest src/agents` — 81 / 81 in skillFiles +
handlers + resources
- [x] `cd api && npx jest server/services/Files server/controllers/agents` —
436 / 436
- [x] `cd api && npx jest server/services/Files/Code` — 98 / 98 (incl.
new "outputs are user-scoped regardless of which skill the execution
invoked" regression and "reupload forwards kind/id/version from
existing ref")
- [x] `npx tsc --noEmit -p packages/data-{provider,schemas}/tsconfig.json
&& npx tsc --noEmit -p packages/api/tsconfig.json` — clean (only
pre-existing unrelated dev errors in storage/balance, untouched here)
## Deploy notes
- **24h cache-miss burst** on first deploy. Inputs (skill caches re-prime
under new sessionKey shape) and outputs (any pre-Phase C skill-output
cached files become unreadable). Bounded by codeapi's 24h TTL.
- **Lockstep with codeapi #1455 and agents #148.** Either repo can land
first since no aliases to drain, but the three deploys must overlap
within the same maintenance window.
- **`@librechat/agents` bump to `3.1.79-dev.0`** required after agents
#148 lands and is published.
## What this enables
Auth bridge work (JWT-based tenant/user identity between LC and codeapi)
— codeapi now derives sessionKey purely from `req.codeApiAuthContext.{
tenantId, userId}`, so the next chapter is replacing the header-asserted
user identity with a verified-claim path.
* 🩹 fix: persist execute_code uploads under codeEnvRef metadata key
Codex review P1 (chatgpt-codex-connector). `Files/process.js` was
storing the upload result under `metadata.fileIdentifier` even though:
- `uploadCodeEnvFile` now returns `{ storage_session_id, file_id }`,
not the legacy magic string.
- The post-cutover schema (`File.metadata.codeEnvRef`) only declares
`codeEnvRef` — mongoose strict mode silently strips unknown keys.
- All readers (`primeFiles`, `getCodeFilesByIds`,
`categorizeFileForToolResources`, controller filtering) check
`metadata.codeEnvRef`.
Net effect of the bug: chat-attached and agent-setup execute_code files
would lose their sandbox reference on save, and primeFiles would skip
them on subsequent code-execution turns — the file blob would still be
available locally but never re-mounted in the sandbox.
Fix: construct the full `CodeEnvRef` (`{ kind, id, storage_session_id,
file_id }`) at the write site and persist under `metadata.codeEnvRef`.
`BaseClient`'s "is this a code-env file" presence check accepts the new
shape alongside the legacy `fileIdentifier` for back-compat with any
pre-cutover records still in the database. Mirrors the same change in
`processAttachments.spec.ts` (which re-implements the BaseClient logic
for testability).
New regression tests in `process.spec.js` cover three cases:
- chat attachments (`messageAttachment=true`) → `kind: 'user'`
- agent setup (`messageAttachment=false`) → `kind: 'agent'`
- legacy `fileIdentifier` key is NOT persisted (would be schema-stripped)
* 🩹 fix: read storage_session_id on primed file refs (Codex P1)
Codex review (chatgpt-codex-connector). After Phase B's per-file
`session_id` → `storage_session_id` rename, `primeFiles` emits the
new field — but `seedCodeFilesIntoSessions` was still reading
`files[0].session_id` for the representative session and `f.session_id`
for the dedupe key. In runs with only primed attachments (no skill
seed), `representativeSessionId` was `undefined`, the function
returned the unchanged map, and `seedCodeFilesIntoSessions` silently
dropped the entire batch. The first `execute_code` call then started
without `_injected_files` and the agent couldn't see prior-turn
artifacts.
Fix:
- `codeFilesSession.ts`: read `f.storage_session_id` for both the
dedupe key and the representative session id. JSDoc updated to
match the new field name.
- `callbacks.js`: the two output-file persistence paths read
`file.session_id` to pass to `processCodeOutput` — switch to
`file.storage_session_id`. The original comment explicitly says
this should be the STORAGE session, which is exactly the field
Phase B renamed.
- `codeFilesSession.spec.ts`: fixture builder uses `storage_session_id`
and `kind: 'user'` to match the post-cutover `CodeEnvFile` shape.
Lockstep coordination: this matches the post-bump shape of
`@librechat/agents` 3.1.79+. CI tsc errors against the currently-pinned
3.1.78 are expected and resolve when the dep bumps in this PR before
merge.
* 📦 chore: Bump `@librechat/agents` to version 3.1.80-dev.0 in package-lock and package.json files
* 🪪 fix: thread kind/id/version through codeapi /download URLs (Phase C α)
Symmetric fix for the upload-side wire change in 537725a. Codeapi's
`sessionAuth` middleware now requires `kind`/`id`/`version?` on every
download/freshness URL — without them it 400s with "kind must be one
of: skill, agent, user" before serving the file.
Three sites construct codeapi-side URLs that go through `sessionAuth`:
- `processCodeOutput` (`Files/Code/process.js`): `/download/<sess>/<id>`
for freshly-generated sandbox outputs. Always `kind: 'user'` +
`id: req.user.id` — code-output files are always user-private,
regardless of which skill the run invoked.
- `getSessionInfo` (`Files/Code/process.js`): `/sessions/<sess>/objects/<id>`
for the 23h freshness check. Pulls kind/id/version straight off the
`codeEnvRef` already in scope — skill files stay skill-bucketed,
user files stay user-bucketed.
- `/code/download/:session_id/:fileId` LC route (`routes/files/files.js`):
proxies to codeapi for manual downloads. Code-output files only on
this route, so `kind: 'user'` + `id: req.user.id`.
The `getCodeOutputDownloadStream` helper in `crud.js` now takes an
`identity` param, validated by a `buildCodeEnvDownloadQuery` helper
that mirrors `appendCodeEnvFileIdentity`'s shape rules: kind required
from the closed `{skill, agent, user}` set, version required for
'skill' and forbidden otherwise. Bad callers fail fast on the client
instead of round-tripping a 400.
Also cleans up two log-noise sources reported alongside the 400:
- `logAxiosError` in `packages/api/src/utils/axios.ts` was dumping
`error.response.data` raw. With `responseType: 'arraybuffer'` that's
a `Buffer` (~4 chars per byte after JSON-serialization); with
`responseType: 'stream'` it's a `Readable` whose internal state
serializes the entire ring buffer + socket. New `renderResponseData`
decodes small buffers as UTF-8 (truncated past 2KB) and stubs streams
as `'[stream]'`. Diagnostics stay useful, log lines stop being
megabytes.
- `/code/download` route's catch was bare `logger.error('...', error)`,
bypassing the redactor. Switched to `logAxiosError` so it benefits
from the same buffer/stream handling.
Tests updated to match the new contract:
- crud.spec: `getCodeOutputDownloadStream` fixtures pass `userIdentity`;
new cases cover skill identity (with version), bad kind rejection,
skill-without-version rejection.
- process.spec: `getSessionInfo` test passes a full `codeEnvRef` object.
* ♻️ refactor: extract codeEnv identity helpers into packages/api
Per the project convention that new backend code lives in TypeScript
under `packages/api`, moves `appendCodeEnvFileIdentity` and
`buildCodeEnvDownloadQuery` from `api/server/services/Files/Code/crud.js`
into a new `packages/api/src/files/code/identity.ts` module.
Both helpers are pure validators that mirror codeapi's
`parseUploadSessionKeyInput` server-side rules (closed kind set,
`version` required for `'skill'` and forbidden otherwise) — they
deserve TS support and a dedicated spec rather than living as
JSDoc-typed helpers in the legacy `/api` workspace. The new module:
- Exports a `CodeEnvIdentity` interface using the
`librechat-data-provider` `CodeEnvKind` discriminated union.
- Adds 13 unit tests in `identity.spec.ts` covering the validation
matrix (skill+version, agent, user, and every rejection path) plus
URL encoding for the download query.
- Re-exported from `packages/api/src/files/code/index.ts` alongside
`classify`, `extract`, and `form`.
Consumer updates:
- `api/server/services/Files/Code/crud.js`: drops the local helpers
and imports them from `@librechat/api`. Net -64 lines.
- `api/server/services/Files/Code/process.js`: same.
- Test mocks for `@librechat/api` in three spec files now stub the
helpers' validation behavior locally rather than pulling them
through `requireActual` (which would drag in provider-config
init-time side effects). The package's `exports` field only
surfaces the root barrel, so leaf imports aren't reachable from
legacy `/api` test setup.
No runtime behavior change. Identity validation rules and emitted
form/query shapes are byte-for-byte identical pre/post.
* 🪪 fix: emit resource_id alongside id on _injected_files (skill 403 fix)
Companion to codeapi #1455 fix and agents 3.1.80-dev.1 — the wire
shape for shared-kind files now requires `resource_id` distinct from
the storage `id`. Without this LC change, codeapi's sessionKey
re-derivation on every shared-kind /exec rejects with 403
session_key_mismatch:
cached: legacy:skill:69dcf561...✌️59 (signed at upload, skill _id)
derived: legacy:skill:ysPwEURuPk-...✌️59 (storage nanoid)
Emit sites updated:
- `primeInvokedSkills` cache-hit path: `resource_id: ref.id` (the
persisted skill `_id` from `codeEnvRef.id`); `id: ref.file_id`
unchanged (storage uuid).
- `primeInvokedSkills` fresh-upload path: `resource_id: skill._id.toString()`
on every primed file (the `allPrimedFiles` builder type now carries
the field).
- `processCodeOutput`'s `pushFile` (Code/process.js): `resource_id: ref.id`
— for `kind: 'user'` this is informational (codeapi derives
sessionKey from auth context) but emitted for shape uniformity
with shared kinds.
Bumps `@librechat/agents` to `^3.1.80-dev.1` (the version that
ships the matching `CodeEnvFile.resource_id` field).
## Test plan
- [x] `cd packages/api && npx jest src/agents` — 67 / 67 pass
(skillFiles fixtures updated to assert `resource_id` on the
emitted CodeSessionContext.files).
- [x] `cd api && npx jest server/services/Files server/controllers/agents` —
445 / 445 pass (process.spec fixtures updated for the reupload
+ cache-hit emission).
- [x] `npx tsc --noEmit -p packages/api/tsconfig.json` — clean.
* fix(skill-tool-call): carry resource_id through primeSkillFiles → artifact
Codeapi was 400ing every /exec following a `handle_skill` tool call
with `resource_id is invalid` (`type: 'undefined'`). Both code paths
in `primeSkillFiles` (cache-hit + fresh-upload) returned files
without `resource_id`/`kind`/`version`, and the artifact in
`handlers.ts` forwarded the stripped shape into
`tc.codeSessionContext.files` → `_injected_files`.
`primeInvokedSkills` (the NL-detected loader) had already been fixed
end-to-end; this commit aligns the tool-invoked path with the same
contract: `resource_id` = `skill._id.toString()`, `kind: 'skill'`,
`version` = the skill's monotonic counter.
Tests added to `skillFiles.spec.ts` lock the contract on
`primeSkillFiles` directly so future refactors can't silently drop
the resource identity again.
* fix(handlers.spec): align session_id → storage_session_id rename + kind discriminator
Pre-existing TS errors against the post-rename `CodeEnvFile` shape:
the test file still used `session_id` on per-file objects (renamed to
`storage_session_id` in agents Phase B/C) and was missing the `kind`
discriminator the discriminated union requires. Both inputs and the
matching `expect.toEqual(...)` mirrors updated together so the
runtime equality check still holds.
Lines 723-732 stay as-is — they sit behind `as unknown as
ToolCallRequest` and TS already skipped them.
* chore: fix `@librechat/agents`, correct version to 3.1.80-dev.0 in package.json files
* chore: bump `@librechat/agents` to version 3.1.80-dev.1 in package.json and package-lock.json
* chore: bump `@librechat/agents` to version 3.1.80-dev.2
* feat(observability): trace file priming chain from primeCodeFiles to _injected_files
Diagnosing the user-upload "files=[] on first /exec" bug requires
seeing where in the LC chain a file ref disappears. Prior to this
patch the chain (primeCodeFiles → primedCodeFiles → initialSessions
→ CodeSessionContext → _injected_files) was opaque end-to-end:
- primeCodeFiles silently dropped files without `metadata.codeEnvRef`
- reuploadFile catches all errors and continues with no signal
- the handlers.ts handoff to codeapi never logged what it was sending
After this patch, a single grep on `[primeCodeFiles]` plus
`[code-env:inject]` shows the full per-file path:
[primeCodeFiles] in: file_ids=N resourceFiles=M
[primeCodeFiles] file=<id> path=skip reason=no-codeenvref filename=...
[primeCodeFiles] file=<id> path=cache-hit-by-session storage_session_id=...
[primeCodeFiles] file=<id> path=reupload reason=no-uploadtime ...
[primeCodeFiles] file=<id> path=reupload reason=stale ...
[primeCodeFiles] file=<id> path=reupload-success oldSession=... newSession=... newFileId=...
[primeCodeFiles] file=<id> path=reupload-failed session=...
[primeCodeFiles] file=<id> path=fresh-active storage_session_id=...
[primeCodeFiles] out: returned=N skippedNoRef=M reuploadFailures=K
[code-env:inject] tool=<name> files=N missingResourceId=K (debug)
[code-env:inject] M/N files missing resource_id ... (warn)
[code-env:inject] tool=<name> _injected_files=0 ... (warn)
The boundary log warns when LC sends zero injected files on a
code-execution tool call — that's the user's actual symptom showing
up at the LC side instead of having to correlate against codeapi's
`Request received { files: [] }`.
Tag chosen as `[code-env:inject]` rather than `[handoff:exec]` to
avoid collision with the app-level "handoff" semantic (subagent
handoff workflow).
Structural cleanup in primeFiles: replaced the `if (ref) { ... }`
nesting with an early `if (!ref) continue` so the per-path
instrumentation hooks land at top-level scope instead of indented
inside a conditional. Behavior unchanged; pushFile / reuploadFile
identical.
Spec fixtures (handlers.spec.ts, codeFilesSession.spec.ts) updated
to include `resource_id` on `CodeEnvFile` literals — required by
the post-3.1.80-dev.2 type now installed.
## Test plan
- [x] `cd packages/api && npx jest src/agents/handlers.spec.ts src/agents/codeFilesSession.spec.ts src/agents/skillFiles.spec.ts` — 69/69 pass
- [x] `cd api && npx jest server/services/Files/Code/process.spec.js` — 84/84 pass
- [x] `npx tsc --noEmit -p packages/api` — clean
- [x] `npx eslint` on all four touched files — clean
* chore: add CONSOLE_JSON_STRING_LENGTH to .env.example for JSON log string length configuration
* fix(files): align codeapi upload filename with LC's sanitized DB filename
User-attached files for code execution were uploading to codeapi
under `file.originalname` (raw upload filename, may contain spaces /
special chars) while LC's DB record stored the sanitized form
(`sanitizeFilename(file.originalname)`, underscores). Codeapi
preserves whatever filename the upload sent, so the sandbox saw
`/mnt/data/<originalname>` while LC's `primeFiles` toolContext text
+ `_injected_files.name` referenced `file.filename` (sanitized).
Visible failure: agent gets system prompt saying
/mnt/data/librechat_code_api_-_active_customer_-_2025-11-05.xlsx
…tries that path, hits `FileNotFoundError`, then notices the
sandbox's actual `Available files` line says
/mnt/data/librechat code api - active customer - 2025-11-05.xlsx
…retries with spaces, succeeds. Wastes a tool call per upload and
leaks raw filenames into model context.
Fix: sanitize once and use the sanitized form in both the codeapi
upload AND the LC DB record. Sandbox path = LC toolContext text =
in-memory ref name. No drift.
Reupload path (`Code/process.js` line 867 `filename: file.filename`)
already uses the sanitized DB name, so it stays consistent with the
fresh-upload path after this change.
## Test plan
- [x] `cd api && npx jest server/services/Files/process` — 32/32 pass
- [x] `npx eslint` on the touched file — clean
* chore: bump `@librechat/agents` to version 3.1.80-dev.3 in package.json and package-lock.json
* chore: Update axios dependency to version 1.16.0 across multiple package files
* chore: Update express-rate-limit and ip-address dependencies to versions 8.5.1 and 10.2.0 in package-lock.json and package.json
* chore: Update mongoose and hono dependencies to versions 8.23.1 and 4.12.18 across multiple package files
* fix: Add type parameters to mongoose lean queries in accessRole and aclEntry methods
* fix: Add type parameters to mongoose lean queries in action, agent, and agentCategory methods
* chore: Update moduleResolution to 'bundler' in tsconfig.json for api and data-schemas packages
* fix: Update mongoose lean queries to include type parameters across various methods for improved type safety
* 📦 chore: bump `@librechat/agents` to v3.1.78
v3.1.78 ships [danny-avila/agents#147](https://github.com/danny-avila/agents/pull/147),
which makes `SubagentExecutor` inherit the parent invocation's
`configurable` (with `thread_id`/`run_id`/`parent_run_id` scrubbed)
into the child workflow. Subagent tool dispatches through the parent's
`ON_TOOL_EXECUTE` handler now arrive with parent's `requestBody`,
`user`, `userMCPAuthMap`, etc. — so `{{LIBRECHAT_BODY_*}}` placeholder
substitution and per-user MCP connection lookup work for subagent
tool calls the same way they do for the parent agent.
Note: `package-lock.json` will need an `npm install` refresh once
v3.1.78 lands on the registry. The user/user_id injection added in
PR #12950 stays as defense-in-depth.
* 🗑️ refactor: drop redundant user/user_id injection from `loadToolsForExecution`
`@librechat/agents@3.1.78` (via danny-avila/agents#147) makes
`SubagentExecutor` forward the parent's `configurable` verbatim into
the child workflow. Subagent `ON_TOOL_EXECUTE` dispatches now arrive
with parent's `user` / `user_id` already in `data.configurable` —
making the host-side injection added in #12950 a no-op.
Removes:
- The conditional `user: createSafeUser(req.user); user_id: req.user.id`
block in `loadToolsForExecution` (req.user.id-guarded so the
`'api-user'` fallback in Responses/OpenAI controllers is preserved).
- The unused `createSafeUser` import.
- The 4 unit tests covering the now-deleted behavior.
The merge in `handlers.ts` (`{ ...configurable, ...toolConfigurable }`)
still produces a `mergedConfigurable` with the right user identity for
both parent and subagent paths — the values just come from
`configurable` (forwarded by the SDK) rather than `toolConfigurable`.
Other fixes from #12950 stay (IUser.id narrowing, the env.ts /
google/initialize.ts / remoteAgentAuth.ts TS-warning fixes) — they
were independent of the subagent identity propagation issue.
* 📦 chore: update `@librechat/agents` to v3.1.78
This update reflects the transition from the development version `3.1.78-dev.0` to the stable release `3.1.78`. The package-lock.json has been refreshed to ensure consistency with the new version, including updated integrity checks and resolved URLs for the package. This change is part of ongoing improvements to enhance the functionality and stability of the agents module.
