488 commits

Author	SHA1	Message	Date
Danny Avila	250aca375a	🔗 fix: Resolve MCP Tool-Key Boundary Against Configured Server Names (#14448) ... * fix: resolve MCP tool-name delimiter collision at invocation time MCP tool keys are identified internally as `${rawToolName}${mcp_delimiter}${serverName}` (delimiter `_mcp_`). Several call sites parsed this back apart with a naive `toolKey.split(Constants.mcp_delimiter)`, assuming the delimiter occurs exactly once. When the raw upstream tool name itself contains the delimiter substring - which happens whenever it's exposed through a gateway that prefixes aggregated tool names by server (e.g. a gateway's own "gitlab-get_mcp_server_version" for GitLab's "get_mcp_server_version" tool) - the combined key has the delimiter more than once. `.split()` then produces more than two segments, and destructuring `[toolName, serverName]` silently keeps only the first two, yielding a bogus server name that matches no configured server. Tool listing still worked (a different code path builds keys directly without re-splitting), but invocation failed with `Tool {name} not found`, and `filterAuthorizedTools` rejected such keys outright as malformed. Add `splitMCPToolKey`, which splits on the *last* occurrence of the delimiter instead: the server-name half is always LibreChat's own normalized suffix (guaranteed not to contain the delimiter), while the raw tool-name half is untrusted and may legitimately contain it. This matches `.split()`'s result whenever the delimiter occurs once, and correctly resolves the collision case. Update the four call sites that parsed this manually (`handleTools.js`, `MCP.js`, `mcp.js` controller, `filterAuthorizedTools` in `v1.js`) plus one in the client (`useVisibleTools.ts`) to use it. Fixes #14440 * fix: resolve MCP tool-key boundary against configured server names splitMCPToolKey moves to librechat-data-provider so the client and backend share one parser, and takes the configured server names when the caller has them: the longest name the key actually ends with wins, which is exact. Position alone cannot identify the boundary because both halves may contain the delimiter. lastIndexOf alone fixes gateway-prefixed tool names but regresses servers whose own name contains it, which ToolService.spec.js already covered; the last-delimiter path now only serves as the fallback for callers with no configured set. Also converts the remaining first-occurrence parsers that the delimiter fix missed - mcp/auth.ts (custom user vars silently unresolved), mcp/oauth/events.ts, agents/initialize.ts, and the three client parsers that labelled tool calls with the wrong server. * fix: keep client tool-call labels on first-delimiter parsing The three client parsers had deliberate, tested first-delimiter semantics (ToolCall.test.tsx asserts the full server name for 'foo_mcp_bar' and the synthetic 'oauth_mcp_server' call), and the client has no configured server list in scope to resolve the boundary exactly, so they are left as they were. Threads the configured names into the event-driven definition loader so it resolves the same boundary as the authorization filter that admits the key, and documents the one case that stays undecidable without provenance. * fix: resolve tool-key boundary against all configured servers resolveConfigServers only returns lazily-initialized config overrides - ensureConfigServers skips unmodified YAML servers - so on a stock deployment the known-name list was empty and suffix resolution never engaged. Adds resolveMcpServerNames, which keeps every configured server in the normalized form tool keys carry, and uses it at the loading, auth-map and definition sites. Background-tool eligibility now resolves against all configured names before testing ephemeral membership, so a non-ephemeral server whose name ends in an ephemeral one is no longer misclassified, and useVisibleTools resolves against the server map it already receives. * fix: use resolved server provenance and one app-config read createMCPTool now uses the serverName loadTools already resolved for the key and only parses as a fallback, so an unmodified YAML server whose name contains the delimiter no longer resolves to the wrong server for auth, reconnection and callTool. resolveMcpServerContext derives config servers and all configured names from a single getAppConfigForRequest, replacing two independent lookups on the chat startup path, and degrades to empty like resolveConfigServers instead of aborting tool loading when the config lookup fails. * chore: drop unused resolveConfigServers import * fix: forward server provenance on the all-tools path and read config once createMCPTools builds each toolKey from the server name it already has but did not forward it, so the sys__all__sys path re-derived it by parsing and bound an unmodified YAML server whose name contains the delimiter to the wrong auth and invocation context. loadAgentTools now resolves the MCP server context once and threads it into loadTools, replacing the second app-config read it had introduced on the non-event-driven chat startup path. * fix: carry resolved MCP server name through tool classification definitions.ts resolves the server for each key and then dropped it when building loadedTools, so buildToolClassification re-derived it with a last-segment split and recorded 'Workspace' for a server configured as 'Google_mcp_Workspace'. The resolved name now rides along on the tool instance and classification prefers it over re-parsing. * fix: consume carried server name when extracting MCP servers extractMCPServers re-derived the name with a last-segment split, so a server configured as Google_mcp_Workspace resolved to Workspace and its instructions were silently omitted. Prefers the name carried on the tool definition instance, falling back to the split. * fix: fail closed on ambiguous MCP keys when persisting server names Persisted mcpServerNames grant agent-scoped access to a DB server by name (ServerConfigsDB.getAccessibleServers), so a wrong guess exposes an unrelated server to everyone who can view the agent. The last-segment split turned search_mcp_Google_mcp_workspace into 'workspace'; such keys were previously rejected outright at agent save, so admitting them opened this path. Derives a name only from unambiguous single-delimiter keys. This is #12250's guard moved to the boundary it was actually protecting, instead of blocking tool admission. * fix: keep DB server access for multi-delimiter tool keys The fail-closed guard was wrong for the case this PR exists to fix. This index only grants DB-backed servers, and DB names are slugs that cannot contain the delimiter (generateServerNameFromTitle strips underscores), so the trailing segment is always the real server for them - dropping it cost every consumer of a gateway-prefixed tool their shared-agent access. Also gates the MCP server-context lookup on the filtered MCP set, so an agent with no MCP tools no longer pays an app-config read on startup. * fix: resolve tool-call display names without breaking OAuth calls The display parsers could not use the shared boundary parser because their tested behavior depends on first-delimiter semantics. That constraint only applies to synthetic MCP OAuth calls, whose tool half is always exactly 'oauth', so everything after the first delimiter is the server even when the server name carries one. splitToolCallName special-cases that form and defers to splitMCPToolKey for real tool keys, so a gateway-prefixed tool now renders its own name and server while oauth_mcp_foo_mcp_bar still resolves to foo_mcp_bar. * fix: persist resolved MCP server provenance on agents Deriving mcpServerNames from the tool key cannot tell a config server's trailing segment from a real DB server name, so a config server named a_mcp_b indexed an unrelated DB server b and shared the agent's viewers into it. Neither string rule works: the suffix guess exposes, and failing closed drops legitimate DB access for gateway-prefixed tools. filterAuthorizedTools already resolves each tool's server against the merged registry config, so it now collects those names and create, update and duplicate persist them. No extra registry queries: the update path unions the newly resolved names with what the agent already had, and duplicate replaces the copied list rather than inheriting the source's servers. Display parsing also takes the configured names, so a real tool call on a delimiter-bearing server renders the right server and icon. * test: teach MCP hook mocks about useMCPServerNames Three specs mock ~/hooks/MCP with a hand-listed factory, so adding the hook to ToolCall made useMCPServerNames undefined under test and every render threw. Returns a stable array so the mock cannot perturb render counts. * fix: rebuild agent MCP server index from surviving tools Unioning the prior names kept a server indexed after its last tool was detached, so viewers of a shared agent retained agent-scoped access to it. The index is now rebuilt from the tools that survive the edit: a prior name carries forward only while some retained tool still resolves to it, using the agent's own persisted names as the candidate set, and the rebuild runs on any tool change rather than only when a new MCP tool is added. * fix: keep duplicate indexes on registry fallback and harden the oauth split Duplication blanked mcpServerNames when the registry was unavailable, because filterAuthorizedTools grandfathers the source's tools without resolving them - the copy kept tools it could no longer resolve. Source names now carry forward for the tools that still point at them. splitToolCallName also treated any oauth_mcp_ prefix as a synthetic OAuth call, so a genuine upstream tool by that name resolved to the wrong server. A configured server name now decides when one matches, since a real key always ends in its server, and the prefix only breaks ties for unconfigured servers. * fix: thread configured server names through display parsing parseToolName and getMCPServerName resolved context-free, so a configured server whose name contains the delimiter showed the wrong server in grouped tool summaries and subagent tool labels, and stacked icons missed its entry in the icon map. Both take the configured names now, supplied by the components that render them. Adds the hook to SubagentCall's mock factory: the spec renders the real component, so an unmocked useMCPServerNames would reach the query with no provider. * test: cover the auth-map boundary, server provenance and context fallback Adds regression coverage for three behaviors this PR changed that no test exercised: customUserVars resolving under the right plugin key for a gateway-prefixed tool name (the failure that made these tools loadable but unusable), the resolved server name reaching createMCPTool instead of being re-parsed, and resolveMcpServerContext degrading to empty rather than aborting tool loading when the config lookup fails. Each was checked against a mutated source to confirm it fails when the behavior is broken. * fix: normalize server-name candidates and cover the boundary guard Tool keys embed normalizeServerName's output while the config is keyed by the raw name, so callers passing raw keys never matched a server whose name needs normalizing and silently fell back to the last delimiter. filterAuthorizedTools now maps normalized names back to their config key, and createMCPTool normalizes its candidates. Adds the cases an audit found surviving mutation: a configured name that is a bare but not delimiter-aligned suffix must not match, an empty candidate list behaves as no list, and splitToolCallName still falls back to the oauth prefix when a list is supplied but nothing in it matches. * fix: keep resolved server names when a non-owner retains MCP tools The shared-agent path keeps an agent's existing MCP tools verbatim but supplied no mcpServerNames, so persistence re-derived them and reduced a configured server like Google_mcp_Workspace to Workspace - which ServerConfigsDB then treats as a DB server, granting the agent's viewers access to an unrelated one. Carries the existing resolved names across instead, and clears the index on the owner path where every MCP tool is removed. * fix: preserve resolved MCP names for every tools update extractMCPServerNames was reachable from any caller that writes tools without mcpServerNames - the Action edit path does exactly that - so a configured Google_mcp_Workspace was reindexed as Workspace and ServerConfigsDB granted shared-agent viewers an unrelated DB server by that name. updateAgent now rebuilds the index from the agent's own resolved names: one carries forward while a retained tool still resolves to it, and only keys matching none of them fall back to derivation. Callers are safe by default rather than by remembering to pass the set. normalizeServerName moves to librechat-data-provider so the client can match its candidates against tool keys, which embed the normalized form; the icon map is keyed the same way since it is looked up with a parsed server name. * refactor: move MCP context resolution into packages/api New backend logic belongs in the TypeScript workspace per CLAUDE.md, with /api kept to a thin wrapper. resolveMCPServerContext now lives in packages/api/src/mcp/context.ts and takes ensureConfigServers by injection, since the registry accessor is still legacy-only; the /api function is reduced to loading the request app config and translating failures into the empty degrade it already promised. * test: teach the MCP service mock about resolveMCPServerContext The spec mocks @librechat/api with a hand-listed factory, so moving the resolver into that package left it undefined and the wrapper degraded into its own catch, returning empty config servers. The stub mirrors the real resolver so these tests still cover what the wrapper owns - loading the request config and degrading on failure - while the resolution logic is unit-tested in packages/api. * fix: only persist an authoritative MCP server index on update Assigning the resolved set unconditionally pinned the index to [] whenever nothing authoritative was available - a legacy agent holding MCP tools with no stored mcpServerNames - which suppressed updateAgent's derivation and stripped agent-scoped access to its DB-backed server. The field is now supplied only when the result is authoritative: names were resolved, or no MCP tool survives so the index genuinely is empty. The retained-tools branch likewise leaves it unset when the agent has none stored. --------- Co-authored-by: Jens Schumann <schumajs@gmail.com>	2026-07-27 14:45:38 -04:00
Danny Avila	a53936d273	🧭 test: Cover Agent Handoffs End to End (#14428) ... * test: cover agent handoffs end to end * style: sort handoff imports * fix: normalize missing agent handoff edges * chore: update package dependencies and versions in package-lock.json and package.json * chore: bump agents SDK	2026-07-27 08:47:15 -04:00
Danny Avila	d8427ffc5e	🛂 test: Cover Tool Approval Workflows End to End (#14427) ... * test: cover tool approval workflows end to end * fix: preserve tool approval state across resume * fix: preserve agent context in mock stream responses * fix: preserve nested approvals in collapsed groups	2026-07-26 21:58:25 -04:00
Danny Avila	73699b5c25	⚡ perf: Reduce Agent Chat Startup Latency (#14423) ... * perf: reduce agent chat startup latency * test: align Redis stream readiness assertions * perf: overlap remaining agent startup work * perf: persist initial agent job metadata atomically * test: add agent startup latency benchmark * fix: harden resumable agent stream lifecycle * fix: isolate replacement stream lifecycles * fix: preserve terminal stream epochs	2026-07-25 07:58:20 -04:00
Danny Avila	00c5a747e9	🧵 feat: Native Background Execution for Code Interpreter Tools (#14386) ... * 🧵 feat: Native Background Execution for Code Interpreter Tools * 🩹 fix: Address Codex Round 1 (fallback dedupe, harvest failure, handle parsing) * 🩹 fix: Live Completion Marker + Unkeyed Attachment Dedupe (Codex Round 2) * 🎨 chore: Sort Imports + Widen Marker Type Comparison (CI) * 🩹 fix: Stale-Harvest Guard, Error Marker Status, Faster Anchor Retry (Codex Round 3) * 🧹 refactor: TS Harvest Module, Claim-Neutral Timestamps, Error Parity (Codex Round 4) * 🩹 fix: Dispatch-Ordered Stale Guard, Foreground Downgrade, Error Wrapper Parity (Codex Round 5) * 🩹 fix: Retry Past Unfinished Rows + Per-Call Attachment Dedupe (Codex Round 6) * 🩹 fix: Writer-Dispatch Ordering, Scoped Live Upserts, Reaped-Task Wrapper (Codex Round 7) * 🩹 fix: Wildcard toolCallId Matching for Bare Attachment Updates (CI) * 🩹 fix: Claim-Insert Dispatch Stamp (Schema-Backed) + Scoped Status Markers (Codex Round 8) * 🩹 fix: Pre-Write Ownership CAS + Agent-Scoped Part Patching (Codex Round 9) * 🩹 fix: Insert-Path Ownership CAS + Agent-Routed Attachments (Codex Round 10) * 🩹 fix: Agent-Scoped Marker Ids and Attachment Dedupe (Codex Round 11) * 🩹 fix: Atomic File Commit and Sibling Preview Fan-Out (Codex Round 12) - Replace the two-step claim-confirm CAS with an atomic conditional updateFile: the ownership predicate (no sourceDispatchedAt, or <= this write's dispatch order) moves into the update filter, removing confirmCodeFileOwnership and the lost-update window between check and write - Thread agentId through createDownloadFallback so fallback download rows scope to the emitting agent like primary rows - Fan terminal preview overlays out to every live attachment sharing the file_id in useAttachmentPreviewSync (sibling tool calls no longer stick on pending) - Restore background artifacts through toStoredArtifact so the size bound applies on re-anchor - Apply filterAttachmentsForPart to grouped tool-call attachments in ContentParts so handoff agents with colliding provider call ids do not cross-contaminate groups * 🩹 fix: Agent-Scoped Live Upserts and Monotonic Dispatch Stamps (Codex Round 13) - Scope the SSE attachment upsert and the useAttachments DB/live merge by agentId with the same wildcard semantics as toolCallId: distinct non-null agentIds stay separate entries, so handoff agents sharing a claimed file_id and a repeated provider tool id (call_0) no longer merge over each other's cards - Extend the attachment identity key to fileKey::toolCallId::agentId and register less-specific key variants so bare and agent-less live records still dedupe after overlay - Stamp background task createdAt from a strictly-increasing per-process dispatch counter: raw Date.now() can tie for same-millisecond dispatches and the stale-output guard accepts equal stamps (needed for idempotent re-commits), which would let an older task overwrite a newer task's committed file	2026-07-22 22:13:15 -04:00
Danny Avila	ad5bb477af	🎞️ fix: Surface Clear Error for Unprocessable Gemini YouTube Videos (#14396) ... Google rejects a YouTube video it cannot ingest with a generic `400 INVALID_ARGUMENT` that names no cause, which LibreChat relayed verbatim. Attribute the failure using request context instead: when a Google/Vertex turn carried an injected YouTube video part and the provider returns that generic rejection, map it to a typed error the client localizes. Verified against the live API: a public 9h15m video is refused this way on gemini-2.5-flash, 3.5-flash, 3.5-flash-lite and 3.6-flash, including at MEDIA_RESOLUTION_LOW, while a short video with an identical payload succeeds. Duration is the dominant trigger; region and access restrictions return the same response, so the copy leads with length without overclaiming. A duration preflight was evaluated and skipped: oEmbed does not expose duration, leaving only watch-page scraping — a blocking call against undocumented markup from rate-limited datacenter IPs that would fail open and still need this mapping underneath.	2026-07-22 12:11:06 -04:00
Danny Avila	3e9f07976a	🧩 fix: Preserve Deployment Skill IDs on Agents (#14368) ... * fix: preserve deployment skills on agents * fix: expose deployment skills to agent viewers * refactor: centralize deployment skill ID merging --------- Co-authored-by: Dennis Schenk <dennis@gridonic.ch>	2026-07-21 19:44:27 -04:00
Danny Avila	9e245aced4	🎟️ fix: Claim Idempotency Keys to Dedup Retried Generation Requests and Prevent Double Billing (#14344) ... * 🐛 fix: Dedup retried start-generation requests to prevent duplicate billing A lost or reset start-generation response makes the client re-POST the identical payload (up to 3x on network errors). The resumable-stream controller had no idempotency: createJob unconditionally overwrote the running job without aborting the prior one, so both requests ran full LLM completions and both billed while the UI showed only one (#14339). Add a stable per-submission clientRequestId (uuid, fresh per ask() so a regenerate differs, reused across the start-generation retries) and an atomic claim on the job store keyed by userId:clientRequestId. The first request wins and generates; a retried POST loses the claim and receives the original stream, which the client subscribes to and replays - no second billed generation. - IJobStore.claimIdempotencyKey/releaseIdempotencyKey (in-memory Map+TTL, Redis single-key SET NX PX + GET Lua, cluster-safe) - GenerationJobManager.claimGeneration/releaseGeneration (20m TTL) - Controller claims before the concurrency check, dedups with a resumed response, releases on start-failure/429 - clientRequestId threaded through TSubmission/TPayload/createPayload * 🐛 fix: Harden start-generation dedup (Codex review) Address three P2 findings on the idempotency path: - Resume replay: a deduped retry now subscribes with resume=true so the client replays prior content and any pending-action from the running stream instead of only live events (cross-replica / HITL correctness). startGeneration returns { streamId, resumed } and the response's status:'resumed' drives the subscribe mode. - Wait for the job record: a duplicate that loses the claim now waits briefly for the winner to create the job before returning the stream (a stream with no job 404s terminally). If the winner has not materialized, return 503 SERVER_NOT_READY so the client retries via the existing readiness path instead of attaching to a dead stream. - Release only owned claims: track whether the request actually won the claim; the 429 and init-error paths no longer release a claim owned by another in-flight generation (fail-open path could erase it and re-enable double billing). Adds controller tests covering dedup, the 503 race fallback, win-then- create, and claim-release ownership on 429 / fail-open. * 🐛 fix: Don't trap deduped retries on missing job records (Codex review) The previous round returned 503 SERVER_NOT_READY when a deduped retry's job record was absent. But a missing job usually means the original generation already completed and was cleaned up (cleanupOnComplete) — the correct recovery is to return the stream and let the client's subscribe 404 handler refetch the persisted messages. The 503 instead trapped the send in a readiness-retry loop until the client's window expired. Keep the bounded wait (it still covers the job-about-to-be-created race) but always return the resumed stream afterward; a gone/never-created job recovers via the client's existing 404 path instead of being treated as indefinitely starting. Updated the controller test accordingly. * 🐛 fix: Gate deduped resume on claim age, not just job presence (Codex review) Removing the 503 entirely (previous round) reintroduced the inverse race: if the winning request stalls between claimGeneration and createJob, a losing duplicate saw no job, returned status:'resumed' anyway, and the client subscribed to a stream that did not exist yet — the 404 handler tore the turn down while the winner went on to generate and bill with no UI attached. Distinguish the two missing-job cases by claim age (claimedAt now travels on the claim value): - fresh claim, no job yet → winner is still starting → 503 SERVER_NOT_READY so the client retries via the readiness path (bounded, not indefinite). - old claim, no job → the original already completed and was cleaned up (or the winner died) → attach; the client's 404 handler refetches. Tests cover both age branches. * 🐛 fix: Scope dedup fail-open + keep resumed convos on 404 (Codex review) - Fail-open only on claim acquisition: a store error while checking an already-confirmed existing claim no longer falls through to createJob (which would start a second billed generation during a Redis hiccup). Once claim.existing is known, a job-lookup error returns 503 retry. - Don't drop a resumed convo on 404: the optimistic-conversation cleanup in useResumableSSE now runs only for fresh (non-resume) subscribes. A deduped resume whose original completed and was cleaned up 404s, but its conversation is persisted and must stay in the sidebar. Adds a controller test for the job-lookup-error path (503, no createJob). * 🐛 fix: Reconcile resumed convos on 404 instead of guessing (Codex review) Round-4's !isResume guard fixed the completed-and-cleaned case (don't drop a persisted convo) but left the inverse: a new-conversation retry deduped to a claim whose original worker died before persisting still resumes, 404s, and — with removal skipped — leaves a phantom /c/<streamId> sidebar entry. Stop guessing keep-vs-remove on a resume 404. Reconcile against the server: invalidate the conversations list so a real (persisted) convo stays and a phantom is dropped. Fresh (non-resume) optimistic streams still prune immediately. Adds a client test for the resume path. * 🐛 fix: Finalize failed job before releasing its claim (Codex review) In the initialization-error catch, the idempotency claim was released before completeJob(streamId). A racing retry could win the released key and createJob() the same streamId while this catch was still running, and completeJob() (not guarded by the original createdAt) would then abort the replacement. Finalize the failed job first, then release the claim. Adds a controller test asserting completeJob precedes releaseGeneration. * 🐛 fix: Clear claims on destroy + survive completeJob failure (Codex review) - InMemoryJobStore.destroy() now clears the idempotencyClaims map, so a reused/reconfigured store instance doesn't dedup a fresh start against a torn-down job's stale claim. - Init-error cleanup: completeJob() is swallowed so a store-hiccup rejection can no longer skip the idempotency-key release and the pending-request decrement (which would wedge the retry behind the claim and leak the concurrency slot). A failed completeJob finalized nothing, so releasing afterward still can't abort a later replacement. Tests: claims cleared on destroy; release + pending decrement still run when completeJob rejects.	2026-07-21 08:16:31 -04:00
Danny Avila	7447fddfb2	🙊 refactor: Clarify Ask Question Schema Errors and Retry Guidance (#14279) ... * fix(agents): clarify ask question validation errors * fix(agents): narrow question failure detection * fix(agents): persist question validation failures * fix(agents): track question validation failures	2026-07-15 11:06:29 -04:00
Jaime Hidalgo	e813934731	🔖 fix: Preserve Ephemeral Agent Params and Identity for ask_user_question Resume (#14254) ... * fix: durable ask_user_question resume for ephemeral agents * 🤖 refactor: Drop chat.js resume hunks in favor of shared packages/api helpers * 🤖 fix: Normalize resume thinking param and replay modelLabel (#14253 Bugs 1&2) * 🤖 fix: Preserve adaptive thinking display and effort across HITL resume * 🔤 style: Sort load.spec.ts imports (repo import-order) * 🤖 fix: Replay paused request body params on HITL resume (UI-form source of truth) --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-07-14 15:12:43 -04:00
Danny Avila	5771bf6e06	♨️ feat: Prewarm Stateful Code Sandboxes with Cold-Boot UX Feedback (#14239) ... * ♨️ feat: Prewarm Stateful Code Sandboxes with Cold-Boot UX Feedback * 🧹 fix: Drain Prewarm Response + Reset Sandbox Atoms on Stream Cleanup * 🚿 fix: Propagate Prewarm Drain Failures + Warm Marker for Host File Tools * 🌡️ fix: Decouple Prewarm In-Flight State from Warm Refreshes + Precise Ready Gates * ☁️ refactor: Redis-Backed Sandbox Prewarm State via standardCache * 🧪 chore: Hermetic Prewarm Spec + Accurate Signal JSDoc (Copilot review)	2026-07-14 10:25:37 -04:00
Danny Avila	9bb351ad9c	🧭 feat: Mid-Run Steering and Queued Messages for Agent Runs (#14220) ... * 🧭 feat: Mid-Run Steering and Queued Messages for Agent Runs Steering: submit a message while a run is generating; the server queues it in the job store (cross-instance) and a run-scoped PostToolBatch hook injects it into graph state at the next tool-batch boundary, records an inline 'steer' content part on the response (replayed as a user message on later turns), and streams on_steer_applied to the client. Queuing: messages composed during a run auto-send as normal follow-up turns after clean completion (one per final event, FIFO); user aborts leave them as chips unless armed by interrupt-and-send. Requires hook injectedMessages support in @librechat/agents (danny-avila/agents#299); hard-gated via a capability probe so older SDKs 501 the steer route instead of draining and dropping messages. * 🧵 fix: Harden Steering Against Finalization Races and Route Guard Gaps Addresses local Codex review findings on the steering feature: - Close-and-drain the steer queue atomically at finalization (final event, abort) so a steer POST racing teardown is rejected instead of 202-ACKed and then silently cleared; the closed flag lives on the job hash and is reset when a replacement job reuses the stream id. - Clear inherited steer queues on createJob — a job replacement must not drain the replaced run's messages. - Keep steers queued across a HITL pause instead of draining them into ephemeral client state: resumeState re-seeds chips on reload and the resumed run injects them at its first tool boundary (steers key TTL now extends to the approval window; on_steers_pending event removed). - Queue the NO_ACTIVE_RUN steer fallback while the final SSE is still settling — a direct send would be dropped by ask()'s in-flight guard. - Reconcile the 202 ACK against on_steer_applied events that beat it over the SSE, so a chip can't be re-minted after its removal event passed. - Allow the per-send Steer override when the default action is queue. - Apply the configured message rate limiters and the PII filter to POST /chat/steer — a steer is model-bound user text. * ✅ ci: Assert Steering Capability Probe Against the Installed SDK CI installs the published @librechat/agents pin (pre-injectedMessages), where isSteeringSupported() is legitimately false — the probe test now asserts it mirrors the installed SDK's capability flag instead of hardcoding the capability-bearing build's value. Verified against both the published 3.2.61 dist and the agents#299 build. * 🛟 fix: Preserve Steer Text Across Run-End, Error, and Abort Races Codex round 2 (4 P2s): - Applied-steer-id set survives run end (capped at 100) and converted ids join it, so a 202 ACK that lands after final/abort drops its chip instead of re-minting a stranded pending one. - Failed runs no longer strand acknowledged chips: both error paths convert local pending chips to queued follow-ups (chip text is client-local), and the server closes the steer queue before emitting the error so a racing steer POST gets 404 fallback instead of a 202 whose payload dies with the job. - sendQueuedNow keys on steer availability, not the default action — send-now on a queued chip is an explicit override for queue-preferring users. - Stop path consumes pendingSteers from the abort HTTP response as a fallback for the SSE final event it may close before processing; conversion is deduped so double delivery is a no-op (shared useSteerConvert hook). * 📎 feat: Carry Attachments Through During-Run Queued Messages Steering stays text-only (SDK injection, inline STEER part, and replay are all text), so a during-run submit with media now queues the whole message as one unit instead of silently stranding the files: - QueuedMessage gains `files`; composer attachments are consumed into the queued item at queue time (steerFromComposer / queueFromComposer / interruptAndSend), fixing the latent hazard where lingering composer files glued onto whatever `ask` vacuumed up next. - Enter-steer with attachments degrades to queue with an explanatory toast; the per-send menu routes through the same composer-aware wrappers. - The drain and sendQueuedNow pass the item's files as `overrideFiles`; media items never steer (send as a normal turn when idle, re-front otherwise). ask() no longer clears composer state for caller-supplied overrideFiles — only regenerate keeps that behavior. - During-run submits hold while uploads are in flight, mirroring the send button's filesLoading gate; queued chips show a paperclip count. * 🎛️ feat: Rework During-Run Chips into Action Rows Full-width rows above the composer (reference-UI parity): each queued message shows a primary Steer/Send-now action, delete, and a "…" menu with Edit message (restores text + attachments into the composer) and a Turn on queueing/steering toggle that flips the Enter default. Steer rows share the layout with status text; failed steers keep retry / edit / queue-convert. The per-send menu gains the same default toggle. Queued file refs now retain filename + bytes so edit-restore rebuilds real composer entries (draft-recovery shape). * 🖇️ feat: Steer With Attachments (Multimodal Mid-Run Injection) Steering now carries media end-to-end instead of degrading to queue: - The steer POST accepts sanitized attachment refs (cap 10; only file_id is trusted — the drain re-fetches owner-scoped and re-derives everything else). SteerQueueItem/TPendingSteer/SteerContentPart carry `files` refs; encoded data is never persisted or queued. - New api/server/services/Files/steering.js decouples attachment building from the request path: encodeSteerContent reuses the exact per-turn pipeline (addFileContextToMessage + processAttachments' single-pass categorize/encode, SDK formatMessage assembly, prependFileContext for extracted text) with zero new encoding code. buildSteerMedia feeds the drain hook's new buildMedia seam (any failure degrades that steer to text-only — words always land); stampSteerPartMedia re-encodes past steer parts per turn with ONE batched owner-scoped fetch and stamps a transient `media` array, replaced immutably so it can never leak into a save. Replay honors resendFiles like regular message media. - The SDK's formatAgentMessages (the formatter agents actually use) gained the steer replay branch on the PR branch; the local formatMessages.js branch now mirrors the media preference. - Client: steerFromComposer consumes composer files into the POST, chips/seeding/conversions carry files everywhere (retry, queue convert, abort/error recovery), queued media items steer for real, and SteerBubble renders the steered attachments inline. * 🧵 fix: Harden Steer Recovery Races and Drain Isolation Codex round 3 (7 fixes): - A 202 ACK landing after the run ended converts straight to a queued follow-up (server queue is gone; no event will ever resolve a pending chip for a finished run). Covers stream errors with in-flight POSTs. - A Stop that lands pre-completion can arrive as a final with unfinished:true and no aborted flag — runEnd now treats it as aborted so queued messages are not auto-sent against the user's Stop. - Leftover-steer conversion merges chronologically by createdAt instead of appending, preserving the order the user composed. - Auto-drained queued messages pass explicit (possibly empty) overrideFiles/overrideQuotes/overrideManualSkills: a drain can no longer vacuum up files, quotes, or skill picks staged in the composer for the user's NEXT message (ask() treats overrideFiles != null as authoritative). - Failed-steer Retry and resume-on-load chip restoration keep the steer's attachments. - The job-replacement guard moved INSIDE the store's atomic drain/close-and-drain (Lua createdAt compare; in-memory equivalent): a stale run's hook or finalization can neither consume, close, nor steal a replacement job's steer queue, and the drain hook drops its separate check-then-drain round trip. * 🧰 refactor: Typed Steer Controller, Single-Query Media Pass, Round-4 Fixes Codex round 4 + efficiency tightening in one pass: - Moved the steer guard ladder (validation, file sanitization via a shared toSteerFileRef picker, ownership/tenant checks, status-guarded enqueue) into packages/api as handleSteerRequest; api/steer.js is now a thin wrapper. Ladder covered against the REAL in-memory job manager in request.spec.ts; the api spec pins only the wrapper contract. - Folded the steer replay stamp into the turn's ONE historical-files query: collectHistoricalFileRefs also gathers steer-part refs, the owner-scoped doc map rides client state, and stampSteerPartMedia consumes it (no second round trip) while encoding parts in parallel. - Stamped steer media now counts against the run budget (existing multimodal counter over the non-text parts, folded into indexTokenCountMap/promptTokens after the stamp). - Steer route runs the PII filter BEFORE moderateText, matching chat.js so blocked sensitive text never reaches the external moderation API. - Interrupt & send survives the abort-response-beats-SSE-final race: stopGenerating writes the run-end signal itself when the one-shot interrupt flag is armed and no signal landed (double-fire safe). - Resume reconciles chips against the server's still-queued list even when EMPTY, clearing chips for steers applied while disconnected. - The local formatter's steer flush preserves non-text assistant parts (array-content AIMessage) instead of folding to text. * 🔒 fix: Replay-Aware Capability Gate and Round-5 Race Closures - isSteeringSupported now requires BOTH halves of the SDK contract: injection (HOOK_INJECTED_MESSAGES_CAPABLE) AND replay (ContentTypes.STEER, shipped in the same SDK commit as the formatAgentMessages steer branch). An SDK that can inject but not replay 501s the steer route — no release window can create steer parts that would leak into provider-facing assistant content. - The local formatter mirrors the SDK's anchor reset: a post-steer tool_call mints a fresh AIMessage instead of attaching to the pre-steer anchor (invalid provider ordering). - Queued-chip send-now and the NO_ACTIVE_RUN fallback pass explicit (possibly empty) overrideFiles so an idle send can't vacuum composer files staged for a different draft. - Redis createJob deletes the stale steer list BEFORE the replacement hash is written as running — a steer 202-accepted against the new job can never be wiped by the reset. - Resumed-turn finalization mirrors the normal path's terminal drain: createdAt-guarded close-and-drain, leftovers ride the resumed final event as pendingSteers instead of being cleared by completeJob. - buildSteerMedia restores composer order over the $in result so multi-attachment steers reach the model in the order the user saw. * ⚛️ fix: Atomic Job Replacement and Boundary-Clean Steering Module Codex round 6 (5 fixed, 1 standing deferral): - createJob resets the steer queue and writes the job hash in ONE same-slot Lua script (JOB_CREATE_LUA): a steer POST can no longer interleave between them on cluster, so a steer accepted against one run can never be drained into another. Redis-validated. - The steering media pipeline moved to packages/api (agents/steering/media.ts) with injected getFiles and a structural client interface — /api keeps zero steering logic; specs ported to the DI seam. - handleSteerRequest checks the job BEFORE the capability gate: a steer racing completion on an unsupported SDK gets 404 (send-now) instead of a 501 queue with no run-end signal left to drain it. - useQueueDrain binds to the active conversation: navigating away between the final SSE and the drain effect leaves the signal unconsumed instead of submitting A's follow-up into B; the drain fires on return. - abortJob closes and drains the steer queue BEFORE the content snapshot, so a drain-hook apply that lands pre-drain is captured inline rather than lost between the snapshot and the terminal drain. * 🚦 fix: Parked Run-End Signals, Interrupt Priority, Settled-Run Fallbacks Codex round 7 (5 fixes): - Run-end signals for a non-active conversation are PARKED per conversation instead of squatting the shared index slot: a later run finishing on the same pane can no longer overwrite them, and the parked drain fires when the user returns. - "Interrupt & send" front-inserts carry a priority flag that outranks createdAt when abort leftovers merge back chronologically — the urgent redirect drains first, not the oldest steer. - STEER_UNSUPPORTED/RUN_PAUSED/QUEUE_FULL rejections landing after the run settled mirror the NO_ACTIVE_RUN fallback and send immediately (queueing would strand the text with no run-end signal left); on the pinned SDK this is the common Enter-near-run-end path. - A failed abort (e.g. 404 when the run completed first) still signals the interrupt drain, so the queued interrupt message can't strand and the armed flag can't leak onto a later run. - Steered-image fallback alt text is localized (com_ui_attached_image). * 📌 chore: Adopt Published @librechat/agents Types Post-Bump dev's pin bump to ^3.2.62 (the release carrying injection + steer replay) landed via merge; the steering runtime now uses the SDK's real InjectedMessage/hook-output types instead of the local structural mirrors that bridged the pre-publish window. The two-half capability probe stays as the defensive gate for mismatched deployments — and the capability spec now exercises its TRUE path against the published package in CI. * 🛅 feat: Park-and-Claim Steer Recovery + Host-View Content Reads Codex round 8 (6 fixed incl. both P1s, 1 push-back): - The long-deferred no-subscriber gap is closed: every terminal drain (final, aborted-final, error, abortJob, resumed finalize) PARKS acknowledged leftovers on the job hash (unrecoveredSteers), and the status route claims them exactly once for inactive jobs — a client that closed/reloaded past the transient final event restores its steers as queued chips within the post-terminal TTL. A replacement run clears the parked copy (a live client started it). - Same-instance content reads are steer-complete: RedisJobStore now caches the HOST content array (WeakRef) via setContentParts and prefers it over the SDK graph cache, whose view never contains host-authored steer parts; the graph fallback splice-INSERTS steer chunks at their recorded host-view indices (the graph array is unshifted, so assignment would overwrite SDK parts). - Replay token accounting now counts prepended file-context text: full stamped content minus the steer body (already counted), so large steered documents hit the budget instead of bypassing pruning. - The queue drain restores an item when ask() refuses without sending (history not yet in cache after navigating back) — text is never silently dropped. - The armed interrupt flag travels WITH a parked run-end signal, so another run on the same pane can neither consume nor clear it. - parseTextParts extracts steer text (search indexing / audio). * 🎛️ refactor: Single Send Slot + In-Thread Steer Messages - Merge the during-run send affordance into the send/stop button slot: with composer text the send button replaces Stop (Enter = default action), hover reveals Steer/Queue/Interrupt rows with shortcuts; drop the separate DuringRunActionsMenu chevron - Add during-run keyboard chords: Cmd/Ctrl+Enter = non-default action, Alt+Enter = interrupt & send (plain-Enter submitters only) - Render steers as standard user messages in the thread: SteerPart (icon + author header + user text presentation) replaces the SteerBubble, and submitted steers appear immediately at the projected injection point via the PendingSteers slot on the streaming message - Keep composer rows only for recoverable states: failed steers (retry/edit/queue) and queued follow-ups * 🩹 fix: Keep the Replacement Submission Alive Across Abort Settlement The aborted run's final SSE event fires before the abort HTTP response resolves, so an armed interrupt & send drains and starts the NEXT submission while the abort POST is still in flight. The response handler's unconditional clearAllSubmissions() then reset the new submission, aborting its stream attach before the subscribe — the follow-up ran and persisted server-side but the live placeholder finalized empty (content appeared only after reload). useAbortCleanup captures the submission before the abort round-trip and both settlement paths (success and 404-catch) clear only when the captured submission is still current; a replacement stays untouched. Plain Stop behavior is unchanged. * 🧭 test: Playwright E2E for Mid-Run Steering and Queuing - Add e2e/specs/mock/steering.spec.ts: steer mid-run (202 + immediate in-thread pending part + real MCP tool boundary + words survive run end), Cmd/Ctrl+Enter queue with auto-send after clean completion, and Alt+Enter interrupt & send with the follow-up streaming into the live view - Add the E2E_STEER_TOOL_REPLY fake-model marker: slow preamble, a real remember_fact MCP tool call (PostToolBatch boundary), then a final turn - Test 1 pins the run-end degradation contract while the SDK's top-level agentId stamping bug blocks live injection; its header documents the assertions to flip once the fixed SDK is pinned * 🧷 fix: Job-Independent Steer Recovery + Expiry and Resume-Gap Parking Codex round 10: the park-and-claim recovery had lifecycle holes. - Move parked steers off the job hash onto their own bounded-TTL store key (JOB_CREATE_LUA resets it; deleteJob leaves it alone): the default completeJob path deletes the job record immediately, and the Redis read path never deserialized the old hash field — recovery previously worked only with STREAM_KEEP_COMPLETED_JOBS on the in-memory store - Carry the owner identity inside the parked payload and authorize the claim against it, so the status route recovers steers on its jobless branch too (the common reload-after-terminal case); a non-owner claim returns nothing and re-parks the payload - Park queued steers on approval expiry: snapshot the frozen queue before the requires_action→aborted CAS (whose terminal cleanup drops the steers key) and park only when the CAS wins - Mirror the terminal drain/park block in resume.js's failure path, which previously let completeJob's backstop clear 202-accepted steers - Close the Redis snapshot→subscribe resume gap: re-peek the queue after attaching and re-surface missed on_steer_applied events from the durable content view (synthesizeAppliedSteerEvents), updating resumeState.pendingSteers to the live queue * 📌 chore: Require @librechat/agents 3.2.63 + Applied-Steer E2E Contract - Bump the @librechat/agents pin to ^3.2.63 in api/ and packages/api/: it scopes the hook agentId marker to subagent child graphs, so the steering drain hook fires at top-level tool-batch boundaries and mid-run injection is active (danny-avila/agents PR 307) - Flip e2e steering test 1 from the documented degradation contract to the applied-steer contract: the optimistic in-thread part transitions to the persisted part at the tool boundary and survives inside the response after run end, with no queued follow-up turn * 🎗️ feat: Steered Messages Join the Message-Nav Ribs Steers are user messages, so they get their own clickable rib on the navigation rail, interleaved at their in-thread position inside the response that absorbed them (one DOM query in document order). SteerPart anchors itself as #steer-<id> with a steer-render marker — both the optimistic pending entry and the persisted part — and the rib carries the user role label with a preview drawn from the steer's text body, skipping the author header. * ❎ feat: Cancel a Queued Steer Before Injection + True User-Message Alignment - Add POST /chat/steer/cancel: removes ONE still-queued steer by id via an atomic list rebuild (Redis Lua preserves order and TTL), authorized against the job owner; removed:false is advisory — the cancel lost its race to the drain or the run end, never an error - Surface an × on the in-thread pending steer (server-acknowledged entries only): optimistic removal, restored if the POST fails since the server would still inject the words - Outdent SteerPart past the response's icon column so steers sit flush with top-level message rows, reading as regular user messages * 🧯 fix: Round-11 Recovery Hardening + Provider-Free Pending Slot - Reconcile the resume steer gap by steerId SETS, not queue length — a steer added in the gap (or an equal-length drain+enqueue swap) now refreshes resumeState.pendingSteers and still synthesizes the missed on_steer_applied events - Make completeJob's terminal backstop park: direct error-path callers without the controllers' close-and-park no longer silently clear 202-accepted steers (createdAt-guarded closeAndDrain + owner park before the terminal write) - Persist the steer part BEFORE media encoding in the drain hook: an abort inside the encode window can no longer lose a file-steer (the part refs come from the enqueue-sanitized item; replay re-encodes per turn unchanged) - Move the parked-claim owner check INSIDE the atomic store claim (substring gate in the Lua / in-memory equivalent): a non-owner probe can no longer transiently delete the recovery payload; the app-side parse stays authoritative - Park queued steers in BOTH stores' own requires_action expiry cleanup, which bypassed the manager-level sweep - Sweep expired parked steers from the in-memory store's periodic cleanup; restore a queued chip when send-now's submit is refused; upsert steer ACKs so an SSE reconnect reseed cannot duplicate chips - Mount the cancel mutation per steer item so the pending slot needs no QueryClient on ordinary streaming renders (fixes the CI failure in ContentParts.integration.test) - Skipped delivery-gated parking (finding 8): transport receiver counts cannot prove browser delivery, and gating the only durable copy on them trades cosmetic chip resurrection for real text loss; the window is already bounded by claim-on-read, createJob reset, and the TTL * 🩺 fix: Annotate PARKED_STEERS_TTL_MS for isolatedDeclarations tsdown's d.ts generation requires explicit types on exported consts with computed initializers; tsc --noEmit does not run that check, so the round-11 export slipped past local verification and broke Build packages (and every downstream CI job that consumes the built dist). * 🛟 fix: Round-12 Terminal-Path Recovery + Durable Steer Events - Park queued steers before the stale-running reap deletes a crashed or hung job in BOTH stores — the one terminal path with no controller finalization; requires_action expiry parking refactored onto the same snapshot/park helpers - Enqueue instead of dropping when a steer fallback send is refused: both the NO_ACTIVE_RUN branch and the settled-run rejection branch now observe sendNow's false return - Recover on the SSE reconnect-404 terminal path: convert local pending steers to queued, claim parked steers via /chat/status, and write a non-completed run-end signal so interrupt flags release without auto-sending an unknown outcome - Fall back to a positive parked-recovery TTL when completedTtl is 0 (SET EX 0 is invalid and silently killed recovery) - Make on_steer_applied durable before publish: emitChunk gains a durable option that awaits the chunk-log append (best-effort) ahead of the transport publish; the default delta path stays fire-and-forget * 🔐 fix: Round-13 Steer Authorization + Trusted File Refs - Resolve client-supplied steer file refs against the DB owner-scoped at enqueue and queue only DB-derived shapes (same filter as the injection fetch, shared via refs.ts); any unresolved id fails loud with 400 — spoofed type/filepath metadata can no longer be persisted into assistant content or rendered in chat/share views - Enforce agent authorization on /chat/steer against the ORIGINATING run's job identity: the chat path's role gate (AGENTS:USE, with the same non-agents-endpoint skip) plus the per-agent ACL check with the capability bypass — revoked access mid-run can no longer inject; cancel stays ownership-only (nothing model-bound) - Mark steered uploads used after a successful enqueue (owner-scoped, best-effort) so the upload-window TTL cannot reap a file the persisted steer part references - Consume the parked recovery copy after live delivery: converting final/abort/error pendingSteers fires one owner-gated claim-on-read, so dismissed chips can no longer resurrect on a later reload * 🎙️ fix: Round-14 Composer-Context Fidelity + TTS and Queue-State Gaps - Keep steer text out of generic assistant text extraction: parseTextParts excludes STEER parts by default with an includeSteer opt-in for the full-record surfaces (Meili indexing, aborted-response persistence) — TTS callers no longer speak the user's own mid-run words - Mark queued uploads used at enqueue time via a minimal owner-scoped POST /files/usage (fail-closed without a user; upload limiters do not apply to a metadata touch), fired once wherever composer files enter the queued state — the upload-window TTL can no longer reap a file waiting out a long run or approval pause - Carry quote chips and manual skill picks on queued items: captured and consumed from the composer at queue/interrupt time exactly like files, threaded through the drain and send-now overrides, and restored by the queued row's Edit message - Key an early-aborted FIRST turn's run-end signal to NEW_CONVO (resolveRunEndTarget) so queued follow-ups stay visible on the restored new-chat composer instead of parking under an optimistic stream id the user never sees again * 🧿 fix: Round-15 Gap Coverage + Consolidated Sweep (Share Leak, Abort Ids, Chip Hygiene) - Run the resume steer-gap check for every still-active job: an empty snapshot no longer skips the re-peek, and synthesis now keys on the FRESH content view so an applied-in-gap steer that was never snapshotted still re-surfaces (over-emission is benign — applied-id dedupe, index-stable parts) - Thread queued context through steer degradation: sendQueuedNow passes the item's quotes/skills into submitSteer, and every fallback (requeue or settled send) restores them instead of dropping to text+files - Stop shared links from leaking steer attachment refs: the share snapshot now walks content — files-excluded shares strip steer-part files entirely; files-included shares sanitize and share-route them like top-level files (copy-on-write, non-steer content by reference) - Seed pending-steer chips unconditionally on load/return so a steer applied while away cannot linger as a stale chip beside its part - Use the abort response's resolved job id: chips/drain-signal land where the user actually is (NEW_CONVO for a new-held first turn, consistent with resolveRunEndTarget) while the parked-copy claim hits the resolved id instead of a no-op /chat/status/new - Open steered documents like normal message files (FilePreviewDialog) - Cap the applied-steer id set on the live path via a shared helper; kept surviving run end deliberately (late-ACK race depends on it) and fixed the atom comment that claimed otherwise * 💡 fix: Un-light Steer Ribs When Their Node Is Replaced Two stacked gaps kept a steer rib lit after scrolling away: the pending→applied swap replaces the DOM node under the same id, which produces no IntersectionObserver exit and — because the entry list dedupes on (id, preview) — no entries change either, so the observer kept watching a detached node; and the rail's mutation filter only reacted to .message-render nodes, so steer-node swaps and removals never triggered a refresh at all. - reconcileObservedElements re-points the observer at replaced nodes from the mutation-driven refresh regardless of entries identity, dropping stale visibility until the fresh node reports (the observer fires its initial intersection immediately, so a truly visible part re-lights within a frame) - The mutation filter now recognizes steer-render nodes alongside message rows * 🪪 fix: Round-16 Recovery Owner Fields + Context Stickiness + Share Labels - Park resumed-run leftovers with the manager facade's metadata owner fields: a bare job.userId is undefined on that shape, which made every parked payload from a resumed HITL run unclaimable - Keep a queued item's quotes/skills sticky through a successful steer ACK: the pending chip carries them (client-only), reseeds preserve them across reconnects, and every terminal conversion — local or server-list, merged by steerId — restores them onto the queued item - Convert resumeState.pendingSteers on the inactive status branch (deduped against unrecoveredSteers) so steers observed in the expired-pause-before-sweeper window convert instead of vanishing until a later reload - Label shared steer parts share-safely via the existing ShareContext: a viewer's own name no longer appears on the sharer's steered messages * ✂️ fix: Carry Steer Context Through the Failed-Chip Edit Action Retry and convert-to-queue already preserve a failed steer's carried quotes/skills; Edit message dropped them on the way back to the composer. It now restores them through the same context path.	2026-07-14 10:11:10 -04:00
Danny Avila	520af663bc	🧵 feat: Background Tool Calls for Agents & Model Specs (#14197) ... * 🧵 feat: Background Tool Calls for Agents & Model Specs Opt-in, poll-based background tool execution. The model marks an eligible tool call with `run_in_background: true`; the host executor registers a task, returns a handle immediately (so the graph turn resolves), runs the tool as a detached promise, and the model retrieves the result via a new `check_background_task` poll tool. Host-side only — no `@librechat/agents` change. - Opt-in mirrors `deferred_tools`: admin capability `run_in_background` (off by default) + per-tool `tool_options.run_in_background`. - Model specs / ephemeral agents: `TModelSpec.runInBackground` / `TEphemeralAgent.run_in_background` synthesize per-tool options; both paths converge at `initializeAgent`. - In-process task registry: scoped per user+conversation, idempotent by toolCallId (safe across resume/replay), capped, TTL-swept. - Excludes direct-path / host-special / code-session tools. Subagents and push notifications are deferred follow-ups. * 🩹 fix: Harden background tool calls (Codex review) - Reliable per-agent execution gate: thread the injected `run_in_background` tool names from `initializeAgent` through `configurable.backgroundToolNames` (`toolRegistry` only reaches the executor for PTC/tool_search), fixing the silent no-op + unstripped-arg leak for ordinary event-driven tools. - Enforce the per-tool opt-in at execution (`backgroundToolSet.has(name)`) so a non-opted-in tool can't be backgrounded via an extra arg. - Gate the `check_background_task` interception on the run actually enabling background, so a user tool sharing that name still executes. - Forward `backgroundToolsAvailable` to added-convo (multi-convo) agents. - Exclude `web_search`/`file_search` from eligibility — their results are turned into user-visible attachments/citations only by the foreground toolEndCallback. * 🩹 fix: Address Codex round 2 on background tool calls - Idempotency scoped to run+turn: provider tool-call ids repeat across turns (e.g. `call_0`), so key the dedupe map by `runId::toolCallId` and sweep orphaned mappings — a later turn no longer collides with a retained task. - Artifacts preserved: a backgrounded tool's artifact is processed through the same `toolEndCallback` as the foreground path (images/files/citations no longer silently dropped), best-effort/guarded. - Forward the `run_in_background` capability to connected-agent discovery and subagent `processAgent` init, so a child agent's own event-driven tools work the same as when it runs as primary. - Strip the injected flag on foreground calls of background-capable tools (the model may emit it as `false`) so strict MCP/action schemas don't reject. - `check_background_task` list path returns metadata only (result_available / result_chars), never full results — prevents context overflow; the full result is returned only when a specific id is requested. * 🩹 fix: Address Codex round 3 on background tool calls - Exclude background-capable tools from eager execution (run.ts): a speculative eager dispatch of a `run_in_background` call could launch the detached task with partial/stale args, and that side effect can't be canceled. - Reserve the `check_background_task` name: overwrite a colliding user/MCP tool with the host poll schema (with a warning) so the advertised schema matches the executor's interception instead of hijacking a mismatched tool. - Don't inject background schemas into pure subagents (spawn-tool child graphs) whose tools don't reach the host interceptor; keep it for primary/added/ connected agents. Subagent background is the durable follow-up. - Thread `backgroundToolsAvailable` + `backgroundToolNames` through the OpenAI-compatible and Responses agent routes (was chat-only), so the same agent/model spec behaves consistently across surfaces. - Exclude image-generation built-ins (dalle/flux/gemini_image_gen/image_gen_oai/ image_edit_oai) — artifact-first tools whose files can't reliably attach to an already-saved turn when backgrounded. * 🩹 fix: Address Codex round 4 on background tool calls - Sanitize self-spawn subagent inputs: strip `run_in_background` + the `check_background_task` def from the parent AgentInputs reused for self-spawn, so the isolated child (direct/child-graph path) doesn't advertise a background schema it can't honor. The SDK resolver keeps a provided `agentInputs` even with `self: true`. - Exclude `check_background_task` from PTC (`run_tools_with_code`) tool definitions — it's host-only and not callable from generated code. - Parse stringified JSON args before deciding background dispatch and before stripping the flag, so string-delivered `run_in_background` is honored and never leaks to strict object-schema tools. - Skip injection for tools that already declare their own `run_in_background` param (would otherwise hijack/strip it), and for non-object (string-input) schemas (would otherwise rewrite the input contract). * 🩹 fix: Address Codex round 5 on background tool calls - check_background_task now parses stringified JSON args, so providers that deliver args as a string can retrieve a specific task by id (not just list). - Include agentId in the background dedupe key (`agentId::runId::toolCallId`): two agents in the same run emitting the same provider id (e.g. `call_0`) now launch independent tasks instead of colliding. - Self-spawn sanitization also strips the background entries from the reused toolRegistry (not just toolDefinitions), so a child using tool_search/deferred loading can't rediscover the host-only run_in_background / check_background_task. * 🩹 fix: Strip run_in_background from PTC target tool schemas (Codex round 6) The PTC path already filtered out the host-only check_background_task poll tool but still exposed target tool schemas with the injected `run_in_background` param (the shared toolRegistry entries were mutated by applyBackgroundToolCalls). PTC codegen doesn't go through the host background interceptor, so it could pass the flag to an MCP/action tool (strict-schema rejection or silent foreground with no poll). Sanitize the PTC toolDefs like the self-spawn path does. * 🩹 fix: Sanitize background from explicit subagent inputs (Codex round 7) A child agent reachable as a top-level/handoff agent is initialized WITH the background capability, then reused as an explicit subagent via buildSubagentConfigs. Round 4 only sanitized the self-spawn case; this now applies the same stripBackgroundFromToolDefinitions/Registry to explicit child agentInputs when `child.backgroundToolNames` is non-empty, so an isolated child graph doesn't advertise a run_in_background / check_background_task contract it can't honor. * 🩹 fix: Reap stuck/expired background tasks (Codex round 8) - get() now sweeps before returning, so repeatedly polling a known background_task_id can't keep an expired completed task (and its retained result, up to 100k chars) alive past the one-hour completed TTL. - sweep() now reaps `running` tasks older than a 30-min running TTL, marking them errored. Previously a detached call that never settled (hung network / lost MCP connection) held a running slot forever, exhausting the per-conversation cap and rejecting every later dispatch. * 🩹 fix: Evict oldest settled tasks instead of blocking at the cap (Codex round 9) Only the running-task cap gates dispatch now. The total-tasks cap (MAX_TASKS_PER_BUCKET) bounds memory but no longer rejects new background calls: when full, it evicts the oldest settled (completed/error) tasks to make room. Previously 200 quick background calls in one conversation would block all new dispatches for up to the completed-task TTL, since polling doesn't remove settled tasks. Running is already capped, so room always frees. * 📝 docs: Frame background tool calls as within-turn (Codex P1 contract) Codex escalated the request-lifecycle findings to P1 on the grounds that the advertised "poll later" contract can't be honored for genuinely long-running calls (request-scoped MCP connections + the run abort signal are torn down at turn end). Align the model-facing contract with what the same-run implementation actually delivers: the run_in_background param, check_background_task, and the dispatch handle now instruct the model to collect the result WITHIN THE SAME TURN (backgrounded work isn't guaranteed to survive past the turn). This is within-turn parallelism; cross-turn survival of long-running calls remains the deliberate durable subagent follow-up. Copy/comment-only; no behavior change. * ♻️ refactor: Cross-turn background tool calls, leak-free Extend background tool calls from within-turn to cross-turn on a single process, since the mechanism already supports it: the run's abort signal never reaches the detached invoke (the graph forwards only configurable/ metadata to the tool-execute handler), so the floating promise keeps running past turn completion and its result stays in the in-process registry for a later turn to poll (get/list key only on user::conversation + id, never the dispatch run/turn). Guarantee no connection leak: ephemeral request-scoped MCP tools (runtime {{LIBRECHAT_BODY_*}} placeholders) capture their request-scoped store at creation and fall back to it, so config manipulation can't redirect them; their connection is torn down at request end. Tag such tools in createToolInstance and run them in the foreground instead of backgrounding them. Pooled/app-level MCP and structured tools are unaffected and survive cross-turn via their managed pools. Reword the model-facing contract (run_in_background, check_background_task, handle message, fileoverview) from within-turn to cross-turn on this server (not across restart/replica, which stays the durable follow-up). Tests: cross-turn poll retrieval; ephemeral MCP tool runs foreground. * 🐛 fix: Guard ephemeral MCP tag against a null server config createToolInstance can be reached with a null/stale capturedServerConfig (cached availableTools + getServerConfig returns null, as several MCP unit tests construct tools). The new unconditional requiresEphemeralUserConnection call then dereferenced config.source and threw during tool construction (CI: Tests api shard 2/3). Guard with the same serverConfig ? ... : false pattern the other callers use; a missing config is not request-scoped. * 🎨 fix: Deliver backgrounded tool artifacts on the poll turn A slow backgrounded MCP/action tool resolves after its dispatch turn is finalized: createToolEndCallback only appends to that turn's artifactPromises (already awaited) and writes to a closed stream, so the artifact (file/citation/ UI resource) was silently dropped — check_background_task recorded only the hasArtifact boolean. The cross-turn contract made this the common case. Hold the artifact on the task and deliver it through the LIVE poll turn's toolEndCallback the first time check_background_task collects that id (once, then cleared to free memory), attributed to the original tool. Same-turn and cross-turn now share this path since the model must poll to collect any result. Tests: registry claim-once; artifact delivered on poll not dispatch, idempotent. * ✨ feat: Agent-builder toggle for background tool calls + cap tool descriptions Add a per-MCP-tool "run in background" toggle in the agent builder, mirroring the programmatic/deferred pattern: gated on the admin `run_in_background` capability via useAgentCapabilities, read/written on tool_options[id] .run_in_background through useMCPToolOptions (per-tool + bulk mark-all), and rendered as a Zap toggle in MCPToolItem and McpSection with new locale keys. Also cap the section tool/server descriptions (McpSection, ToolSection, SkillSection) with max-h-40 overflow-y-auto so a long description scrolls instead of overflowing the dialog, matching MCPToolItem's existing cap. Tests: MCPToolItem renders/toggles the background button only when enabled. * 🧪 fix: Mock new background hook functions in McpSection spec * 🎨 fix: Restore background artifact when poll-turn delivery fails * 🛡️ fix: Harden background tool call edges from review findings - Error immediately (matching foreground) when a background-requested tool failed to load, instead of returning a success handle for a dead task - Exclude ephemeral request-scoped MCP tools at injection time so the model never sees a run_in_background param the executor would silently downgrade; flip the execute-time tag to fail closed on a missing server config - Source image-tool background exclusions from the shared imageGenTools set (adds missing stable-diffusion, an artifact-first live tool) instead of a hand-copied list - Add check_background_task to the eager-execution exclusion list: artifact collection is a one-shot claim that must not fire from a speculative snapshot the SDK may discard - Strip an imitated run_in_background arg on tools the executing agent never opted in (multi-agent history bleed), unless the tool's own schema declares the parameter - Truncate oversized stored results with an explicit marker via the shared truncateMiddle (moved to utils/text) instead of a silent slice - Document the at-most-once artifact delivery semantics honestly (the callback's downstream persistence is fire-and-forget, as in foreground) * ♻️ refactor: Deduplicate background tool-call plumbing and tighten types - Use the SDK's JsonSchemaType instead of a local duplicate; drop all as-unknown casts and type the poll-tool serializer explicitly - Drop derivable BackgroundTask state (progress, hasArtifact) and the dead `enabled` param/return on applyBackgroundToolCalls (guarded at the call site), which also skips the defs pass when nothing opted in - Fold the enable expression into synthesizeBackgroundToolOptions so the three load/added call sites can't drift - Throttle the registry's all-buckets sweep and always sweep the accessed bucket, so a hot poll loop is no longer O(total tasks server-wide); bound retained artifact memory with a size cap - Single-pass stripBackgroundFromToolDefinitions; pass metadata through to the poll-turn callback instead of a no-op reconstruction - Collapse the client's copy-pasted boolean option families into a keyed factory (also removes the shared-object mutation in the bulk toggles) and the six toggle-button copies into one OptionToggle component * 🧪 test: e2e coverage for cross-turn background tool calls Proves the full contract through the real pipeline (mock harness): an agent opts an MCP tool in via tool_options.run_in_background, the model dispatches it detached and receives the synthetic handle while the tool is still running (status=running in the rendered ack — the non-blocking guarantee without timing assertions), the tool completes after its turn finalized, and a later user turn recovers the task id from replayed history, polls check_background_task, and renders the collected result. - fake-mcp-server: slow_echo fixture tool (delayed echo) - fake-model: E2E_BACKGROUND_DISPATCH / E2E_BACKGROUND_COLLECT markers - e2e yaml: agents capabilities = defaults + run_in_background * 🔧 fix: Close two background capability gaps from review - Thread backgroundToolsAvailable through the OpenAI-compatible service (derived from app capabilities like codeEnvAvailable/statefulSessions), so agents with tool_options.run_in_background keep the feature on that route; fold the three capability derivations into one helper - Index ephemeral MCP servers by normalizeServerName when excluding tools from background injection: tool names embed the normalized server name while mcpConfig keys the original, so exotic server names previously escaped the injection-time exclusion * 🛂 fix: Fall back to configurable user identity for background task scoping The in-repo routes merge req into the tool-execute configurable, but external hosts of the exported OpenAI-compatible service inject their own loadTools and may not — tasks would then register under an empty user id, collapsing registry isolation to conversationId alone. Resolve the scoping id from req.user.id, then configurable.user_id / user, and cover the isolation with a foreign-user not_found test. * 🧹 chore: Apply repo import sorter to PR-touched files	2026-07-13 12:51:36 -04:00
Danny Avila	53e369fba8	🧪 feat: stateful_code_sessions capability for warm Code API sandbox sessions (experimental) (#14150) ... * ✨ feat: stateful_code_sessions capability for warm Code API sandbox sessions Wire the @librechat/agents stateful sandbox sub-config behind a new, off-by-default stateful_code_sessions agent capability. createRun sets toolExecution.sandbox.statefulSessions when code execution is active in the run AND the capability is enabled; execute_code and bash_tool factories get the param so their descriptions hedge toward persistence. Rides the existing variable-not-literal runConfig pattern, so it no-ops until @librechat/agents is bumped to the version shipping the sandbox sub-config. * ✨ feat: per-agent stateful code sessions (builder toggle + init gating) Stateful sessions now require the agent's own opt-in, not just the admin capability. New agent field stateful_code_sessions (schema + validation + types) surfaces as a toggle in Agent Builder Advanced settings, gated on the app capability and disabled without Code Interpreter. initializeAgent resolves the per-agent truth (admin capability AND builder opt-in AND code env) once: the registered bash_tool description, the execute_code factory, and createRun's toolExecution.sandbox gate all read the same resolved value. statefulSessionsAvailable threads through the same call sites as codeEnvAvailable, including handoff discovery and added convos. * 🐛 fix: propagate runtime_session_hint to sandbox executor in event-driven tool path The event-driven ON_TOOL_EXECUTE handler built config.toolCall without the resolved runtime_session_hint, so BashExecutor/CodeExecutor never sent runtime_session_hint to the Code API. Every conversation then collapsed onto the server-derived default session (no per-conversation isolation). Copy tc.runtimeSessionHint onto toolCallConfig._runtime_session_hint, mirroring the SDK direct-execution path. * 🐛 fix: address Codex review findings for stateful code sessions - OpenAI-compatible service (packages/api/src/agents/openai/service.ts) now derives and passes statefulSessionsAvailable alongside codeEnvAvailable, so the feature activates on that route (previously statefulCodeSessions resolved false there and createRun never sent toolExecution.sandbox). - Thread runtime_session_hint through the host file-authoring tools (create_file/edit_file/read_file): those host branches return before the generic tool path, so readSandboxFile/writeSandboxFile now forward the per-conversation hint instead of falling back to the Code API default session. - StatefulSessions builder toggle clears its form value when Code Interpreter is disabled, so a saved agent matches the disabled UI and re-enabling code doesn't silently reactivate stateful sessions. * 🐛 fix: normalize stateful_code_sessions on save when Code Interpreter disabled Addresses Codex review (round 2): a stale `stateful_code_sessions` opt-in could persist when Code Interpreter (`execute_code`) is disabled from the main agent builder without opening Advanced settings, silently reactivating warm sessions if code was later re-enabled. - AgentPanel: normalize in `composeAgentUpdatePayload` (the always-run save path) so `stateful_code_sessions` is forced to `false` whenever `execute_code !== true`, regardless of whether Advanced was opened. - StatefulSessions: revert the mount-scoped useEffect (round-1 approach) — it only fired while the Advanced panel was mounted, missing this path. - Add spec coverage for both branches of the normalization.	2026-07-12 08:12:04 -04:00
Danny Avila	b753da163e	🤖 feat: Add GPT-5.6 (Sol/Terra/Luna) OpenAI Models (#14206) ... * ✨ feat: Add GPT-5.6 (Sol/Terra/Luna) OpenAI models Adds the GPT-5.6 family (GA 2026-07-09) across the context, output, pricing, cache, premium, and default-model maps, mirroring gpt-5.5. - gpt-5.6 (Sol alias), gpt-5.6-terra, gpt-5.6-luna - 1.05M context / 128K output for all tiers - Standard + long-context (>272K input) tiered pricing and cache rates * 🐛 fix: Bill GPT-5.6 cache writes at documented 1.25x input surcharge OpenAI prices GPT-5.6 cache writes above the base input rate (Sol $6.25, Terra $3.125, Luna $1.25 per 1M vs $5/$2.50/$1 input). Correct the cacheTokenValues write rates so explicit prompt-caching usage is billed and reported accurately, and lock the surcharge with a test. * ✨ feat: Expose GPT-5.6 max reasoning effort + long-context cache premium Folds in the two deferred Codex findings: 1. Add `max` to the OpenAI `ReasoningEffort` enum and the reasoning_effort parameter options/labels so GPT-5.6 (Sol/Terra/Luna) can request its documented highest reasoning setting. Backend passthrough and zod validation pick it up via the nativeEnum schema. 2. Apply the long-context (>272K input) premium to cache tokens. Adds `premiumCacheTokenValues` + `getPremiumCacheRate`, threads `inputTokenCount` into `getCacheMultiplier`, and wires it through both structured-spend paths. Covers the gpt-5.4/5.5/5.6 family whose cache write/read previously stayed at flat base rates on long-context calls. * 🐛 fix: Bill GPT-5.6 cache writes + map max effort for OpenRouter Claude Addresses Codex round-3 findings: 1. (P1) splitUsage only read `cache_creation`/`cache_creation_input_tokens`, so OpenAI GPT-5.6's `cache_write_tokens` fell into inputOnly and billed at the input rate instead of the 1.25x write rate. Extend UsageMetadata and single-source the cache-creation read to also recognize `cache_write_tokens` (nested and top-level). 2. (P2) `max` was exposed via OpenRouter (spreads OpenAI settings) but the adaptive-Claude verbosity map had no `max`, so it was silently dropped. Map max -> 'max' verbosity. * 🐛 fix: Forward GPT-5.6 cache_write_tokens into emitted usage Local Codex review (P1): the cache-write fix reached balance billing (splitUsage/getCacheCreationTokens) but not the emitted-usage pipeline. ModelEndHandler built the emitted event's cache_creation from only cache_creation/cache_creation_input_tokens, so GPT-5.6 cache_write_tokens were dropped and aggregateEmittedUsage classified them as ordinary input — displayed/persisted cost undercounted and disagreed with the balance charge. Fold cache_write_tokens (nested + top-level) into the emitted cache_creation.	2026-07-12 07:52:59 -04:00
Danny Avila	3945d293de	🗂️ feat: Per-Agent Memory Partitions (#14084) ... * feat: per-agent memory partitions (memory_scope) Adds an optional agentId partition to MemoryEntry so agents can opt into isolated memory via a new memory_scope field ('user' | 'agent'). Partition derives from agentId presence ({agentId: null} matches legacy docs, no migration). Inline set_memory/delete_memory tools, the post-turn memory agent, the request-scoped memory cache, and context injection are all partition-aware; context is only injected into agents whose resolved partition matches. Memory routes accept the partition param, scope duplicate/token-limit checks per partition, and enrich entries with agent names. Memories panel gains a partition filter and agent badges; the agent builder gains an agent-scoped memory toggle. * fix: address Codex review findings on memory partitions - strip runtime ____N id suffixes in getMemoryAgentId so added-conversation runs share the persisted agent's partition - load each agent's own partition in multi-agent context injection instead of skipping foreign partitions entirely - clear memory_scope to 'user' on save when Enable Memory is unchecked - fall back to 'all' when the selected panel partition no longer exists - restrict GET /memories agent-name resolution to agents the requester can VIEW	2026-07-09 10:48:51 -04:00
Danny Avila	988a14a405	🙋 feat: ask_user_question - agent-initiated questions with durable pause/resume (#14139) ... * feat: ask_user_question tool — agent-initiated questions with durable pause/resume The HITL runtime merged in #13942/#14024/#14025/#14123 already ships the full ask_user_question lifecycle (payload-agnostic handleRunInterrupt, resume validation via mapAskUserAnswer, reconnect rehydration, and the client question card) — but nothing ever raised the interrupt. This adds the producer: - packages/api/agents/hitl/askUserQuestionTool.ts: LLM-callable tool whose func calls the SDK askUserQuestion() helper (LangGraph interrupt() from the tool body); zod schema with length caps mirroring AskUserQuestionRequest, plus a JSON-schema twin for the schema-only registry - Registration: agentToolDefinitions, manifest.json (Tools dialog, admin filteredTools/includedTools kill switch), basicToolInstances, handleTools constructor branch - run.ts gating: checkpointer now attaches for hitlCapable runs whose agents carry the ask tool even with the tool-approval policy disabled (the interrupt needs only durability, not humanInTheLoop/hooks); the tool is stripped fail-closed from non-HITL callers (OpenAI-compat/Responses) and subagent child configs; excluded from eager event execution (interrupts must be raised inside the Pregel task frame) - resume.js: 16k length cap on the answer wire field - e2e (real Run + FakeChatModel + LazyMongoSaver + supertest resume): tool-body interrupt pauses durably with NO approval policy, answer round-trips as the ToolMessage content, tool body re-runs once on resume, sequential questions re-pause * fix: adversarial-review findings — in-graph execution, orphan prunes, endpoint scoping, real kill switch Pre-PR multi-agent review confirmed 5 defects in the initial commit; all fixed: 1. CRITICAL — the tool never paused on the real agents endpoint: production loads tools definitions-only, flipping the SDK ToolNode to event-driven dispatch, and the host ON_TOOL_EXECUTE handler runs outside the Pregel task frame (under runOutsideTracing), where interrupt() throws and becomes an error ToolMessage. Reworked: the ask tool never rides toolDefinitions/ toolRegistry — on HITL-capable top-level agents a real instance is supplied via AgentInputs.graphTools (agents#289, requires @librechat/agents > 3.2.57), the SDK's in-graph direct-tool seam; new production-shape e2e pins the event-driven mode end to end. 2. CRITICAL — ask-only runs left orphaned interrupted checkpoints (silent context duplication on every later turn): both orphan prunes were gated on toolApproval.enabled. The pre-turn prune now also fires for ask-capable agents (exported agentRequestsAskUserQuestion), and the abort-route prune fires when the aborted job carries a pendingAction. 3. MAJOR — self-spawned subagents bypassed the strip (self config resolves from the parent's _sourceInputs): fixed SDK-side (buildChildInputs clears graphTools) and the tool is now never present on child surfaces host-side. 4. MINOR — the manifest entry leaked into the Assistants tools dialog and the legacy plugins endpoint, where tools execute with no run to pause: new agentsOnly manifest flag, scoped out of both listings. 5. MINOR — filteredTools/includedTools only hid the tool from the dialog: now enforced at run build (strip + no checkpointer), making the admin filter a real kill switch for already-saved agents. * chore: update @librechat/agents dependency to version 3.2.58 in package-lock.json and package.json files * fix: reject agents-only tools at assistant create/update (Codex round 1) The tools-dialog scoping keeps ask_user_question out of the assistants LISTING, but the v1/v2 create/update handlers resolve arbitrary posted tool strings from the shared getCachedTools map — a REST client or stale saved payload could still attach it, and the assistants runtime executes tools with no run to pause, so every call would error. New isAgentsOnlyTool(tool) (manifest-driven, handles string and function-object shapes) drops such tools with a warn at all four resolution sites (v1+v2, create+update). * fix: offset resumed-run content indices past the pre-pause seed A resumed run rebuilds the graph from the checkpoint, and the fresh graph numbers content indices from its own empty contentData — starting at 0. The resume path seeds the (also fresh) content aggregator with the pre-pause parts at exactly those indices, so the resumed model turn collided with the seed: type-matching parts silently MERGED (post-resume text appended into a pre-pause text block), and type-mismatching parts (a reasoning/think part at index 0 — any Anthropic reasoning agent) dropped EVERY delta with 'Content type mismatch', losing the entire post-resume output from the live stream and the saved message. Latent since #13942 — tool-approval resumes corrupt content the same way (probe-verified); it surfaced now because ask_user_question makes pausing a first-class flow and reasoning models make the loss total. - createContentIndexOffsetHandlers(handlers, offset): wraps ON_RUN_STEP (the single point where a content index enters the pipeline — deltas resolve through the aggregator's stepMap) and ON_AGENT_UPDATE's inline index; every other handler passes through by reference. Probe-validated: resumed output now lands as a new part after the paused tool call. - resumeCompletion wires it with offset = seedContent.length. - logToolError: a GraphInterrupt unwinding out of a tool body is the HITL pause working as designed — no longer logged as a Tool Error. * fix: unblock live streaming of the resumed segment after an answer With resume indices now ABSOLUTE (server continues after the pre-pause parts), the synthetic ask-user-question card was squatting on exactly the index the resumed segment streams into: applyAskUserQuestion appends the card at the end of the message content, so on the answering device every incoming part at that index was blocked and nothing rendered between the answer submission and the finalize replacing the message. removeAskUserQuestionPart(message, actionId) strips the pause-scoped card on successful answer submission (useResumeSubmit onSuccess) — the durable record of the Q&A is the ask_user_question tool call itself. Pure helper + specs; same-reference no-op when nothing matches. * fix: displace the synthetic question card in the streaming content writer The store-level strip on answer submit wasn't enough: the SSE step handler keeps its own in-flight copy of the streaming message, so on the answering device the synthetic ask-user-question card still occupied the ABSOLUTE index the resumed segment streams into — every delta warned 'Content type mismatch' (existing ask_user_question vs incoming text) and nothing rendered between the pending_action and finalize. Displace the card inside updateContent when any real part claims its slot — the same displacement pattern as the OAuth prompt part directly above it. Covers the streaming handler's own copy, reconnecting tabs, and other devices; once real content streams, the pause is over by definition. Spec drives a runStep + text delta into the card's index and pins: no mismatch warn, card gone, text rendered. * feat: dedicated UI + durable data for completed ask_user_question calls The completed ask call rendered as a generic tool card labeled 'Cancelled' with raw (and empty) JSON args. Two layers fixed: Data: the saved tool_call part had args:'' and no output — streamed arg chunks carry no tool name so the aggregator drops them (normal tools recover via the completion event, which never fires for a tool that interrupts mid-execution and resumes on a rebuilt run with no step id). The resume controller now stamps the paused ask part with the pendingAction's authoritative question as args and the user's answer as output (attachAskUserQuestionAnswer — pure, targets the newest unanswered ask part, so sequential questions each keep their own answer). UI: Part.tsx routes ask_user_question tool calls to AskUserQuestionCall — a compact Q&A record ('Asked a question' header, question, description, 'You answered: <label>' preferring the picked option's label, or 'No answer was given' for an abandoned pause) instead of the generic card. New i18n keys; parseAskUserQuestionArgs degrades to null on malformed model args. * fix: single question UI per pause + immediate answer display Two live-turn issues with the new durable Q&A card: 1. Duplicate question on ask: during a live pause the message carries BOTH the ask tool_call part (now rendered by AskUserQuestionCall, showing a misleading 'No answer was given' while paused) and the synthetic interactive card. The durable card now defers while the turn is live and unanswered (isSubmitting) — the interactive card owns the question UI until it's answered; an abandoned pause still shows its no-answer state once the turn settles. 2. 'No answer was given' after answering: the server stamps the answer onto the part at resume seed, but the client only received that at finalize. No stream emission needed — the client knows the answer it just submitted: resolveAskUserQuestionPart (replacing the plain strip on submit success) removes the synthetic card AND stamps output/progress onto the newest unanswered ask tool_call, seeding args from the synthetic part's question when the streamed args were lost — mirroring the server-side attachAskUserQuestionAnswer, so the Q&A record shows the answer the moment the user submits. * fix: keep the Q&A record visible while the resumed segment streams The optimistic output stamp lives in the message store, but the SSE step handler evolves its own cached copy of the streaming message (created at turn start) — the first resumed event overwrites the store with that copy, wiping the stamp, so the Q&A card blinked out during streaming and only returned at finalize. Render-layer fallback instead of fighting the handler's copy: submitted answers are recorded by ask tool_call id when resolveAskUserQuestionPart stamps the part, and AskUserQuestionCall reads the recorded answer whenever the part's own output is missing — the record survives any message-copy churn until finalize delivers the server-stamped part. * feat: present Ask User as a native builtin in the tools dialog It ships with the app and pauses the run like a first-class feature, so it belongs with the builtins (Run Code, Web Search, Memory, ...) rather than in the third-party plugin list — while its mechanics stay exactly a plugin's: - BuiltinId += 'ask_user_question' (documented exception: a native TOOL, not a capability; selection reads agent.tools, the toggle emits tool-add/remove patches instead of a capability field) - buildCatalog surfaces it as a builtin gated on the same signals as before (tools capability on + the server lists the plugin, i.e. not admin-filtered) and skips it in the plugin loop so it never double-lists - On-theme icon: lucide MessageCircleQuestion in a teal chip via the builtin icon map, matching the other native entries; the bespoke purple SVG and the manifest icon field are gone - i18n'd name/description keys like the other builtins * feat: composer popover for answering questions (mentions-style) Answering moves to the composer, matching the existing mentions/prompts popover pattern: while an ask_user_question pause is live, a popover anchors above the textarea with the question as its header, numbered option rows (hover/click, or ↑/↓ + Enter from the empty composer), and an × to dismiss. The main textarea doubles as the free-form answer — its placeholder flips to 'Something else...' and form submit routes the text to the paused run as the answer instead of starting a new turn. Dismissing (× or Escape) restores normal sends; the inline transcript surfaces stay as before (interactive card while paused, durable Q&A record after) so the question remains visible in history. - findLiveAskUserQuestion (pure, spec'd): newest unanswered synthetic part across the conversation IS the popover signal — applied on on_pending_action, stripped on answer submit, so visibility tracks the pause lifecycle with no extra state - useLiveAskUserQuestion hook shared by the popover and ChatForm; dismissals in a recoil atom so both react - popover only mounts on the primary composer (index 0), mirroring QuoteButton * feat: number-key selection + return glyph in the question popover Pressing 1-9 in the empty composer picks the matching option directly, mirroring the numbered row chips; the highlighted row shows a return-key glyph as the Enter affordance. Same empty-composer guard as the arrow keys — typing a free-form answer is never intercepted. * refactor: first-class composer answer mode (useAskAnswerMode) Replaces the bolted-on integration (inline onSubmit interception + raw capture-phase keydown listeners on the textarea ref) with a single hook that owns the whole answer mode: live-question derivation, dismissal + highlighted option (shared recoil state), option selection, free-form submit routing (submitText returns whether it consumed the submission), and keyboard handling (handleKeyDown returns whether it consumed the key, composed ahead of the textarea's normal handler — no more addEventListener). The popover is now pure rendering off the hook; ChatForm wires placeholder, onKeyDown, and onSubmit through the same instance. Deliberately scoped to the composer rather than useSubmitMessage: starters/prompt-commands keep new-turn semantics (and the existing job-replacement behavior while paused). * fix: Codex round 2 — inline answer input, approval exemption, pause-time args F1 (composer submit unreachable while paused — isSubmitting keeps Stop shown and useTextarea eats Enter): redesigned around it, borrowing Claude Code's AskUserQuestion semantics. The popover now owns free-form input via an inline 'Other' row (numbered last, 'Something else…'), with select-then-confirm rows (click/arrows/digits highlight; Submit ↵, Enter, or double-click fires; Skip dismisses). The composer returns to being a plain composer — no placeholder swap, no submit interception; Stop keeps meaning stop. F2: ask_user_question is exempt from the tool-approval prompt unless the admin explicitly lists it (allow/ask/deny all win) — approving the right to ask a question was a pure double pause; the tool is side-effect-free. F3: the question is stamped onto the paused ask tool_call's args at PAUSE time (attachAskUserQuestionArgs in handleRunInterrupt), so abandoned/expired/ stopped turns persist with the question intact and the record card can render it — previously only the answer-resume path stamped args. * fix: fold model-supplied 'Other' options into the inline free-form row The model can generate its own catch-all option ('Other (type your own)', value 'other'), duplicating the popover's built-in free-form row — two other-ish rows, one pickable as a literal answer. Two layers: - Tool description now tells the model NOT to include catch-all options (the answer UI always offers free-form input on its own) - splitOtherOption (pure, spec'd) folds a catch-all option that arrives anyway out of the choice rows and uses its label as the inline input's placeholder — conservative match (value 'other', or a label reading as a free-form invitation), no false positives on real choices * fix: single question surface + clean free-form-only popover Two live-pause confusions: (1) the inline transcript card and the composer popover both rendered — the card now defers while the popover is up for its action, returning as the fallback surface when the user dismisses the popover (and in contexts without a ChatContext, where the popover can't exist); (2) an options-less question showed a pointless numbered '1 Something else…' row — free-form-only questions now render the inline input alone, with the 'Type your answer…' placeholder (a folded model 'Other' label still wins). * feat: the composer is the free-form answer box (like the main chat input) While a question pause is live, the main chat textarea composes the free-form answer — placeholder swaps to 'Something else…' (or a folded model 'Other' label), Enter with text submits the answer through answer-mode key handling (composed BEFORE useTextarea's submitting-lock, so the lock can't swallow it), and the Stop button swaps to Send (enabled despite isSubmitting) per the select-then-confirm design. The popover slims to the question header, numbered option rows, and Skip/Submit — its inline input is gone since the composer owns free-form now. Dismissing the popover restores normal composer semantics (Stop button, normal sends). * fix: Codex round 3 + real Skip semantics - Skip now ANSWERS instead of hiding UI (danny): it resumes the run with a decline notice ('The user chose not to answer this question.') so the model moves on — a client-side dismiss left the run paused until expiry, a hung turn. × / Escape remain pure dismiss (switch to the inline card surface). - P1 (resumed approval tool indices): resumed tool_calls steps whose tool_call id matches a seeded UNRESOLVED part now rebind to that seeded slot instead of offsetting — the original part resolves in place (output attaches) and no duplicate appears; message steps keep the offset, so the text-loss fix stands. createContentIndexOffsetHandlers now takes the seed array; resolved seeded calls are not rebind targets. - P2 (stale selection across questions): selection state resets when the live actionId changes; the vestigial inline-Other state ('other' selection + text atom) is gone — the composer owns free-form. - P2 (Redis abort path loses the args stamp): the abort route re-stamps the question onto the ask tool_call in the reconstructed abort content, so a Stop-abandoned question persists with its question intact. - P2 (malformed args crash): parseAskUserQuestionArgs normalizes untrusted shapes (options: {} / non-string entries) instead of throwing in render. * feat: free-form hint in the question popover footer Left-aligned in the footer row (opposite Skip/Submit): 'Or type your answer below' — points open-ended answering at the composer, whose placeholder already reads 'Something else…'. * feat: preserve composer drafts across the answer-mode swap The answer phase gets its own draft key (ask-answer:<actionId>), passed as a draftId override into useAutoSave — the key change itself drives the existing save/restore machinery, so the conversation draft (or mid-run PENDING draft) is stashed when a question pause takes the composer and restored once the user answers, skips, or dismisses. Ask keys are exempt from the PENDING migration branch, which would otherwise move-and-delete the stashed draft. A half-typed answer survives reload/navigation while its question stays live. Answer submission (option pick, free-form, skip) resets the composer via a new non-throwing useOptionalChatFormContext, so the swap-back restores into an empty box even outside ChatView-less render contexts (Share/search). * fix: rebind resumed steps for ALL seeded tool call ids The resume controller pre-stamps the user's answer onto the seeded ask_user_question part, so the unresolved-only rebind predicate treated it as settled and shifted the tool's re-run step to a fresh offset slot, leaving a duplicate ask record in streamed/saved content. Tool call ids are provider-minted per call: a resumed step bearing a seeded id can only be the interrupted batch re-executing, so rebinding every seeded id is always correct. * feat: popover UX round 4 — clickable hint, collapse, click-submit, multiSelect - Footer hint is a button that focuses the composer; reads 'Type your answer below' (no 'Or') when the question has no options. - Collapse (chevron) hides the popover WITHOUT closing the pause: answer mode stays live (placeholder, Enter routing, draft key), the chat card renders the question with a ChevronUp affordance to re-expand. x remains dismiss. - Single-select options submit on a single click; the Submit button renders only for multi-select. - multiSelect end-to-end: tool zod schema + JSON definition twin, wire type, client parse, popover check-chips, card toggles, record-card label mapping; answer = option values joined ', '; composer Enter and the multi Submit button both fold free-form text in with the checked values. - Hardening from adversarial review: in-flight status guard on every submit path (no duplicate resumes on double-click), popover locks while submitting, collapsed mode disarms invisible digit/arrow steering, the card shares the hook's checked state while the pause is live, the card folds catch-all 'Other' options, record mapping is all-or-nothing to avoid phantom labels, composer resets only when its text was consumed or the draft machinery will restore the stash. * feat: ask_user_question in model specs and ephemeral agents A librechat.yaml modelSpec can now equip the tool the same way it equips webSearch/executeCode/fileSearch/memory: modelSpecs: list: - name: my-spec askUserQuestion: true loadEphemeralAgent pushes the tool name when the spec flag (or the ephemeralAgent request flag, wired for parity) is set; everything downstream is the existing persisted-agent machinery — createRun's hitlCapable gating, graphTools injection, checkpointer attach, subagent strip, and the admin filteredTools/includedTools kill switch all apply unchanged. * feat: tense-aware Q&A record label (Asking / Asked) Shorten the record card header per feedback: 'Asking' while the question is still unanswered (abandoned/awaiting), 'Asked' once answered — replacing the single 'Asked a question' label. * fix: Codex round 4 — added-agent ask parity + preserve answer on failed resume F1 (added.ts): mirror loadEphemeralAgent's ask_user_question branch in the added-agent loader so a model spec's askUserQuestion flag (or the ephemeral request flag) equips added top-level agents too, matching execute_code / web_search / memory. Two load.spec cases added. F3 (composer): submitAskAnswer now takes an onSuccess callback and useAskAnswerMode defers clearing the selection/composer until the resume is accepted. A failed resume (16k answer-cap 400, expired action, network error) leaves status re-answerable, so wiping the composer up front lost the user's only copy of a free-form answer; now it survives for trim/retry. (F2 — a claimed Tools-capability bypass — was verified NOT reproducible: agentRequestsAskUserQuestion matches only loaded instances/toolDefinitions/ toolRegistry, all capability-filtered; a raw tools string has no .name and never triggers the install. Replied on-thread with the probe evidence.) * fix: Codex round 5 — expired question exits answer mode so its message shows An expired question (e.g. resume returns the stale-action 409) previously left the popover open with locked controls and no explanation, because the chat card — which carries the only 'this action expired' message — was suppressed by the popover-open guard. Treat 'expired' as no longer active: the popover closes, the composer reverts to normal, and the card becomes the sole surface and renders the expired message. 'error' stays active (retryable). * feat: group ask_user_question calls as their own category A homogeneous group of ask_user_question tool calls now reads 'Asked N questions' (present tense 'Asking N questions' while the turn streams) with a question glyph and no raw-name suffix — mirroring the subagent 'Ran N agents' category treatment, instead of 'Used N tools — ask_user_question'. Mixed groups keep 'Used N tools' but humanize the suffix to 'Question' and show a question icon for the ask entries (TOOL_FRIENDLY_NAME_KEYS + ToolIcon map). A group only forms at count >= 2, so the plural is always grammatical. Three ToolCallGroup.test cases cover homogeneous label/icon/suffix, present tense while streaming, and the mixed-group fallback. * fix: Codex round 6 — composer submit lock + abort stamp before emit F7 (composer status lock): the ask submit status lived on ApprovalContext, a React context mounted only around message content (ContentParts). The PRIMARY answer surface — the composer in ChatForm — renders outside it, so useApprovalContext returned the inert FALLBACK: status was always 'idle', setStatus a no-op. The in-flight double-submit guard (round 4) and the expired-exits-answer-mode fix (round 5) therefore never engaged for the composer. Move ask submit status to a global Recoil atom (useAskSubmitStatus) read/written by the composer, the popover, and the card alike, so a fast double-click/Enter is actually blocked and expired/error surfaces on every surface. Tool-approval status stays on the context (unchanged). F5 (abort stamp before emit): the abort route re-stamped a paused ask_user_question's args AFTER GenerationJobManager.abortJob had already emitted the final SSE from the unstamped content, so a Redis/cross-replica Stop left the live client showing an empty question until reload. abortJob now takes an optional transformAbortContent applied to the persistable content BEFORE the final event is built (and returned), so the live client and the saved message agree. New abort.spec case + updated call assertions. * feat: gate ask_user_question behind its own agent capability Add a first-class AgentCapabilities.ask_user_question (in defaultAgentCapabilities, on by default) so admins can enable/disable questions independently via endpoints.agents.capabilities, exactly like execute_code / web_search — not lumped under the generic tools capability. - ToolService: both filteredTools predicates (definitions-only and instance loaders) gate ask_user_question on checkCapability(ask_user_question) before the generic tools fallthrough. When off, the tool is dropped from toolDefinitions/toolRegistry, so run.ts's agentRequestsAskUserQuestion (which keys on the loaded surface) declines to install it and attach a checkpointer — the capability is enforced end-to-end at the loader, no run.ts change needed. - Tools dialog catalog: surface the ask builtin under its own capability rather than the generic tools one, so the UI matches the backend gate. - Tests: ToolService capability on/off filtering + defaults membership; catalog builtin visibility keyed on the dedicated capability. * style: sort imports in ToolCallGroup.test (CI import-order gate) * fix: Codex round 7 — surface ask-answer errors in the open popover A failed answer submission (16k reject, network error) sets the ask status to 'error', which — unlike 'expired' — deliberately keeps the question active and retryable. But the chat card that renders the error message is suppressed while the popover is open, so a composer/popover answer failed silently. Expose an 'errored' flag from useAskAnswerMode and render a warning line (com_ui_ask_answer_error) in the popover, so the user gets feedback and retry guidance without having to collapse/dismiss. It clears automatically on retry (status flips to 'submitting'). * fix: Codex round 8 — respect IME composition before submitting answers handleComposerKeyDown runs before useTextarea's composition guard, so with a CJK/IME keyboard the Enter that commits an in-progress composition was being intercepted and submitting the partial answer (and the composition buffer can leave value empty mid-compose, mis-triggering digit/arrow steering too). Bail at the top when composing — nativeEvent.isComposing, or key==='Process' / keyCode===229 for Safari's inconsistent reporting — mirroring the existing composer guard so the character commits normally. * chore: update `@librechat/agents` to v3.2.60 * 🔧 chore: Update @opentelemetry/core to version 2.9.0 and clean up package-lock.json * feat: digit shortcuts select options when the popover has focus Previously a number key (1..N) only selected an option from the empty composer (handleComposerKeyDown on the textarea) — if focus moved into the popover (a row/Skip/Submit button clicked or tabbed to), the number keys went dead. Add handlePopoverKeyDown, wired to the popover container's onKeyDown so it catches digits bubbling from the focused control: a digit activates its option exactly like a click (single-select submits, multi toggles). No highlight/Enter dance on this path — the options are buttons whose action is the click, and intercepting Enter would fight the focused button. Gated on active && !locked so it no-ops while a submit is in flight. * chore: update @librechat/agents to version 3.2.61 and @opentelemetry packages to latest versions	2026-07-08 15:31:05 -04:00
Serhii Zghama	96d213afe6	🧊 fix: Include Conversation Starters in Agent View and List Responses (#14142) ... * fix: include conversation_starters in agent view and list responses * test: cover conversation_starters in agent view and list projections * test: include conversation_starters in agent list whitelist --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-07-08 12:54:54 -04:00
Danny Avila	96367828e1	🧷 fix: Align Agent File Attachment Ownership (#14149) ... * fix: Align agent file attachment ownership * fix: Harden agent file unlink validation * test: Align file preview agent attachment access * test: Add agent file ownership e2e regression	2026-07-07 16:23:48 -04:00
Danny Avila	dcdaaeac67	🤐 fix: Exclude Provider Secrets from HITL Pending Actions (#14136) ... * 🤐 fix: Exclude Provider Secrets from HITL Pending Actions Sanitize resolved model parameters before persisting them in the pending action's resumeContext, and strip resumeContext/requestFingerprint from every client-facing copy (SSE emit, reconnect gap-fill, resume state, status route). The full record stays server-side for resume replay. * 🤐 fix: Treat Header-Carrier Keys as Sensitive Wholesale Google llmConfig places the Authorization header in customHeaders, and header names like Ocp-Apim-Subscription-Key defeat name heuristics. Match 'header' as a key fragment so every header-carrier object is dropped rather than relying on exact carrier names. * 🤐 fix: Strip Preconfigured Client Instances from Resume Params Bedrock stores a BedrockRuntimeClient on llmConfig.client when PROXY or a bearer token is configured; replaying it would also fold a mangled client object into additionalModelRequestFields on resume.	2026-07-07 07:22:50 -04:00
Ravi Kumar L	44d1275f36	⚙️ perf: reduce first-load MongoDB round trips (#14101) ... * perf(api): reduce first-load database round trips * docs: move agent guidance to claude docs * refactor(api): move message validation into api package * fix(api): narrow active generation job lookup * fix(api): preserve omitted source identity	2026-07-06 09:36:34 -04:00
Danny Avila	2d4ef52c22	🧮 fix: Prevent String Corruption of Numeric Agent Model Parameters (#14119) ... * 🧮 fix: Prevent String Corruption of Numeric Agent Model Parameters * 🧮 fix: Support Partial Numeric Input and Cover Parameter Aliases	2026-07-05 12:03:41 -04:00
Danny Avila	84fa6aa820	🧹 feat: Eager HITL Checkpoint Cleanup (Expiry + Deletion) & Full-Wiring E2E (#14123) ... * feat: eager HITL checkpoint cleanup on expiry + deletion, full-wiring e2e Follow-up to the lazy checkpointer (#14024): two paths still left a paused run's durable checkpoint to the 24h Mongo TTL, and no test exercised the whole HITL seam with real components. 1. Approval expiry: GenerationJobManager.setApprovalExpiredHandler(fn) — a non-destructive host hook fired after expireApproval's CAS succeeds (periodic sweeper AND stale-submit path), safe on startups that run constructor defaults. Both startups (index.js configureGenerationStreams, experimental.js) register a handler that prunes the checkpoint, resolving config lazily per expiry (streamId === conversationId === thread_id). 2. Conversation deletion: deleteConvos now returns the deleted conversationIds; the three deletion paths (DELETE /convos, DELETE /convos/all, account deletion) prune them via the new bulk deleteAgentCheckpoints (one $in deleteMany per collection). The delete routes gain configMiddleware for the checkpointer config. 3. Full-wiring e2e (hitlCheckpoint.e2e.spec.js): real SDK Run driven by FakeChatModel calling a gated tool, real PreToolUse/humanInTheLoop wiring, real LazyMongoSaver over mongodb-memory-server, real GenerationJobManager, real /resume controller via supertest. Asserts: clean turn persists nothing; error turn persists nothing; pause -> HTTP approve -> gated tool executes exactly once -> finalize prunes the checkpoint; expiry prunes the abandoned pause eagerly. Tests: 3 handler unit tests (pendingAction.spec), 2 bulk-prune integration tests (checkpointer.integration.spec), convos route + deleteUser specs updated, 4 e2e scenarios. 207 tests green across changed areas. * fix: tenant-scoped expiry prune, store-won expiry relay, resilient deleteConvos ids Codex round 1 on #14123 — all three valid: 1. The approval-expired handler now receives the expired JOB so both startups resolve config in the paused job's tenant/user scope (getAppConfig({userId, tenantId})) — a tenant checkpointer override no longer sends the prune to the base config's collections. expireApproval fetches the job best-effort. 2. Multi-replica: when RedisJobStore.cleanupRequiresActionIndex wins the expiry CAS on another replica, this replica's sweeper relay branch now runs the approval-expired cleanup too (prune is idempotent) — store-driven expiry no longer bypasses the hook. 3. deleteConvos: post-delete cleanup (deleteMessages, project stats refresh) is now best-effort — the conversations are already gone, so throwing hid the deletion and dropped the conversationIds the checkpoint prune needs, unrecoverable on retry. Updated the existing tag-decrement-on-failure test to the new contract (ids still returned). Tests: handler-receives-job, store-won relay path, ids-survive-cleanup-failure. 134 tests green across changed suites. * fix: relay cleanup independent of cached errorEvent; enter tenant ALS context Codex round 2 on #14123: 1. The sweeper's relay branch gated BOTH the terminal-error emit and the new checkpoint cleanup on !runtime.errorEvent — but a reconnect seeds errorEvent from the aborted job (runtime-state creation), which then suppressed the cleanup entirely. The emit stays gated; the idempotent cleanup now runs independent of the cached error, once per runtime lifetime (approvalCleanupRan flag — the aborted job is swept repeatedly). 2. Passing userId/tenantId to getAppConfig only keys the config cache; the Config query is ALS-scoped by the tenant-isolation plugin. Both startup handlers now ENTER the paused job's tenant context via tenantStorage.run before resolving config + pruning, so a tenant checkpointer override is honored in strict and non-strict modes. Tests: relay-cleanup-with-cached-error (reconnect simulation), repeated sweeps run the cleanup once. 27+4 green. * fix: dedup expiry cleanup across winner and relay paths Codex round 3 (P3): expireApproval ran the handler without marking the runtime's approvalCleanupRan flag, so the next sweep's relay branch (the aborted job outlives expiry for the completed-job TTL) ran the cleanup a second time. The dedup now lives inside runApprovalExpiredHandler — the single choke point both paths call — set-before-run, once per runtime lifetime. Test: local expiry followed by a sweep fires the handler once.	2026-07-05 11:29:30 -04:00
Danny Avila	ed8547018c	⚡ perf: Persist HITL checkpoints only on pause (lazy checkpointer) (#14024) ... * ⚡ feat: Persist HITL checkpoints only on pause (skip clean-exit writes) With `durability: 'exit'` (set by the SDK whenever a checkpointer is active) LangGraph persists ONE checkpoint at the exit boundary on EVERY run — paused or not. So a non-paused HITL turn writes a dead checkpoint whose only fate is to be pruned by deleteAgentCheckpoint: pure write+delete churn on the common path, given HITL only ever resumes an *interrupt* checkpoint. `InterruptOnlyMongoSaver` (a MongoDBSaver subclass) persists only interrupt checkpoints and discards clean-exit ones, so a non-paused turn writes nothing. How it tells them apart (verified empirically against @langchain/langgraph, not docs): when a run interrupts, the runner calls `putWrites` with the `INTERRUPT` ("__interrupt__") channel for the checkpoint it's about to create, and that write's `config.checkpoint_id` equals the `checkpoint.id` of the `put` that immediately follows. A clean exit calls `put` with no preceding interrupt `putWrites`. So we record the checkpoint id of any interrupt `putWrites` and persist a `put` only when its `checkpoint.id` was so marked. Keying on the globally-unique checkpoint id (not thread_id) keeps this correct even when two runs race on the same conversation (the job-replacement scenario). Correctness is preserved end-to-end: interrupt checkpoints + their pending writes persist exactly as before (resume unchanged); clean checkpoints were only ever written-then-pruned, so not writing them is observationally equivalent. The eager prune stays as the backstop. Tests (mongodb-memory-server): a bare put() is discarded; an interrupt-seeded checkpoint is persisted with its __interrupt__ pending write; and an end-to-end real-graph run writes 0 checkpoints on a clean completion and a resumable one on interrupt. NOTE: a non-paused turn's deleteAgentCheckpoint now finds nothing to delete (a 0-match no-op) — a follow-up can skip that call entirely once the lingering-abandoned-pause cleanup role is reassigned to the TTL + expiry sweeper. * ⚡ feat: Drop the redundant clean-path checkpoint prune With the lazy checkpointer (InterruptOnlyMongoSaver) a non-paused turn no longer writes a clean-exit checkpoint, so the post-completion prune in chatCompletion's finally had nothing left to delete. It was also already redundant: every fresh turn runs a pre-run prune (`deleteAgentCheckpoint` before `processStream`) that clears any checkpoint orphaned by a prior abandoned pause — verified empirically that a lingering interrupt checkpoint WOULD otherwise poison a fresh turn (LangGraph continues the abandoned state + re-interrupts), and that the pre-run prune is what prevents it. The Mongo TTL remains the backstop, and the resume path still prunes after a successful finalize. Removing the clean-path prune also deletes its job-replacement race surface (round-17 F21): an older run's late finally can no longer delete a newer paused run's checkpoint, because there is no longer a clean-path prune to race. Dropped the now-dead F21 predicate test. Net per non-paused HITL turn: from {pre-run prune + checkpoint write + post-run prune} down to {pre-run prune} — no write, no post-completion delete. * 🛡️ fix: Anchor any pending-write checkpoint; stale-only eviction (Codex) Broaden the lazy saver's keep-rule from "interrupt-only" to "persist any checkpoint that carries pending writes" (renamed InterruptOnlyMongoSaver → LazyMongoSaver). This makes it robust to delta-channel graphs without changing behavior for LibreChat's graph: - K1 (P1): a delta-channel graph can write a synthetic PARENT/anchor checkpoint (no __interrupt__ mark) that the interrupt checkpoint then points at, with the delta writes stored under the parent id. The old rule discarded that parent, breaking delta-state resume. Now any checkpoint that received putWrites is persisted, so the anchor parent and its writes survive and resume can walk the chain. - K3 (P2): for the same reason, clean delta-write rows are no longer orphaned — their checkpoint is persisted alongside them. (For LibreChat's standard Annotation/messages graph a clean run makes no putWrites at all — verified empirically — so the common path still writes nothing and the optimization is unchanged.) - K2 (P2): the 1024 FIFO cap could evict a valid in-flight id whose put() was just behind Mongo I/O, mis-classifying its interrupt checkpoint as a clean exit. Replaced with time-based eviction: only ids older than 5 min (a put always follows its putWrites within ms) are swept; a recent in-flight id is never dropped, and the map grows rather than evict a valid id if nothing is stale. New integration test: a checkpoint anchored by a NON-interrupt write is persisted. Full agents/HITL suites green (108). * style(checkpointer): fix import order to satisfy sort-imports CI * fix(checkpointer): don't persist failed-turn (error-only) checkpoints LazyMongoSaver anchored on ANY pending write, so a non-paused turn that errors (LangGraph records an __error__ write then a put) was persisted and, with the clean-path prune removed, lingered until the next fresh-turn prune or the Mongo TTL. Anchor only on resumable writes — INTERRUPT or a real (non-__-prefixed) state/delta channel — so error/bookkeeping-only checkpoints are discarded at the source. Addresses Codex P3. Codex P2 (delta-stub parent orphan) is not reachable: the SDK graph uses standard Annotation/MessagesAnnotation channels (no DeltaChannel), and under durability:'exit' putWrites precedes put with a parentless boundary checkpoint — probe-confirmed against @langchain/langgraph@1.4. Documented the durability:'exit' invariant the saver depends on. Tests: error-only put discarded; e2e throwing graph persists 0 checkpoints. * fix(checkpointer): drop bookkeeping-only write batches, not just the checkpoint The prior fix stopped the failed-turn CHECKPOINT from persisting, but putWrites still forwarded the __error__ batch to MongoDBSaver.putWrites — writing a row to agent_checkpoint_writes whose parent checkpoint is then discarded. With the post-run deleteThread removed, that orphan row lingered until the Mongo TTL or the conversation's next pre-run prune. putWrites now drops a non-resumable (bookkeeping-only) batch entirely instead of forwarding it. Probed against a real MongoDBSaver (mongodb-memory-server): a throwing graph now leaves 0 checkpoints AND 0 write rows (was 0 + 1 orphan), while interrupt->resume is unaffected — the __interrupt__ write is resumable so it is still forwarded. Addresses Codex P2 (round 3). Tests: error-only put leaves no checkpoint and no write row; e2e throwing graph leaves both collections empty; new e2e interrupt->resume completes with the approval value. * fix(checkpointer): un-anchor a checkpoint whose putWrites failed; freshen comments Self-review findings on the converged PR: 1. LangGraph dispatches put() concurrently with putWrites (probe-confirmed on 1.4.5), and put() still completes when putWrites rejects — so a transient Mongo failure during the interrupt write could persist a checkpoint whose __interrupt__ row is missing (an unresumable phantom pause). putWrites now deletes the write anchor on rejection (best-effort) and rethrows, so that put() discards the checkpoint instead. The pre-recorded anchor stays where it is — recording after the await would drop slow-I/O interrupts on the success path, which the same probe showed is reachable. 2. Renamed leftovers: two comments still said InterruptOnlyMongoSaver; the class is LazyMongoSaver. 3. Documented why the pre-run prune is deliberately unconditional per HITL turn (any cheaper gate can go stale across replicas and skip the prune exactly when an orphaned interrupt exists). Test: failed putWrites → subsequent put persists nothing (14/14 green). * fix(checkpointer): bookkeeping write batches follow their checkpoint's fate The round-3 rule dropped bookkeeping-only putWrites batches (__error__/ __resume__/__no_writes__) unconditionally — batch-scoped, when the decision must be checkpoint-scoped. Probe-confirmed (langgraph 1.4.5, durability:'exit'): a Send fan-out that pauses on one sibling records the completed siblings as pure __no_writes__ batches on the RETAINED interrupt checkpoint; dropping those markers makes resume re-execute the completed siblings (side effects measured twice). Addresses Codex M2 (P2). putWrites now PARKS a bookkeeping-only batch in memory until the checkpoint's fate is known: forwarded when the checkpoint is anchored (or was just persisted — put is dispatched concurrently), dropped when put discards it. Net: an errored turn still leaves nothing durable (0 checkpoints, 0 write rows), and a retained checkpoint stores byte-for-byte what a plain MongoDBSaver would. Codex M1 (__resume__ lost on re-pause) did not reproduce: the re-pause emits [__interrupt__,__resume__] as ONE batch (anchored, forwarded whole) and a second resume on a rebuilt graph replays both answers correctly — but the fate-scoped buffering now covers a lone __resume__ batch in any ordering too. Tests: bookkeeping preserved on a retained checkpoint in either arrival order; e2e Send-sibling pause/resume with side-effect counters (was {a:2,c:2} under the drop rule, now {a:1,c:1}); error-only turn still leaves both collections empty. 16/16 green.	2026-07-05 08:30:06 -04:00
Danny Avila	53b3e166d8	📉 perf: Skip Redundant Permission Queries on MCP Servers List (#14077)	2026-07-02 11:45:23 -04:00
Arjun Vijay	89931baf22	🚪 fix: Support Admin Redirect Detection for Same-Origin Subpaths (#14040)	2026-07-01 11:40:02 -04:00
Danny Avila	6dbf9d5ad3	🪝 feat: Human-in-the-Loop Runtime - Tool Approval + Ask-User-Question (Slice B) (#13942) ... * chore: add @langchain/langgraph-checkpoint-mongodb for HITL durable resume * feat: HITL tool approval runtime — backend (Slice B) - endpoints.agents.checkpointer config + durable Mongo checkpointer (seam over the app connection; SDK MemorySaver fallback) with a TTL index + deleteThread pruning - HITL run wiring (PreToolUse policy hook + humanInTheLoop) attached in createRun, fully inert when toolApproval.enabled is off - interrupt gate (pause job -> requires_action + emit on_pending_action) and a resume route that rebuilds the run from the durable checkpoint and run.resume()s it - atomic single-winner resolve; agent-consistency guard; expireStaleApprovals terminal event; checkpoint pruned on every non-paused completion (thread_id == conversationId) * feat: HITL tool approval UI — frontend (Slice B) approve/reject/edit/respond + ask-user controls in the tool card (OAuth-button precedent), batch-aware single submit, live + reconnect (resumeState.pendingAction) wiring, and resume mutations posting to /agents/chat/resume. * fix(hitl): decouple ApprovalProvider from chat context ApprovalProvider is now pure state (safe to mount in provider-less / shared / test renders); the context-dependent submit moved to a useResumeSubmit hook the cards call. Part imports getAskUserQuestionPart from ~/utils/approval directly so suites that partial-mock ~/utils render Part without throwing. * fix(hitl): address Codex review — backend - P1: enforce per-tool allowed_decisions on resume (reject a crafted decision the policy disallows) via findDisallowedDecisions - prune the durable checkpoint on user-abort of a paused run, and before a fresh HITL turn, so a new turn cannot rehydrate an expired/aborted interrupt (thread_id is the stable conversationId) - persist + use isTemporary and the original parentMessageId on resume (temporary chats stay temporary; initializeAgent scopes thread files off the right parent) - generate a deferred first-turn title BEFORE completeJob so its event reaches the client and the final event carries the real title - moderateText: skip when there is no text (tool-approval resume) and moderate the ask-user answer, instead of denying on an empty input * fix(hitl): address Codex review — frontend - render ToolApproval for ANY paused agent tool card (bash/code/file/etc.), not just the generic ToolCall, by wrapping the tool-card branch in Part (moved the rendering out of ToolCall) - findPendingActionMessageIndex only matches an assistant message, never the user message (the underscore-strip could target the user bubble before the assistant placeholder exists) * fix(hitl): address Codex re-review - title eligibility checks the user message’s parent (first turn), not the response’s parent — the previous check could never be true and skipped title generation - use client.buildResponseMetadata() for the resumed message so contextUsage / thoughtSignatures survive (the abort-only helper dropped them) - moderate decisions[].responseText (the respond action’s user text) - give /chat/abort req.config (configMiddleware) so the HITL checkpoint prune on abort actually runs - read resume state BEFORE setContentParts so the in-memory store does not lose the pre-pause seed content - count resumes against LIMIT_CONCURRENT_MESSAGES (increment/decrement) so paused-then- resumed turns cannot bypass the limit - require actionId on resume so a body without it cannot resolve the current action * fix(hitl): address Codex re-review (round 3) — resume fidelity Bring the lean resume path to parity with sendMessage for things it bypassed: - carry userMCPAuthMap into the rebuilt run so approved MCP tools keep the user's creds - seed initialSessions (buildInitialToolSessions) so approved code/file/skill tools have the pre-pause uploaded-file context (esp. cross-replica / after restart) - await client.artifactPromises and persist them as response attachments (else tool artifacts created after the pause vanish on reload / for late subscribers) - merge metadata: cumulative usage (+ summary marker) from the job, contextUsage / thoughtSignatures from the client — fixes the round-2 regression that underreported post-resume cost * fix(hitl): address Codex re-review (round 4) — resume hardening - resume: require an EXACT paused agent_id match (reject omitted/ephemeral agent_id, not just a different one) and reject an endpoint mismatch, so a request can't rebuild the claimed checkpoint on a different graph - moderateText: also moderate a tool-approval decision's reject `reason` and stringified `editedArguments`, not just `responseText` - request: re-mark the paused response `unfinished:true` after BaseClient saves it as completed, so an expired / never-resumed approval doesn't leave a "finished" response in history; the resume path overwrites it on success * test(hitl): route-level integration test for the resume controller Adds api/server/controllers/agents/__tests__/resume.spec.js, a supertest integration test that drives the real ResumeAgentController over the full pause -> approve -> resume -> finalize lifecycle with the SDK run, durable checkpointer, Mongo, and concurrency cache mocked. The pure decision/liveness helpers run for real via requireActual, so the guard ladder is exercised end to end rather than stubbed. 25 cases covering: - the authorization / staleness / agent-and-endpoint / actionId guard ladder - tool_approval validation (undecided tool call, policy-disallowed decision) - ask_user_question answer requirement - the concurrency gate (429) and the atomic single-winner claim (409) - the happy path: ACK, run reconstruction, decision->SDK mapping, finalize (save the now-finished response, emit done, complete job, prune checkpoint) - first-turn title generation before stream completion - re-pause (no double finalize), abort-during-resume (no double finalize), and the resume-failure terminal path (emitError + completeJob + prune) * test(hitl): strengthen resume coverage + add approval util tests Acts on a self-audit of the new resume integration test. resume.spec.js (25 -> 32 cases): - replace the tautological emitDone assertion (it only checked the hardcoded `final: true`) with a structural check of the finalEvent payload — responseMessage content/id/unfinished, requestMessage identity, title - cover the previously-unwalked finalize branches: tool-artifact attachments (null-filtered), the aggregatedContent fallback when live content is empty, and client response-metadata attachment - add guard cases: unsupported pending-action type (400) and the pre-multi-tenancy null-tenantId pass-through (must not 403) - add error-path cases: first-turn title generation throwing must still finalize, and a completeJob failure during a resume error must force a terminal job state via the last-resort updateJob client/src/utils/approval.spec.ts (new, 15 cases): - applyPendingAction tool_approval: join by tool_call_id not position, skip completed calls, default allowed_decisions to [], referential stability when nothing changes - applyPendingAction ask_user_question: append, idempotent replace on replay, non-array content coercion - getAskUserQuestionPart type guard; findPendingActionMessageIndex assistant-only resolution (never resolves to the user bubble) * fix(hitl): address Codex re-review (round 5) Five findings verified against the code before fixing: - resume: require an EXACT endpoint match (like agent_id) — a resume that OMITS endpoint must not fall through, since the shared chat middleware treats a missing/non-agents endpoint as the ephemeral agent and could rebuild the claimed checkpoint on a different graph - resume: filter malformed content parts before saving the finished response, matching the normal AgentClient path (a resumed turn could otherwise persist an empty/invalid tool_call part that breaks reload/rendering) - resume: accumulate tool artifacts across pause segments — persist them on re-pause and MERGE (not overwrite) at finalize, so artifacts produced before a second approval pause aren't dropped by the next rebuilt client - approval (client): findPendingActionMessageIndex returns -1 when a provided responseMessageId isn't found, so the caller retries instead of attaching the prompt/approval to a prior assistant reply; fall back to the last assistant only when no responseMessageId is given - RedisJobStore: make appendChunk extend-only (XADD + EXPIRE-if-shorter via a single eval) so the on_pending_action chunk emitted after a pause can't reset the chunk-stream TTL back to the running window and evict pre-pause content before the approval is resolved Tests: +endpoint-omitted/unsupported-type/malformed-filter/attachment-merge/ re-pause-persist cases in resume.spec.js (36); ask-retry -1 semantics in approval.spec.ts (16); extend-only TTL assertion in the RedisJobStore Redis integration spec. * test(hitl): mongodb-memory-server integration test for the checkpointer seam The checkpointer unit spec covers config/selection with no DB connection; this exercises the durable Mongo seam against a real (in-memory) MongoDB — the part correctness actually depends on: - getAgentCheckpointer builds a real MongoDBSaver when Mongo is connected and setup() creates the TTL index (expireAfterSeconds) on the checkpoint collection - memory type returns undefined (SDK MemorySaver fallback) even when connected - saver is memoized per resolved config - deleteAgentCheckpoint prunes a thread's persisted checkpoint (the cross-turn isolation guarantee: turn N+1 on the same conversationId can't rehydrate it) - pruning is thread-scoped — deleting one conversation leaves others intact - undefined threadId is a no-op * fix(hitl): address Codex re-review (round 6) Four findings verified against the code before fixing: - messageFilterPii: scan the resume payload's user-authored text (ask-user `answer`, and a tool-approval decision's `respond` text, `reject` reason, and edited tool arguments) — the shared /resume route ran through the PII filter but it only inspected req.body.text, so a blocked token rode the resume payload back into the model/tool (mirrors the earlier moderateText fix) - resume: re-prime skill files invoked in the pre-pause segment before rebuilding the run, so an approved code/file-backed tool keeps the injected skill-file session refs instead of running without them (mirrors the normal path's primeInvokedSkills; the pre-pause content stands in for the message payload) - hitl: pin the graph identity. Persist a fingerprint of the graph-determining request fields (endpoint, agent_id, model, spec, ephemeralAgent — normalized) on the pending action at pause, and reject a resume whose recomputed fingerprint differs. This closes the ephemeral-agent gap, where agent_id is undefined so the id guard can't tell two ephemeral configs apart - resume: reject incomplete edit/respond decisions (findIncompleteDecisions) — an `edit` without an object editedArguments or a `respond` without non-empty responseText is 400'd before mapping, rather than defaulting to {} / '' and resuming with behavior the user never approved Tests: incomplete-decision + fingerprint match/mismatch cases in resume.spec.js (41); findIncompleteDecisions + computeAgentRequestFingerprint unit tests; and resume-field PII cases in messageFilterPii.spec.ts. * fix(hitl): address Codex re-review (round 7) Four findings verified against the code before fixing: - RedisJobStore: clear `agent_id` on createJob (add it to staleHitlFields). The job hash is keyed by conversationId and reused across turns; updateMetadata only writes agent_id when truthy, so a conversation that switched from a saved agent to an ephemeral/no-agent turn kept the old id and the resume guard rejected the valid pause as a different agent. (real correctness bug) - fingerprint: include `promptPrefix` in computeAgentRequestFingerprint, and re-send it on resume (ResumeAgentFields + buildResumeFields). Ephemeral agents derive their system instructions from promptPrefix, so a resume changing it previously passed the pin and rebuilt different instructions. (completes the round-6 fingerprint) - resume: the re-pause branch now persists the segment's accumulated CONTENT (filtered), not just artifacts, so an approval that expires/reaps without a final resume no longer loses everything streamed during the resumed segment. - request: carry `manualSkills`/`alwaysAppliedSkills` on the persisted user message so a resumed turn's reconstructed requestMessage keeps its skill pills instead of dropping them until a full reload. Deferred (narrow, no safe contained fix yet — see PR thread replies): - resume rebuild without `addedConvo` for a multi-conversation/added-agent pane - cross-replica re-prime of manually-selected (not model-invoked) skill files Tests: stale-agent createJob clearing (Redis integration), promptPrefix fingerprint match/mismatch (resume.spec.js + policy.spec.ts), re-pause content persistence (resume.spec.js). * fix(hitl): address Codex re-review (round 8) Five findings verified against the code before fixing; the headline is a durable- resume correctness fix (the fingerprint had surfaced it as a 403): - resume durability (the important one): persist the graph-determining request fields (endpoint, agent_id, model, spec, promptPrefix, ephemeralAgent) on the pending action as `resumeContext`, and REPLAY them onto the resume request via a router-level middleware that runs before buildEndpointOption. The client can't reconstruct the ephemeral-agent config after a reload/cross-session, so the round-6/7 fingerprint would 403 a valid durable resume — and even without it the rebuilt agent would lose its tools. Replaying server-side rebuilds the SAME graph regardless of client state (and a crafted resume can't swap it; the fingerprint still matches because the body is restored first). - RedisJobStore: also clear `isTemporary` on createJob (same class as agent_id): a prior temporary turn's flag would otherwise survive a reused conversation hash and a later non-temporary resume would save its response as temporary. - resume: persist `contextMeta` (context-window calibration) onto the saved response like BaseClient does, so the next turn can seed its pruner. - request: carry manualSkills/alwaysAppliedSkills into the onStart metadata update (not just the preliminary one it overwrites), so a resumed turn's requestMessage keeps its skill pills. Deferred (narrow — see thread reply): - saved-agent edited WHILE a run is paused: agent_id matches but the definition changed; needs an agent version/config hash, which is a larger change for a narrow window. Tests: resumeContext pick/apply + round-trip (policy.spec.ts), contextMeta + manualSkills-on-requestMessage (resume.spec.js), isTemporary clearing (Redis integration). * style(hitl): prettier line-wrap in policy.spec.ts (R8 lint fix) * fix(hitl): address Codex re-review (round 9) Five findings, all fixed (addedConvo — deferred in rounds 7/8 — is now trivial thanks to the round-8 replay): - replay addedConvo: add it to RESUME_CONTEXT_KEYS so the resume middleware restores the parallel/secondary-agent config from the paused request; the client can't reconstruct it, and it determines the rebuilt graph. - skill pills (the real fix this time): the round-8 onStart metadata write was overwritten by trackUserMessage (the authoritative userMessage writer). Carry manualSkills/alwaysAppliedSkills in the emitted `created` message and persist them in trackUserMessage; widen UserMessageMeta + SerializableJobData.userMessage. - execute-code files on resume: seed the paused user message's own files onto req.body.files before initializeClient — they're excluded from the parent-walk code-session rebuild, so an approved code/read-file tool would otherwise resume without them. - in-memory pending-action UI: route ApprovalEvents.ON_PENDING_ACTION in the resume replay/pending-event loops to applyPendingActionToMessages (mirror the live handler), so a pause that lands in the snapshot window still renders its approval controls instead of sitting paused with no UI. - abort isTemporary: the /chat/abort partial-save now sources isTemporary from the job metadata, not req.body (the stop button posts only conversationId), so aborting a paused temporary chat no longer persists an orphaned partial. Tests: addedConvo in pickResumeContext (policy.spec.ts), file-restore on resume (resume.spec.js), abort-from-job-isTemporary (abort.spec.js). * fix(hitl): address Codex re-review (round 10) — resume/expiry races Three concurrency/coherence findings, verified against the code before fixing: - expiry-sweep CAS scope: both stale-approval sweeps (GenerationJobManager expireStaleApprovals and the RedisJobStore requires_action cleanup) called expire()/transitionStatus WITHOUT the observed pendingAction.actionId, so the CAS only checked status===requires_action. Between the read and the CAS a user could resolve the observed action and the run re-pause on a FRESH action; the stale sweep would then abort that valid new pause. Now both pass the observed actionId as expectActionId, so the CAS only fires for the action read as stale (a re-paused action has a different id → no-op). - resume graph cache: resumeCompletion cached the rebuilt graph (created with messages:[]) via setGraph; RedisJobStore.getContentParts prefers a cached graph over reconstructing from the chunk log, so a same-replica reload/status poll mid-resume returned aggregatedContent missing the pre-pause content. Skip setGraph on resume so introspection falls back to the complete chunk reconstruction (setContentParts still seeds the in-memory store). - pending-action UI: applyPendingActionToMessages scheduled a SINGLE animation-frame retry then dropped the pending action; Recoil/React updates can take several frames under load, leaving a valid requires_action run with no approval controls. Retry across frames (bounded at 120) until the target message commits. Test: expire() with a mismatched expectedActionId no-ops while the matching id expires (pendingAction.spec.ts). * chore(deps): update @librechat/agents to version 3.2.53 and @langchain/langgraph to version 1.4.7 in package-lock.json and related package.json files * refactor(hitl): add resolveToolApprovalPolicy seam for layered policy Extract the single point where tool-approval policy is resolved for a turn (`resolveToolApprovalPolicy`) and route the run call site through it instead of reading `endpoints.agents.toolApproval` inline. Behaviour-preserving: only the `endpoint` layer is wired today, so the result is identical to reading the app policy directly. The `agent` and `skills` layers are reserved seams with documented precedence (endpoint owns the `enabled` kill switch; agent overrides mode/allow/deny/ask/reason; skills may only tighten), so future per-agent and per-skill policy plumbing lands in one function rather than at the `createRun` site. Adds focused unit tests. * fix(hitl): address Codex re-review (round 11) — resume hardening F1 (P2, security) — applyResumeContext now DELETES any RESUME_CONTEXT_KEY absent from the persisted context, so the resume body carries exactly the graph-determining fields the pause had. Previously only defined keys were overwritten, leaving a client-supplied `addedConvo` (which the request fingerprint does not cover) in place — a crafted resume could rebuild a single-agent checkpoint as a different multi-agent graph/tool set. F3 (P2) — the resume route ACKs (res.json) before initializeClient, so a post-ACK getMCPRequestContext(req, res) saw the response as finished and returned undefined, leaving the resumed run without its run-scoped MCP connection store (approved MCP / OAuth-overlay tools then ran without their request-scoped connections). Pre-seed the store with a null res + cleanupOnResponse:false before the ACK and tear it down in the finally, mirroring the normal stream path (request.js). userMCPAuthMap was already preserved separately, so credentials were not lost — only the connection store. Declined: the ApprovalContext NEW_CONVO guard (P2) is a false positive — the `created` SSE event updates the conversation atom before any pause renders, so the id is concrete by click time (details in the PR thread). Tests: policy.spec (absent-key delete) + resume.spec (MCP context pre-seed/cleanup order). * fix(hitl): address Codex re-review (round 12) — resume fidelity + multi-tool UI F4 (P2) — temporal prompt vars: resume rebuilt the agent without restoring req.conversationCreatedAt or req.body.timezone, so {{current_datetime}}-style vars compiled a different system prompt than the paused graph (resume wall-clock, unzoned). Add 'timezone' to RESUME_CONTEXT_KEYS (persisted at pause, replayed by the resume middleware) and restore conversationCreatedAt from the convo before initializeClient — mirroring the normal path's resolveConversationCreatedAt. F5 (P2) — multi-tool approval: applyPendingActionToMessages stopped retrying once ANY tool-call part was tagged, so siblings that rendered on later frames never got approval controls and the resume route 400'd the partial batch. Add countTaggedApprovalParts and keep the bounded RAF retry going until every action_request is tagged (ask_user_question unchanged — one synthetic part). F6 (P3) — Edit accepted `null`/`[]` (valid JSON, non-object), enabling Submit for a value the resume route rejects via findIncompleteDecisions. Mirror the server's plain-object check in the client (store + editIsValid) so Submit only enables for an accepted value. Tests: policy.spec (timezone round-trip), resume.spec (conversationCreatedAt restore), approval.spec (countTaggedApprovalParts). * fix(hitl): address Codex re-review (round 13) — recurse into subagent approvals F9 (P2) — a tool paused INSIDE a subagent has its tool_call_id in the parent subagent tool_call's nested `subagent_content`, not as a top-level message part. applyToolApproval and countTaggedApprovalParts only scanned top-level content, so the approval never attached and the round-12 retry loop counted 0 tagged parts and spun to its frame cap with no controls. Both now recurse into `subagent_content` (immutably, so React refs update): the nested call gets tagged and is counted, so the retry terminates. Added approval.spec cases for the nested tag + count. Note: surfacing the interactive approve/reject controls inside the subagent view is a deliberate follow-up — ToolApproval -> useResumeSubmit -> useChatContext crashes when rendered in the portaled subagent dialog (outside the chat/approval providers), so that needs the controls scoped to the in-provider inline render (or the dialog wrapped with the providers). This commit fixes the data/traversal layer only. F7 (discovered-tool history on resume) and F8 (redis chunk TTL pause race) were verified false positives — see the PR threads. * fix(hitl): address Codex re-review (round 14) — resume fidelity + expiry relay F13 (P2) — manualSkills are graph-determining (skill allowed-tools union into the tool set before tools load) but weren't replayed, so a reload lost the skill tools and a crafted resume could inject a different skill past the fingerprint. Add 'manualSkills' to RESUME_CONTEXT_KEYS (same replay-only pattern as timezone/ addedConvo; the delete-absent half blocks injection). Not alwaysAppliedSkills — that's resolved server-side from the DB, not req.body. F12 (P2) — the resume final SSE built requestMessage from job.metadata.userMessage (persisted without files), so attachments vanished from the user bubble on resume. Spread the already-restored req.body.files onto it, matching the normal path. F11 (P2) — multi-replica approval expiry: RedisJobStore.cleanupRequiresActionIndex on another replica can win the requires_action->aborted CAS (it sets the hash error but has no event transport), and the local sweep then skips because the job is no longer requires_action, so a client subscribed here never gets the terminal error until the reap path. expireStaleApprovals now relays APPROVAL_EXPIRED_ERROR for a locally-subscribed job already aborted FOR approval expiry (error-string gated, idempotent via the errorEvent flag). emitError already publishes cross-replica. Tests: policy.spec (manualSkills round-trip + inject-drop), resume.spec (final requestMessage carries restored files). * fix(hitl): render approval controls for subagent-nested tool pauses (F10) Round-13 made applyToolApproval/countTaggedApprovalParts recurse into subagent_content (data), but SubagentDialogPart rendered nested TOOL_CALL parts with <ToolCall> only and never mounted <ToolApproval>, so a tool paused inside a subagent showed no controls and the run was unresolvable. Render <ToolApproval> in SubagentDialogPart's TOOL_CALL branch when the nested tool_call carries an approval and isn't yet resolved, mirroring the top-level Part.tsx render. The subagent dialog portals (OGDialog → ReactDOM.createPortal), but React context flows through the React tree, not the DOM tree, so ToolApproval resolves ApprovalProvider/ChatContext and the controls work + submit. Also harden useResumeSubmit: read ChatContext via useContext (non-throwing) instead of the throwing useChatContext wrapper, so the cards never crash when rendered outside a ChatContext.Provider (e.g. a search/citation render that passes chat context as a prop) — they degrade to inert (buildResumeFields returns null). * style(hitl): re-sort run.ts imports after dev rebase * fix(hitl): address Codex re-review (round 15) — resume content fidelity F14 (P2) — hide_sequential_outputs was applied in chatCompletion before saving/emitting content but not on resume, so a sequential-agent chain that pauses for HITL and resumes persisted/emitted intermediate outputs the setting is meant to hide. Extracted the filter into applyHideSequentialOutputsFilter() and call it from both chatCompletion and resumeCompletion (after handleRunInterrupt, covering the finalize + re-pause reads of client.contentParts). F16 (P2) — on a reloaded HITL pause, the DB already holds the paused user row + partial assistant row; useResumeOnLoad fed those as submission.messages, then finalHandler/createdHandler appended the same pair via requestMessage/responseMessage, duplicating the turn (buildTree doesn't dedupe children by messageId). buildSubmission- FromResumeState now strips the paused user/response rows (by messageId, incl. the padded/unpadded response id) from submission.messages — they're re-supplied by the placeholders + final event. Frontend-only; live (non-reload) pause path untouched. Deferred: F15 (collapsed-card subagent approval registration/visibility) — see thread. Tests: client.test (filter keeps last + tool_call parts / no-op when off), useResumeOnLoad.spec (paused pair stripped from submission.messages). * fix(hitl): address Codex re-review (round 16) — chunk TTL, slot, job replacement F17 (P2) — chunk-stream TTL on pause-before-chunk. CHUNK_APPEND_LUA derived its ceiling only from the chunk key's current TTL, so when the chunks key didn't exist at pause (fire-and-forget append in flight, or an ask-user pause before any chunk), the on_pending_action append created the stream with only the 20m running TTL while the approval window is 24h — content evicted before resume. The Lua now also reads the job key (KEYS[2]); when status == requires_action it takes max(running, TTL(jobKey)) (the approval window transitionStatus set), else the running TTL. Extend-only preserved; gated on paused status so normal runs never inflate. Both keys share {streamId} (cluster-safe). F19 (P2) — with LIMIT_CONCURRENT_MESSAGES, the approval prompt was emitted before the original request released its slot, so a fast Approve got /resume 429'd. handleRunInterrupt now releases the slot (idempotent via pendingRequestReleased) right after the pause, before the prompt; the request.js pause branch and resume.js finally only release if it didn't (no double-release). F20 (P2) — finalizeResumedTurn never checked the job wasn't replaced before emitDone/ completeJob/saveMessage, so a stale resume could clobber a newer turn that reused the conversationId. Added the createdAt guard the normal request path uses (skip finalization when the live job's createdAt != the paused job's). Deferred: F18 (subagent_content not reconstructed on Redis resume) — joins the subagent cluster (F15). See thread. Tests: RedisJobStore integration (pause-before-chunk gets approval TTL; running stays short), resume.spec (skip finalization on replacement; no double slot release on re-pause). * 🛡️ fix: Guard HITL terminal side-effects against job replacement Jobs are keyed by streamId == conversationId, so a new request REPLACES the running one on the same conversation. The replaced generation's tail must not clobber the live generation's state. Each path now re-reads the live job and compares createdAt against the generation's captured identity before acting. - Thread the generation's createdAt onto the client (request.js + resume.js) as client.jobCreatedAt — the identity every guard compares against. - handleRunInterrupt: skip approvals.pause when this run is no longer the live job, so a stale interrupt can't flip the NEWER job to requires_action. - chatCompletion finally: skip the checkpoint prune when replaced, so an older run's late finally can't delete the newer run's resume checkpoint. - resume catch-path: gate emitError/completeJob/prune behind a stillLive check (fail-open if the read throws), mirroring finalizeResumedTurn's success guard. - Persist the turn's uploaded files on job.metadata.userMessage (authoritative trackUserMessage writer) and prefer them on resume over the user DB row, whose save can still be racing a fast /resume. Tests: 13 guard-predicate cases in jobReplacement.spec.js. * 🔁 fix: Harden HITL resume — ownership re-check, file seeding, deferred-tool replay Three follow-ups to the round-17 job-replacement guards (Codex review 4594099963): - G1 (resume.js): the success-path ownership guard runs at the START of finalizeResumedTurn, but saveMessage + first-turn title generation await long enough for a new request to replace the job on the same conversationId. Re-read the live job immediately before emitDone/completeJob/prune so the terminal writes can't tear down the REPLACEMENT job — mirrors the catch-path guard. - G2 (request.js): onStart's metadata/chunk writes that persist the turn's files are fire-and-forget, so a fast approval could read job.metadata.userMessage before files landed. Seed files into getPreliminaryUserMessage instead — that write is AWAITED before the run starts, so files are durable before any interrupt can emit. - G3 (run.ts + client.js + resume.js + IJobStore.ts): the resumed graph is rebuilt with messages: [], so createRun's tool_search-discovery scan finds nothing. A deferred tool discovered earlier in the turn (and targeted by the paused call) was therefore absent from the rebuilt schema-only toolMap — resume would throw "unknown tool" (no loadRuntimeTools fallback is wired). Capture discovered tool names at pause via extractDiscoveredToolsFromHistory(run.getRunMessages()), persist them on job.metadata.discoveredTools, and replay them into createRun's new discoveredToolNames input (merged with message-extracted names, gated on hasAnyDeferredTools — inert otherwise). A new createRun test proves the deferred tool is promoted with the replay and absent without it (reproducing the bug). Tests: real createRun deferred-replay suite (run-summarization.test.ts) + G1/G2/G3 guard predicates (jobReplacement.spec.js). Full suite green. * 🔒 fix: Close HITL resume metadata + file-substitution + pause-race gaps Four findings on the round-18 commit (Codex review 4594430222): - H1 (P1, regression in round-18 G3): the discoveredTools captured at pause never reached resume — three metadata allowlists dropped it: GenerationJobManager .updateMetadata, RedisJobStore.deserializeJob, and buildJobFacade (plus the GenerationJobMetadata type). Added discoveredTools to all four, so the deferred-tool replay actually works end-to-end (in-memory store already kept it via Object.assign). - H2 (P2, security): /resume honored a client-supplied `files` array, letting a crafted client resume an approved code/read-file tool against a DIFFERENT file set than the one approved (files aren't in the resume fingerprint/context). Resume now ALWAYS sources files from the paused job (metadata → DB row), clearing any client-supplied set. - H3 (P2, ephemeral fidelity): non-default model parameters (temperature, max tokens, custom endpoint params) were lost on resume — ephemeral agents derive them from the request body, which the resume payload omits. Capture the resolved model_parameters in resumeContext at pause and replay them onto the body on resume (excluding `model`, which is replayed via the fingerprinted RESUME_CONTEXT_KEYS path). Saved agents already source these from the DB. - H4 (P2, Redis race): a pause landing between the resume snapshot and the Pub/Sub subscription reached neither resumeState.pendingAction nor (Redis) pendingEvents, and approval events aren't persisted to replayEvents — the client attached to a paused job with no approval UI. subscribeWithResume now re-reads the live job AFTER subscribing and surfaces the pending action if the snapshot missed it (live read, no staleness). Tests: discoveredTools metadata round-trip + subscribeWithResume re-read (pendingAction .spec.ts); client-file substitution rejection (resume.spec.js); model-parameter replay predicate (jobReplacement.spec.js). * 🧹 fix: Clear stale discovered tools, release slot on claim error, extend run-step TTL Three follow-ups on the round-19 commit (Codex review 4594783691): - I1 (P2): the round-19 discoveredTools field wasn't cleared on Redis streamId reuse. HSET only overwrites listed fields and handleRunInterrupt only writes discoveredTools when THIS turn discovers a deferred tool — so a replacement turn that pauses without its own discovery inherited the prior run's tool names and force-loaded undiscovered deferred tools on resume. Added discoveredTools to createJob's staleHitlFields HDEL list (the in-memory store already builds a fresh object, so it was Redis-only). - I2 (P2): with LIMIT_CONCURRENT_MESSAGES, approvals.resolve runs after the slot increment but before the run's try/finally, so a store/Redis error there leaked the slot until the counter TTL expired (spurious 429s on retry of the still-paused approval). Wrapped the claim in try/catch that decrements the slot and returns 500. - I3 (P3): saveRunSteps did SET ... EX running unconditionally, resetting the run-steps key to the 20-min running TTL even while the job is paused for the longer approval window — a reload after that window lost the tool timeline. Now uses a paused-window TTL script mirroring the chunk-stream no-shrink behavior (extends to the approval window when the job hash is requires_action). Also fixes a latent strict-tsc cast error in the round-19 pendingAction test. Tests: claim-throws-releases-slot (resume.spec.js); discoveredTools cleared on reuse + saveRunSteps preserves the paused TTL (RedisJobStore integration, USE_REDIS). * 🛡️ fix: Guard fast-resume save race, gate HITL to resumable routes, expire on stale submit Three findings on the round-20 commit (Codex review 4595045652): - J2 (P1): a fast /resume can claim + finalize the COMPLETED response while the original request's pause branch is still awaiting `response.databasePromise`; the later unfinished-save then overwrites the completed content. Re-check the job is still paused on THIS generation's action (a claim leaves requires_action; a replacement bumps createdAt) before marking the row unfinished; fail open on a read error. - J3 (P1): the tool-approval wiring (humanInTheLoop + PreToolUse hook + checkpointer) was applied to EVERY createRun caller when toolApproval.enabled, but the OpenAI-compatible and Responses controllers never inspect run.getInterrupt() or persist a pending action — an approval-gated tool would pause there with no approval surface or resume endpoint and the route would emit a normal final response / [DONE] with the tool call dangling. Gate the wiring on a new createRun `hitlCapable` flag, set only by AgentClient (chat + resume). - J4 (P2): a stale-action 409 on submit returned without driving expiry, leaving the job requires_action with a dead action until the periodic sweeper ran — any attached SSE client got no terminal event and the stream appeared to hang. Extracted GenerationJobManager .expireApproval(streamId, actionId) (expire CAS + terminal SSE, shared with the sweeper) and call it from the resume route when the observed action is stale. J1 (nested subagent approval controls not mounting while the details dialog is closed) is a valid frontend issue in the deferred subagent-HITL path — tracked separately (replied on the thread) since the fix touches the shared dialog primitive and needs UI verification. Tests: HITL-gate both directions (run-summarization.test.ts); expire-on-stale-submit (resume.spec.js); fast-resume unfinished-save guard predicate (jobReplacement.spec.js). * 💄 style: Wrap captureAgents signature to satisfy prettier (CI lint)	2026-06-29 16:56:41 -04:00
Danny Avila	12fea693bb	🦥 perf: Lazy-Load Agent Version History in Editor (#13977) ... Opening the agent editor fetched the full `versions` array (each a complete config snapshot) alongside the agent, so agents with large histories were slow to open. Version history is now loaded only when the user opens it. - Add `getAgentWithVersionCount` (aggregation: version count, no versions array) and `getAgentVersions` data-schemas methods. - `getAgentHandler` returns the version count without the heavy array; add `GET /agents/:id/versions` (EDIT-gated) for lazy retrieval. - Add `useGetAgentVersionsQuery`; VersionPanel reads current config from the cached expanded query and fetches versions on open. Revert keeps the expanded cache and versions query in sync.	2026-06-26 12:19:54 -04:00
Peter	abf9fc307d	📇 feat: Agent Contact Visibility with Owner Fallback (#13663) ... * Shared Contract * Backend Resolution * Frontend Display * Contact Styling and more tests * fix contact flicker when saving an agent * fix display owner when contact deleted * simplification of the last fixes * github action fixes * fixes failing tests --------- Co-authored-by: Peter Rothlaender <peter.rothlaender@ginkgo.com>	2026-06-25 15:58:15 -04:00
Danny Avila	376370d610	♻️ refactor: Compute Context Gauge Client-Side, Drop Projection Endpoint (#13953) ... * ♻️ refactor: Compute Context Gauge Client-Side, Drop Projection Endpoint The /api/endpoints/context-projection endpoint re-fetched a conversation's messages from Mongo and re-tokenized them to project the context gauge for snapshot-less branches. The browser already holds those messages and their per-message tokenCounts, so this duplicated work on the request path (an unbounded read + server-side BPE tokenization until it was later capped). Move the snapshot-less estimate fully client-side, from the in-memory index: - sumBranch accumulates an uncalibrated char/4 estimate (estTokens) for count-less messages (imports / pre-feature) under the same summary cutoff - useTokenUsage folds estTokens (calibrated via the existing calibrationFamily ratio) into the existing fallback; known per-message counts render unchanged - delete the endpoint, controller, rate limiter, route, the getMessageTextStats data-schemas method, and the data-provider surface (endpoint/key/type/service/query) No DB read, no server tokenization, no rate-limit knobs; the gauge recomputes reactively from the index. Net -793 lines. * 🩹 fix: Count quotes and object-form content in client context estimate Address Codex review on the client-side context estimate: - messageChars now reads object-form content text (part.text.value), not only string text/think, so imported / pre-feature messages whose body lives in content parts are no longer estimated as zero. - Count-less user messages include their merged quote excerpts in the estimate, mirroring what the send path prepends into the prompt. * 🩹 fix: Cap over-window estimate and surface estimated tokens in breakdown Address remaining Codex review on the client-side context estimate: - Clamp the snapshot-less estimate's displayed usedTokens to maxTokens. The send path prunes an over-window branch before calling the model, so the gauge never actually exceeds the window; this avoids impossible values (e.g. 50k / 8k) without re-introducing client-side pruning. - Surface the calibrated count-less estimate as its own "Estimated" row in the breakdown popover, so a branch of only count-less imported / pre-feature messages is no longer shown as Input 0 / Output 0 under a non-zero header. * 🩹 fix: Refine client context estimate per Codex re-review - Drop calibration from the snapshot-less estimate. The removed projection never actually calibrated (the client never sent a ratio), and a ratio inflated by provider-injected context over-estimates visible imported text. - Exclude reasoning (think) / error parts from the estimate; the send path strips them, so they are not part of the next call's context. - Fold quote text into the estimate even when a tokenCount is present, since the edit route recounts tokenCount from text only and drops the merged quote. * 🩹 fix: Recount quoted user turns instead of topping up the stored count The previous round added quote chars on top of a quoted message's stored tokenCount, which double-counts the common (unedited) case where the count already includes the merged quote prompt. Match the removed projection instead: for quoted user turns, ignore the stored count and estimate the full merged text. This both avoids the double-count and still corrects the stale text-only count an edit leaves behind. * 🩹 fix: Trust stored counts for quoted turns; count tool-call parts - Quoted user turns: revert to trusting a present tokenCount. The send path's stored count already includes the merged quote (and any calibration), and the client's char/4 path is coarser, so recounting regressed normal turns. Only count-less messages estimate quotes from text. - Count tool-call name/args/output for count-less assistant messages; the formatter sends them back as context, so omitting them under-reported imported branches with tool history. * 🩹 fix: Exclude in-flight tail from estimate to avoid resume double-count On resume the live path seeds liveTokens from the partial response and also writes that content into the messages cache, where the count-less response is estimated into estTokens too — double-counting the in-flight output on the snapshot-less estimate path. sumBranch now exposes the tail message's own estimate (tailEstTokens); the estimate path drops it while a stream is live, so the in-flight response is counted once (via liveTokens). The breakdown's Estimated row uses the same in-flight-adjusted value. * 🩹 fix: Recount quoted user turns in context estimate (match send path) A quoted user turn's stored tokenCount is unreliable for the gauge: a text-only Save edit recomputes it from text alone, and the send path (needsCanonicalTokenCount in agents/client.js) recounts the quote-merged prompt every turn regardless of the stored value. Mirror that on the client — estimate quoted turns from the merged text+quotes and ignore the stored count — so snapshot-less branches don't under-report by the quote block. Reverts the earlier "trust the count" assumption, which the server disproves. * 🧹 chore: Route useResumableSSE diagnostics through the frontend logger Convert the [ResumableSSE]/[Debug] console.log and console.error diagnostics to the gated frontend `logger` (client/src/utils/logger), splitting the tag from the message so object arguments are passed through as real args (logged expandably, not stringified) and the logs stay tag-filterable and off the production console unless explicitly enabled. All log statements preserved; nothing removed. * 🩹 fix: Prefer content over text when estimating count-less messages A stopped agent response is saved with both a `text` field and a structured `content` array, and the send path formats from content. messageChars early-returned on `text`, dropping the content array (and the tool-call tokens it carries) from the snapshot-less estimate — also making the tool_call handling dead for such messages. Prefer content when present, fall back to text.	2026-06-25 15:29:31 -04:00
Danny Avila	397ddc5366	🧠 feat: Add Memory as an Agent Capability with Inline Tools and Ephemeral Badge (#13869) ... * 🧠 feat: Memory Agent Capability with Inline Tools and Ephemeral Badge Add `AgentCapabilities.memory`, which expands into the inline set_memory/delete_memory tool pair (mirroring the execute_code expansion via registerMemoryTools) when a run-level memoryAvailable gate holds: capability enabled, memory configured, MEMORIES.USE permission, and personalization not opted out. Surfaces the memory artifact as an attachment in the agents tool-end callback. Adds the ephemeral path (TEphemeralAgent.memory, load/added agent tool injection), a fully-gated memory badge plus tools-dropdown entry, the agent-builder Memory toggle with form round-trip, and a mock e2e test asserting the badge reaches the request payload. Additive to and independent of the existing post-turn memory extraction agent. * 🩹 fix: Address Codex review on memory capability (gating, validKeys, usage guard) - Strip the memory capability from the served agents capabilities when memory is not configured/enabled, so the badge, tools dropdown, agent-builder toggle, and backend capability gate stay consistent instead of exposing an inert toggle on default installs (where MEMORIES.USE defaults true). - Surface configured memory.validKeys in the inline tool definitions so the model is told the allowed keys up front, matching the runtime createMemoryTool schema. - Append a strict explicit-request usage guard to the agent instructions when inline memory tools are registered, preserving the memory-agent's privacy behavior. - Add AppService tests covering memory-capability stripping. * ✅ test: Update AppService capability snapshots for memory strip AppService now strips the memory capability from the served agents defaults when no memory block is configured; update the spec's expected capability lists to defaultAgentCapabilitiesWithoutMemory for the no-memory-config cases. * 🛡️ fix: Address Codex re-review on memory capability (round 2) - Strip the memory capability from the FINAL served agents config, not just defaults; loadEndpoints reparses any endpoints.agents block, so memory was still exposed in that common shape (packages/data-schemas/src/app/service.ts) + regression test. - Re-check the full memory gate (config, opt-out, MEMORIES.USE) inside handleTools before constructing set_memory/delete_memory, so an unsolicited tool call from a model/custom endpoint can't bypass the runtime gates (api/app/clients/tools/util/handleTools.js). - Restore the persisted memory toggle for model-spec conversations via applyModelSpecEphemeralAgent (client/src/utils/endpoints.ts). - Clear LAST_MEMORY_TOGGLE_ on logout and clear-all-chats so a stale memory preference can't leak across users on a shared browser (client/src/utils/localStorage.ts). * 🧠 fix: Address Codex re-review on memory capability (round 3) - Serialize set_memory writes and advance a running token total inside createMemoryTool, so parallel batched calls in one event-driven turn can't each pass the limit check against a stale total and collectively exceed memory.tokenLimit (packages/api/src/agents/memory.ts) + tests. - Inject the keyed memory context (withKeys) instead of withoutKeys when the running agent has the inline memory capability, so delete_memory has a visible key to target (api/server/controllers/agents/client.js). * 🔐 fix: Address Codex re-review on memory capability (round 4) - Detect inline memory by tool NAME (set_memory/delete_memory) across an initialized agent's tools + toolDefinitions, since the 'memory' marker is expanded at init and the prior string check never matched; inject the keyed memory context for any primary OR sub-agent that carries the inline memory tools (api/server/controllers/agents/client.js). - Enforce memory WRITE permissions in the inline tool gate: set_memory requires CREATE+UPDATE and delete_memory requires UPDATE (matching the REST memory routes), so a USE-only role can't mutate/delete memories via agent tool calls (api/app/clients/tools/util/handleTools.js). * 🔒 fix: Address Codex re-review on memory capability (round 5) - Gate inline memory registration (memoryAvailable) on the memory WRITE permissions (USE+CREATE+UPDATE), so a read-only-memory role no longer has set_memory/delete_memory shown to the model only for the runtime loader to refuse them (api/server/services/Endpoints/agents/initialize.js). - Enforce the per-agent memory opt-in at execution: handleTools now refuses to construct set_memory/delete_memory unless the agent actually declared them (toolDefinitions/tools), blocking hallucinated/undeclared memory tool calls from mutating memory. - Fail closed when getFormattedMemories errors with a configured tokenLimit, instead of writing as if storage were empty and bypassing the cap (api/app/clients/tools/util/handleTools.js). * 🩹 fix: Address Codex re-review on memory capability (round 6) - Fix a P1 regression from the prior round: the execution-context agent keeps the raw 'memory' capability marker (not the expanded set_memory/delete_memory names), so the opt-in check now matches the marker. This restores memory writes/deletes AND avoids hijacking an MCP tool that merely shares the set_memory/delete_memory name (api/app/clients/tools/util/handleTools.js). - Count repeated set_memory writes to the same key as replacements, not additions, against tokenLimit — set_memory upserts, so a same-key rewrite swaps its prior token contribution instead of double-counting (packages/api/src/agents/memory.ts) + test. - Gate the memory badge, tools dropdown, and agent-builder toggle on the full memory write permissions (USE+CREATE+UPDATE) via a shared useHasMemoryAccess hook, so a read-only-memory role no longer sees an enabled Memory control the backend would refuse to wire up. * 🧷 fix: Address Codex re-review on memory capability (round 7) - Recognize inline memory across both execution-context agent shapes: initializeAgent now sets a LibreChat-only memoryToolsRegistered flag on the InitializedAgent, and the opt-in/detection checks accept that flag OR the raw 'memory' marker. Fixes memory failing for processAddedConvo agents (which store the initialized config, marker already expanded) while staying MCP-name-collision-safe (api/app/clients/tools/util/handleTools.js, packages/api/src/agents/initialize.ts, api/server/controllers/agents/client.js). - Scope keyed memory context to memory-enabled agents only: useMemory now returns both keyed and unkeyed contexts, and buildMessages injects the keyed one (memory keys + token metadata) only to agents that can call delete_memory, while the primary/post-turn path keeps the unkeyed values — so a primary without memory tools no longer sees memory keys it doesn't need. * 🔏 fix: Address Codex re-review on memory capability (round 8) - Enforce memory size limits on inline writes: createMemoryTool now rejects keys over 1000 chars and values over memory.charLimit, matching the REST memory routes, so an inline-memory agent can't persist blobs the memory UI/API would reject (packages/api/src/agents/memory.ts, api/app/clients/tools/util/handleTools.js) + test. - Recheck the agents 'memory' endpoint capability at execution time, so a stale/hallucinated set_memory/delete_memory call can't mutate memory after an admin removes the capability while the agent document still carries the marker (api/app/clients/tools/util/handleTools.js). * ♻️ refactor: Move inline-memory backend logic into packages/api + share memory load Workspace boundary: the inline-memory gating/detection logic that had crept into /api now lives in packages/api/src/agents/memory.ts (TS), with /api kept as thin wrappers. - Add agentHasInlineMemoryTools, isMemoryToolAllowed, and buildInlineMemoryTool to packages/api; handleTools.js now calls buildInlineMemoryTool instead of constructing/gating the tools inline, and client.js imports agentHasInlineMemoryTools instead of redefining it. - Optimize repeated memory loads: getRequestMemories memoizes getFormattedMemories per request (WeakMap keyed by req), so the run's memory-context load and every memory-enabled agent's set_memory token-usage load share a single DB fetch instead of one per agent. * 🧠 fix: Invalidate request memory cache after inline writes Inline set_memory/delete_memory now invalidate the request-scoped getFormattedMemories cache on a successful write, so a later tool round in the same response is seeded with the post-write usage total instead of the stale pre-write one (multi-round writes no longer collectively exceed tokenLimit, and a set after a delete is not over-counted). The within-round sharing across multiple memory-enabled agents is preserved. * 🧠 fix: Persist memory capability on saved agents; honor registration flag - Add Tools.memory to the v1 systemTools allowlist so filterAuthorizedTools no longer silently drops the memory marker when an agent with the Memory capability is created/updated/duplicated through the builder (previously the capability only worked for ephemeral chats, not persisted agents). - agentHasInlineMemoryTools now honors an explicit memoryToolsRegistered boolean before falling back to the raw `memory` marker, so an initialized config whose registration was denied (memoryAvailable false) is not given keyed memory context just because the marker survives in tools. * 🧩 fix: Bring memory tool to parity with other ephemeral tools - Add `memory` to the model-spec schema/type and honor `modelSpec.memory` in both ephemeral paths (load.ts, added.ts) and the frontend spec application, so admins can pre-enable Memory from a model spec exactly like webSearch/fileSearch/executeCode. - Add LAST_MEMORY_TOGGLE_ to the timestamped-storage cleanup list so stale per-conversation memory toggles are purged on startup like the others. - Hide the agent-builder Memory toggle for users who disabled memory in personalization (memories === false), mirroring the chat badge's opt-out gate, so the setting isn't shown as inert/misleading. * ✅ test: Cover memory in applyModelSpecEphemeralAgent spec defaults Update the exact-object assertions to include the new `memory` field and add positive coverage that `modelSpec.memory` maps to the ephemeral agent's `memory` flag. Fixes the shard 2/4 failure from 672a03b05.	2026-06-24 17:14:13 -04:00
Marco Beretta	b84e26671e	🕒 feat: Track Terms Acceptance Timestamp (#10810) ... * feat: add terms acceptance timestamp tracking and migration script * feat: update migration script to use countUsers method for user count * Update config/migrate-terms-timestamp.js Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * feat: enhance terms acceptance response to include acceptance timestamp * fix: make terms acceptance idempotent and fail migration on partial errors Preserve the original termsAcceptedAt on repeat accepts within a terms cycle so retried or duplicate requests no longer overwrite the first acceptance time. Exit the migration script with a non-zero status when any per-user update fails so partial failures are not reported as successful. * style: fix import ordering in data-provider mutations * refactor: record terms acceptance atomically to preserve first-accept time Replace the read-then-write in acceptTermsController with a single atomic acceptTerms method that conditionally stamps termsAcceptedAt via an $ifNull aggregation update. This removes the TOCTOU window where two concurrent first-time accepts could overwrite the earlier acceptance timestamp, while still preserving an existing timestamp and backfilling legacy accepted users. * fix: run terms timestamp migration under system tenant context Wrap the count, cursor scan, and per-user updates in runAsSystem so the tenant isolation plugin does not throw under TENANT_ISOLATION_STRICT or scope the cross-tenant migration to a non-existent tenant, matching the other maintenance migrations. * fix: guard terms backfill against concurrent acceptances Add the missing-timestamp predicate to the per-user updateOne filter so a user who accepts through the API between the cursor read and the write keeps their real acceptance time instead of being overwritten with createdAt. Track modified vs skipped so the summary reflects skips. * fix: scope terms backfill to still-accepted users Add termsAccepted: true to the per-user updateOne filter so a reset that clears acceptance between the cursor read and the write is not re-stamped with createdAt, which would otherwise poison the next acceptance cycle through the $ifNull preserve in acceptTerms. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2026-06-24 16:26:42 -04:00
Danny Avila	1662adc581	📺 feat: Google URL Context Param with Native YouTube Video Understanding (#13924) ... * ✨ feat: Add Google url_context Param with Native YouTube Video Understanding Mirror the web_search grounding wiring for a new Google/Gemini `url_context` model param (resolves to the native `urlContext` tool). When enabled, YouTube URLs in the latest user message are injected as Gemini video parts (fileData), since the URL Context tool does not support YouTube. * 🎞️ fix: Provider-aware YouTube injection limits for url_context Address Codex review on the YouTube video-understanding path: - Cap injected YouTube parts per request by provider/model (Vertex: 1; Gemini Developer API: 10 on 2.5+, 1 on earlier models) so multi-link messages cannot exceed the provider limit and get rejected. - Set a video/mp4 mimeType on Vertex YouTube fileData (matching Vertex samples); the Developer API still omits it. * 🧩 fix: Round-trip url_context for Google-compatible custom endpoints Add url_context to openAIBaseSchema so the per-chat value persists for custom endpoints configured with customParams.defaultParamsEndpoint: 'google', matching how web_search is already picked there. * 🚦 fix: Gate url_context tool to Gemini 2.5+ models Per Google's URL Context supported-models list (2.5+/3.x only), skip the native urlContext tool on earlier models (debug-log + no-op) instead of sending it and triggering a provider 400. This also gates the coupled YouTube video-understanding injection to 2.5+, since it keys off the resolved urlContext tool. * ✂️ fix: Strip YouTube URLs from urlContext text; keep url_context out of OpenAI schema - Remove url_context from the shared openAIBaseSchema (revert): it is Google-only and would otherwise leak as an unsupported param to OpenAI/Azure/OpenRouter requests. On Google-compatible custom endpoints url_context is enabled via admin addParams/defaultParams, same as web_search. - When injecting YouTube video parts, strip the matched YouTube URLs from the prompt text so the urlContext tool (which reads URLs from text and cannot fetch YouTube) does not consume its URL budget on them. Non-YouTube URLs are left intact. * 🎯 fix: Refine url_context model gating and YouTube injection edges Address Codex round 4: - Exclude non-text modality variants (image/live/tts) from URL Context support, mirroring the Google tool-combination modality exclusion. - Use the resolved run model (model_parameters.model) for YouTube injection limits instead of the saved base model. - Strip only the YouTube links actually routed to video (id-aware); keep over-limit links in the text so the model can still reason about them. - Keep timestamped YouTube links (?t=/&start=) in the text so the moment cue survives. - Recognize youtube-nocookie.com/embed links. * 🎚️ fix: Exclude audio Gemini variants + preserve pre-id YouTube timestamps Address Codex round 5: - Add `audio` to the url_context modality exclusion so audio-only Gemini variants (e.g. gemini-2.5-flash-preview-native-audio-dialog) skip the tool instead of 400ing. - Detect YouTube timestamps anywhere in the matched URL (incl. before `v=`, e.g. watch?t=90&v=<id>), so timestamped links are kept in the prompt text as intended.	2026-06-23 22:42:06 -04:00
Danny Avila	d9a76fca90	🧠 feat: Configurable Reasoning Replay for Custom Endpoints (#13921) ... * 🧠 feat: Configurable Reasoning Replay for Custom Endpoints Adds customParams.includeReasoningContent so OpenAI-compatible custom endpoints (e.g. Xiaomi MiMo, Kimi) can replay reasoning_content on tool-call turns natively, without impersonating the moonshot provider. * 🔁 feat: Replay reasoning_content across turns for opted-in custom endpoints Extends the DeepSeek reasoning-content format spoof to honor customParams.includeReasoningContent, so custom OpenAI-compatible endpoints (Xiaomi MiMo, Kimi) reconstruct reasoning_content from persisted history on later turns, matching DeepSeek thinking-mode parity. Adds shouldReplayReasoningContent predicate (tested) and surfaces the flag on the initialized agent. * 🪢 refactor: Split within-run vs cross-turn reasoning replay flags moonshot only replays reasoning_content within a run's tool calls, not across turns. Decouples the two: includeReasoningContent = within-run replay (exact moonshot parity), new includeReasoningHistory = cross-turn reconstruction from persisted history (implies includeReasoningContent, since reconstruction is a no-op without the within-run replay flag). * 🩹 fix: Apply reasoning replay across all param-format branches Move the within-run includeReasoningContent application out of the OpenAI-only branch in getOpenAIConfig to after the branch dispatch, so custom endpoints using anthropic/google defaultParamsEndpoint gateway modes also honor includeReasoningContent/includeReasoningHistory. Addresses Codex finding. * chore: Update @librechat/agents to v3.2.46 * 🧽 refactor: De-spoof reasoning replay via explicit preserveReasoningContent Now that @librechat/agents 3.2.46 exposes an explicit preserveReasoningContent option on formatAgentMessages, pass it directly instead of impersonating provider: deepseek. Behavior is unchanged (shouldReplayReasoningContent still gates DeepSeek + the custom includeReasoningHistory flag); also corrects the comment to reference includeReasoningHistory. * 🌳 fix: Walk subagents in the reasoning-history replay gate The gate only checked the primary agent and top-level handoff/parallel configs, so an opted-in custom endpoint used solely as a nested subagent had its persisted reasoning dropped on later turns. New exported anyAgentReplaysReasoningContent walks subagentAgentConfigs (cycle-safe, mirrors anyAgentHasCodeEnv); client.js uses it. Addresses Codex finding.	2026-06-23 21:08:47 -04:00
Danny Avila	1eb460eb03	🧾 fix: Harden Historical File Authorization (#13918) ... * fix: Harden historical file authorization * chore: Sort file authorization imports * fix: Preserve authorized historical artifact refs * chore: Format historical artifact hardening	2026-06-23 15:49:57 -04:00
Danny Avila	bc6b032421	🛑 refactor: Demote User Abort Logs (#13904) ... * fix: Demote user abort logging * fix: Handle abort causes * fix: Demote user-aborted agent completion to debug log The error users still saw originated in AgentClient's completion catch, which logged every caught error (including user aborts) at error level before checking the abort signal. Branch on abortController.signal.aborted so user-initiated aborts log at debug while real failures stay error-classified. Also give the handleAbortError it.each cases distinct titles.	2026-06-23 09:55:21 -04:00
Danny Avila	77854decdf	🪣 fix: Cap Context Projection Workload Before Tokenization (#13910) ... * fix: bound context projection workload * fix: Address context projection CI failures * fix: Bound context projection database reads * fix: Sort projection spec imports * fix: Cap projection body reads with stats	2026-06-23 08:43:09 -04:00
Danny Avila	5eb1c2c107	🖇️ feat: Reference Selected Chat Text with Multi-Quote Popup (#13868) ... * 🖇️ feat: Reference Selected Chat Text with Multi-Quote Popup Add a ChatGPT/Codex-style quote feature: selecting text in any message shows an 'Add to chat' popup that accumulates removable quote chips above the composer. On submit, the excerpts are merged into the user message text as Markdown blockquotes (counted in the user message token count, not a system message) and persisted on the message so they render on the user bubble and survive reload. - packages/api: add getReferencedQuotes + mergeQuotedText helpers (blockquote merge, length/count caps) with unit tests - BaseClient.sendMessage: temporarily merge req.body.quotes into userMessage.text before buildMessages, restore clean text, persist quotes array - data-schemas + data-provider: add optional quotes field to message schema/type - client: pendingQuotesByConvoId atom, QuoteButton selection popup, PendingQuoteChips composer row, MessageQuotes persistent display - useChatFunctions: drain pending quotes onto the message, carry forward on regenerate - add localization keys and component/integration tests * 🧪 test: Add Playwright e2e for chat quote feature Add e2e/specs/mock/quotes.spec.ts covering select -> 'Add to chat' popup -> chip -> send -> persistent reference block -> reload, plus multi-select accumulation and chip removal. Selection is driven programmatically (real DOM Range + dispatched mouseup) to summon the popup deterministically. Add data-testid hooks (add-to-chat-button, pending-quote-chips, message-quotes) to the quote components for stable selectors. * 🛡️ fix: Address Codex review on quote feature - Run PII filter + OpenAI moderation over req.body.quotes (P1): quoted excerpts are merged into the model-facing user message, so they must clear the same filters; a crafted quotes payload could otherwise bypass them. Adds tests. - Carry quotes through edit/save-and-submit replays (overrideQuotes in EditMessage), mirroring overrideManualSkills, so edited turns keep context. - Hide the quote UI for Assistants endpoints (which bypass BaseClient merge), so users can't queue quotes the assistant never receives. - Clear pending quote/skill queues by resolved conversationId in useClearStates, not the UI index, so queued-but-unsent selections don't linger in Recoil. - Cap queued quotes client-side at 10 to match the backend QUOTE_MAX_COUNT, so the composer never shows more quotes than are actually sent. * 🧵 fix: Durably re-merge quotes + Codex round 2 Address Codex's re-review of the quote feature: - Durable history re-merge (per maintainer decision): quotes are no longer merged at request time and stripped; instead each user message's persisted message.quotes is merged into its formatted content in AgentClient.buildMessages (new prependQuotes helper) for current AND historical turns. The model receives the referenced context on every prompt and the token count stays consistent with what was persisted; stored text stays clean for display. - Attach normalized quotes to the user message in handleStartMethods (before getReqData/onStart) so the optimistic bubble, resumable abort metadata, and saved row all carry them (fixes the abort-metadata gap). - Skip the quote drain entirely for Assistants endpoints in useChatFunctions, leaving the pending atom intact (UI is already hidden there). - Normalize req.body.quotes via getReferencedQuotes before moderation/PII so only the trimmed/truncated/capped excerpts the model will receive are checked. - Tests: prependQuotes unit tests; BaseClient quote tests assert early attachment + clean text; e2e now verifies the model receives the merged blockquote on the current turn and re-merged from history on a later turn (new E2E_ASSERT_QUOTE mock marker). * 🔗 fix: Quote share/memo/abort/PII gaps (Codex round 3) - Shared links: include quotes in the anonymized projection + SharedMessage type (+test) so the /share view renders the same reference blocks as the owner, mirroring manualSkills/alwaysAppliedSkills. - MessageRender memo: compare quotes length so a server/resume copy whose only change is the quote list re-renders (the block no longer goes stale/missing). - Resumable job metadata: include quotes in the userMessage written to GenerationJobManager so a reload/reconnect mid-stream reconstructs the chips. - PII + moderation: also scan the merged blockquote+text exactly as the model receives it, so a secret split across a quote and the typed body (each clean alone) is caught (+cross-boundary test). - e2e: make quote-add robust against the auto-scroll-dismisses-selection race via a retried select+click helper. * 🛑 fix: Keep quotes on aborted turn's request message (Codex round 4) abortMiddleware reconstructs finalEvent.requestMessage from jobData.userMessage but only copied ids + text; include quotes so a stopped quoted turn keeps its MessageQuotes in the UI and a regenerate-before-reload still sends the referenced context. Completes the resumable-metadata fix from the prior round. * 🧮 fix: Quote recount + preliminary abort metadata (Codex round 5) - Force a canonical token recount for messages carrying quotes in AgentClient.buildMessages, so a plain text-only Save edit (which recomputes tokenCount from text alone) can't leave a stale, quote-excluding count that undercounts context on later turns — recount from the quote-merged copy self-heals it. - Seed normalized quotes into the preliminary userMessage metadata (getPreliminaryUserMessage), so an abort during init/tool-loading (before onStart) still reconstructs the stopped turn's MessageQuotes. * ✅ fix: Add getReferencedQuotes to controller test mocks (CI) request.js's getPreliminaryUserMessage now calls getReferencedQuotes; the agents controller specs mock @librechat/api wholesale, so the mock must export it or the call throws and cascades. Added a faithful mock (normalize/cap, null when empty) to request.resumeMetadata.spec.js and jobReplacement.spec.js. * 📐 fix: Quotes in context projection + resumable metadata (Codex round 6) - Context-usage projection (resolveContextProjection): select message.quotes, prepend them into the projected user text, and recount quoted messages so the context gauge counts the same prompt the model receives (a text-only Save edit no longer makes the gauge undercount / over-report remaining budget). - Resumable job metadata: trackUserMessage (created-event rewrite) and abortJob (final requestMessage) now carry quotes; SerializableJobData.userMessage and CreatedEvent.message gained an optional quotes field. With the cross-replica created-event spread, stopping/reconnecting a quoted turn after the created event keeps its MessageQuotes. * 💬 feat: Collapse multi-select quotes into one chip with hover popup Composer feedback: the quote chip area now shows a single chip — the excerpt text for one selection, or a collapsed "{n} selections" pill for multiple, with a hover popup (HoverCard) listing every excerpt and a per-item remove. The chip is taller (py-1.5/text-sm) to read less skinny. Adds com_ui_quote_selections and com_ui_remove_all_quotes; updates unit + e2e tests (e2e drives the count via a data-quote-count hook and exercises the hover popup). * ♿ fix: Make multi-selection quote popup keyboard accessible The collapsed "{n} selections" pill used a HoverCard, which Radix only opens on pointer hover — its interactive content was unreachable by keyboard. Replaced it with a Popover: the trigger is a real button that opens on click / Enter / Space (focus moves into the list, each excerpt's × is tab-navigable, Escape closes and restores focus), with hover-open preserved for mouse via controlled open state + a close grace period. Hover-initiated opens skip auto-focus so they don't pull focus off the composer. Adds an e2e asserting keyboard open/close. * 📐 fix: Clamp the Add-to-chat button within the viewport (Codex round 7) The floating selection button positioned via translate(-50%,-100%) (bottom-center anchor) but clamped top/left as if they were its top-left, so a selection near the viewport top or sides could render the button partly/fully offscreen. Now it measures the button (ref + useLayoutEffect) and computes an on-screen top-left — clamping by the full width within side margins and flipping below the selection when there's no room above — with no transform, and stays hidden until measured so it never flashes at an unclamped spot. * ↩️ fix: Restore pending quotes on early-abort draft (Codex round 8) When a turn is stopped before the created event (e.g. during tool/MCP init), the final handler restores requestMessage.text to the draft, but the pending-quote atom was already drained on submit — so a retry sent no quotes. The abort requestMessage now carries quotes (preliminary metadata + abort fixes), so the three early-abort/no-response draft-restore paths in useEventHandlers now also re-queue pendingQuotesByConvoId from requestMessage.quotes. * ♿ fix: Use Ariakit Popover for quote selections (keyboard focus) The multi-selection popup used a hand-rolled Radix Popover with Popover.Anchor + a manual button, so Radix had no trigger to return focus to — Escape dumped focus to the page top. Refactored to Ariakit (the codebase's popover primitive, per DropdownPopup/Fork): the `PopoverDisclosure` is the real trigger, so Escape closes and returns focus to the composer instead of the top of the page. Keyboard opens (Enter/Space) autofocus into the list and tab through each excerpt's remove; hover opens for mouse with autofocus suppressed so it never pulls focus off the composer. e2e asserts the keyboard open/navigate/Escape flow keeps focus on a real control (never BODY).	2026-06-21 08:33:11 -04:00
Airam Hernández Hernández	3926fda234	🎒 fix: Apply OCR Context to Responses API Agents and Handoffs (#13707)	2026-06-20 10:17:09 -04:00
Ravi Kumar L	27b0782201	📛 feat: Tag Langfuse Traces With Tenant ID (#13808) ... * feat: tag Langfuse traces with tenant id * fix: propagate tenant id to agent Langfuse config	2026-06-17 20:27:55 -04:00
Danny Avila	49f4b659f6	🔐 fix: Honor Admin-Panel MCP Allowlist Overrides Without Restart (#13814) ... * 🔐 fix: Honor Admin-Panel MCP Allowlist Overrides Without Restart MCPServersRegistry was built once at boot from getAppConfig({ baseOnly: true }), freezing allowedDomains/allowedAddresses to YAML. Admin-panel mcpSettings overrides were ignored by both inspection (addServer/ reinspectServer/updateServer/lazyInitConfigServer) and runtime connection enforcement (assertResolvedRuntimeConfigAllowed), so a domain allowed only via the panel failed inspection and never connected. Make the registry's effective allowlists mutable and refresh them from the merged admin-panel config: seed at boot, and re-apply on every config mutation via invalidateConfigCaches -> clearMcpConfigCache. Both inspection and connection paths read the same getters, so both honor overrides without a restart. Fail-safe: current allowlists are preserved when the merged read fails. * 🛡️ fix: Scope MCP allowlist refresh to global config, fail-safe on DB error Address Codex P1 review findings on the allowlist-refresh path: - Tenant-scoped config mutations no longer push one tenant's merged mcpSettings into the process-wide registry singleton (read by all MCP connection paths), which would leak allowlists across tenants. Only global (non-tenant) mutations refresh the registry; tenant mutations still evict the config-server cache. - The refresh read now uses strictOverrides:true so a transient DB error throws instead of silently returning YAML base config — preserving the last-known allowlists rather than overwriting them with fallback values. Adds the strictOverrides option to getAppConfig (default off, no behavior change for existing callers). * ♻️ refactor: Resolve MCP allowlists per-request (tenant-scoped) instead of a global singleton Supersedes the prior global-mutation approach. MCP allowlists live in mcpSettings, which is tenant/principal-scoped admin config, so a process-wide singleton value is the wrong model — it caused cross-tenant bleed and stale reads. Instead, inject a resolver (from the app layer, where the merged config lives) that the registry calls per inspection and per connection. It reads the ALS tenant context via getAppConfig and accepts the acting user so user/role-scoped overrides resolve; config-source inspection (no user) resolves at tenant scope. Falls back to the YAML base allowlists when no resolver is set or the lookup fails, so a transient error fails to the operator baseline rather than disabling the allowlist. Removes the now-unnecessary setAllowlists / boot-seed / invalidateConfigCaches refresh / getAppConfig.strictOverrides machinery. * 🔒 fix: Scope config-source cache by allowlist; resolve OAuth allowlists per-request Address Codex review of the per-request resolver: - Config-source cache key now folds in the resolved allowlists, not just the raw-config hash. Inspection results became allowlist-dependent, so without this a tenant whose allowlist rejects a URL could poison the shared key with an inspectionFailed stub for a tenant that allows it (and vice versa). The tenant-scoped allowlist is resolved once per ensureConfigServers pass and threaded through the cache key + inspection. - The two remaining request-time OAuth allowlist reads now use the merged config instead of the YAML base getters: the fallback OAuth-initiate path (routes/mcp.js) via resolveAllowlists, and OAuth revocation (UserController.maybeUninstallOAuthMCP) via the request's already-merged appConfig.mcpSettings. Without this, an OAuth endpoint allowed only by an admin-panel override was rejected while inspection/connection allowed it. * ✅ test: Update MCP OAuth registry/config mocks for per-request allowlists CI fix for the Finding-12 change. The OAuth-initiate route now calls registry.resolveAllowlists() and the revocation path reads the merged appConfig.mcpSettings, so the affected specs' mocks were asserting the old base-getter values: - routes/__tests__/mcp.spec.js: add resolveAllowlists to the registry mock. - UserController.mcpOAuth.spec.js: provide mcpSettings on the getAppConfig mock so revokeOAuthToken still receives the expected allowlists. * 🧪 test: e2e proof that admin-panel MCP allowlist override takes effect Adds a Playwright mock-harness spec for #13809. A URL-based MCP fixture (e2e-http, streamable-http SDK server) boots inspectionFailed because its origin is omitted from the YAML mcpSettings.allowedDomains; the spec adds that origin via an admin config override (PUT /api/admin/config/user/:id) and asserts the server reinitializes — exercising the real resolver path through the backend + DB. Before the fix, reinspection used the frozen YAML allowlist and the server stayed unreachable. - e2e/setup/fake-mcp-http-server.js: streamable-HTTP MCP fixture (health GET /). - e2e/playwright.config.mock.ts: boot the fixture as a second webServer. - e2e/config/librechat.e2e.yaml: mcpSettings.allowedDomains (excludes 127.0.0.1) + the e2e-http server. - e2e/specs/mock/mcp-allowlist-override.spec.ts: login → baseline reinit fails → apply override → reinit succeeds.	2026-06-17 20:14:53 -04:00
Danny Avila	fdc7e64bb7	🪙 feat: SDK-Aligned Context-Usage Projection (gauge for window-switch & snapshot-less branches) (#13801) ... * 🪙 feat: Context-usage projection — data-provider + client wiring Consumer side of the SDK-aligned context projection (agents `projectAgentContextUsage`). Adds the `/api/endpoints/context-projection` data-provider plumbing (endpoint, service, query key, `TContextProjectionRequest`) and a `useContextProjectionQuery` gated to fire only when no fresh snapshot covers the viewed branch. Wires `useTokenUsage` precedence to: live snapshot → fresh persisted snapshot (window matches the resolved one) → server projection → per-message estimate. A model/window switch marks the baked snapshot stale (its `maxContextTokens` no longer matches) and falls to the projection — closing the gauge's window-switch (G1) and snapshot-less-branch (G2) gaps. Snapshot and projection share the render-relevant fields, so they render uniformly. Backend endpoint + agents version bump land in follow-up commits. Includes the design spec (CONTEXT_PROJECTION_SPEC.md). * 🪙 feat: Context-projection backend endpoint POST /api/endpoints/context-projection → resolveContextProjection (packages/api): reconstructs the viewed branch (parent-chain walk from messageId), resolves the agent config (instructions/provider/model/maxContextTokens), reuses LibreChat's stored per-message tokenCounts as the index map (no re-tokenizing), and calls the agents SDK projectAgentContextUsage — no model call. Thin controller injects db.getMessages/db.getAgent; route mirrors /token-config. First cut targets message-windowing accuracy; tool-schema tokens are deferred to a follow-up that reuses the full initializeAgent path. * 🩹 fix: Codex review on context projection (G1 guard, IDOR, recount, summary) - Guard `currentActive` against a stale window: a model/window switch on the current branch left the live snapshot outranking the projection (G1 didn't fire). Now defers to the projection unless streaming or the window matches. - Scope branch lookups to the authenticated user (`getMessages` filter + injected `userId`) — was loading any conversation by id (IDOR). - Recount messages with no stored `tokenCount` via the tokenizer instead of charging 0, so snapshot-less/imported histories don't under-report. - Fall back (null) for already-summarized branches rather than projecting from the full raw parent chain (the next call would send summary + tail); the client's summary-baseline-aware estimate handles them until a follow-up replays the summary boundary. * 🩹 fix: Codex round 2 — drop agent load, summary marker, edit-invalidation - Stop loading agent/model-spec config server-side (closes the agent-access IDOR and the spec-prompt special-casing). Provider/model/window now come from the client-resolved request (`limits.endpoint`/model — the agent's real provider, not the `agents` endpoint, so the tokenizer is right). Agent/spec/ promptPrefix instructions are uniformly deferred to the full-fidelity follow-up. - Detect summarized branches via the live path's `metadata.summaryUsedTokens` marker (was the wrong `summaryTokenCount` field) and fall back to the summary-aware estimate. - Invalidate the projection query on in-place message edits via a branch content `revision` in the cache key (the tail id is unchanged on edit). Deferred (valid, not a regression): same-window endpoint/model switch keeps a window-matched snapshot — needs endpoint/model persisted on the snapshot, which lands with the fidelity follow-up. Smoke-tested: fits / prunes / summarized→null / no-window→null. * 🛡️ fix: make context projection strictly additive (no-regression) Revert the G1 window-match guard on the live/branch snapshot. When no explicit maxContextTokens is set (the common default), the SDK's snapshot window is reserve-derived (~0.9·(modelContext − maxOutputTokens)) while useTokenLimits resolves the raw model context — so `snapshot.maxContextTokens === resolvedMax` is false for the SAME model, and the guard would wrongly drop a valid current-branch snapshot to projection/estimate post-stream (a regression in the default case, per initialize.ts:1240-1243). The projection now activates ONLY for snapshot-less branches (G2): the precedence is live snapshot → persisted branch snapshot → projection → estimate, where the first two are byte-for-byte the prior behavior and the projection just slots ahead of the estimate. Window/model-switch (G1) detection needs the snapshot to carry its model/window and defers to the fidelity follow-up. * 🩹 fix: surface projections as estimates, not authoritative snapshots A first-cut projection carries the SDK's windowing but omits instruction/tool overhead, so rendering it as `isEstimate: false` showed a confident under-count for snapshot-less branches. Mark projection-sourced views `isEstimate: true` + `snapshotActive: false` (and drop the snapshot field) so they present as a better estimate than sumBranch — improved used/window number, estimate framing, no misleading granular breakdown with ~0 tools. Real snapshots stay authoritative. (Codex round 3, projection.ts:139.) * 🧹 chore: drop CONTEXT_PROJECTION_SPEC.md from the PR * 🎨 style: fix import-sort order in projection.ts (CI sort-imports check) * 🔧 chore: update @librechat/agents dependency to version 3.2.36 in package-lock.json and related package.json files * chore: npm audit fix * 🎨 style: fix import-sort order in data-service.ts (CI sort-imports check) * 🩹 fix: drop dead calibrationRatio in projectionParams (tsc never error) Inside the ternary, branchSnapshot is narrowed to null (the gate is ), so accessed a property on (frontend typecheck failure). It was also dead — there is never a snapshot to seed from in this branch — so just remove it. * Revert "chore: npm audit fix" This reverts commit 4cdb862d0c.	2026-06-16 17:54:13 -04:00
Dustin Healy	054fa4bfa7	🥽 fix: Restrict MCP Server URL Disclosure to Admins, Owners, and Editors (#13784) ... * 🥽 fix: Redact Non-User-Sourced MCP Server URLs by ACL Edit Permission GET /api/mcp/servers and GET /api/mcp/servers/:serverName return MCP server configs to any caller with MCP-use permission. For user-sourced configs (DB-stored, UI-submitted), the URL is the caller's own and is intentionally disclosed. For non-user-sourced configs (YAML or config-tier, operator-defined), the URL and OAuth flow endpoints (authorization_url, token_url) are operator-sensitive: they can encode internal infrastructure hostnames and are not editable through the API. This change redacts those fields on non-user-sourced configs unless the caller has edit authority on the resource, using the same ACL check (PermissionBits.EDIT) that the PATCH and DELETE routes already enforce via canAccessMCPServerResource. Callers with broad MANAGE_MCP_SERVERS capability bypass the per-resource check, matching the existing capability bypass in canAccessResource. customUserVars is intentionally not redacted: its values are UI hint metadata (title, description, sensitive), not user-supplied secrets; blanking it would give non-editor callers a Configure form with no field labels. * 🥽 fix: Correct getResourcePermissionsMap import path + tighten redact comments The MCP server redaction commit imported getResourcePermissionsMap from ~/server/controllers/PermissionsController, but that controller is a consumer of the helper, not its exporter. The canonical export lives in ~/server/services/PermissionService (which controllers/agents/v1.js already imports from). Fixes the runtime getResourcePermissionsMap is not a function failure on GET /api/mcp/servers and the four downstream route-spec failures whose config mocks lacked a source field and were therefore wrongly treated as non-user-sourced; mocks now reflect the real registry behavior (addServer/updateServer tag DB-stored configs with source: 'user'). Trims narrating JSDoc on the redact helpers and resorts the librechat-data-provider destructure by length. * chore: import order * 🥽 fix: Redact OAuth Revocation Endpoint Alongside Authorization And Token URLs The OAuth-URL strip path only dropped authorization_url and token_url. The UserOAuthOptionsSchema in packages/data-provider/src/mcp.ts (line 146) accepts revocation_endpoint as another operator-configurable URL, and the OAuth handler uses it to revoke tokens; it can hold the same internal IdP hostnames the existing strip is trying to hide. Adds revocation_endpoint to the destructure so a non-user-sourced YAML/config MCP server config no longer leaks the revocation URL to non-editor callers. The existing strip url and oauth flow URLs spec is extended with a revocation_endpoint value to lock in the new field. * 🥽 fix: Gate Shared DB Server URL Disclosure On ACL Edit Permission source-driven URL disclosure was incorrect for shared DB-backed MCP servers. ServerConfigsDB.mapDBServerToParsedConfig (packages/api/src/mcp/registry/db/ServerConfigsDB.ts:465) sets source: 'user' on every DB-stored config it returns, regardless of who is accessing it. A user with only VIEW share on a DB server, or with agent-mediated access, was therefore treated by the redaction layer as if they owned the URL, and GET /api/mcp/servers disclosed the owner's URL and OAuth flow URLs to viewers who could not edit the resource. The redaction is now driven purely by ACL edit authority: computeCanEditByServer routes every dbId-bearing config through PermissionBits.EDIT regardless of source; redactServerSecrets strips on !canEdit regardless of source. POST and PATCH controllers explicitly pass canEdit: true since both endpoints establish edit authority (POST creates the resource, PATCH is gated on the EDIT middleware). Legacy/ephemeral configs without a dbId still fall back to the source heuristic. * 📝 docs: correct redactServerSecrets URL-disclosure comment --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-06-16 11:20:52 -04:00
Danny Avila	d18d62e7c1	🪙 refactor: Reconcile Context Gauge to Actual Provider Tokens (#13780) ... * 🪙 fix: Reconcile Context Gauge to Actual Provider Tokens The context gauge could read several× too high (e.g. 213K when the real prompt was 56K) and stay there across reloads. Root cause: the SDK's calibrationRatio is `cumulativeProviderReported / cumulativeRawSent`, but a provider's server-side web search injects large fetched content into the prompt that the SDK never sent or counted — pinning the ratio at its cap (5) and multiplying every later message estimate, including post-summary ones. The gauge rendered (and persisted) that inflated estimate, never the provider's actual token count. Fix: reconcile the snapshot to the call's ACTUAL prompt tokens (input + cache), which already arrive in on_token_usage. Only messageTokens is calibration-scaled (instructions/summary are raw tiktoken), so keep those and set messageTokens to the remainder, recomputing free space. Shared `promptTokensFromUsage` + `reconcileContextUsage` in data-provider; applied server-side in buildPersistedContextUsage (reload-stable) and client-side in useUsageHandler on each primary usage (corrects at turn-end, no follow-up needed). Also drop the summary double-count from the Breakdown Messages row. Deferred (separate agents PR): the SDK over-calibration also fires summarization prematurely; fixing it needs decoupling real-content estimation from server-side injection headroom without weakening pruning-overflow safety. * 🪙 fix: Harden Token Reconciliation for Provider-less + Resume Paths Codex review on the reconciliation: - promptTokensFromUsage: when the provider is absent (custom/OpenAI-compatible payloads), fall back to the same magnitude heuristic normalizeUsageUnits uses (cache ≤ input ⇒ already included) so cached events aren't re-inflated. - Resume: backfillUsage restores a primary call's usage without replaying a live on_token_usage (Redis mode), so the live reconcile never ran and a reconnected session stayed on the inflated estimate. New reconcileBackfill reconciles the restored snapshot from the final primary call after contextHandler installs it. * 🪙 fix: Reconcile Resume Snapshot Server-Side, Not via Backfill Codex: the client reconcileBackfill scanned the resumed run's collectedUsage and applied the final primary to the latest snapshot — but on a mid-call resume that usage belongs to an EARLIER call, corrupting the restored gauge. Move the resume reconciliation server-side: GenerationJobManager.persistTokenUsage reconciles the stored contextUsage to a primary usage's actual prompt tokens as it arrives. That usage is the post-invoke truth for the call the latest stored snapshot precedes (no snapshot is captured between a call's pre-invoke dispatch and its usage), so it's correct by construction and run-matched. A mid-call resume (no usage yet) keeps the raw snapshot instead of mis-applying an earlier call's tokens; it reconciles once the call completes. Removed client reconcileBackfill; the live-path reconcile (non-resume) stays. * 🪙 fix: Guard Reconciliation Against Replays and Snapshot Races Two Codex concurrency findings on the reconciliation: - Client: reconcile only on a NEWLY folded primary usage. A replayed duplicate (folded=false on resume) can be an earlier tool-loop call sharing the run id, which would overwrite the latest snapshot with an earlier, smaller prompt. Moved the reconcile after the folded guard. - Server: serialize the context-usage write through the same per-stream queue as the token-usage write. persistTokenUsage reconciles the stored snapshot (read-modify-write); an unserialized trackContextUsage could store a newer snapshot between the read and write — or a stale reconciled write could land after a newer snapshot — clobbering the newer run's gauge when calls interleave. FIFO keeps each call's snapshot ahead of its own usage and behind the next. * chore: import order in GenerationJobManager.ts	2026-06-16 11:05:44 -04:00
Danny Avila	055585f9f1	🪢 fix: Tie MCP Cleanup To Resumable Runs (#13769) ... * fix: Clean up request-scoped MCP connections * test: Format MCP request context spec * refactor: Move MCP request context to API package	2026-06-15 15:26:03 -04:00
Danny Avila	0537930144	🗂️ fix: Scope Token Config Cache (#13770) ... * fix token config tenant cache scope * fix token config scoped cache backfill * chore sort token config imports	2026-06-15 15:25:19 -04:00
Danny Avila	44c253d48a	🪙 fix: Correct Context Usage Gauge After Summarization (#13744) ... * 🪙 fix: Persist Context Snapshot + Summary Marker After Summarization The post-summarization context is correctly compacted by the SDK, but the breakdown wasn't reliably reaching the client, leaving the gauge on the whole-history estimate (stuck at 100% forever once a conversation compacts). Two server changes in buildResponseMetadata: - Snapshot guard: persist the breakdown when a PRIMARY usage event follows the latest snapshot (tracked via contextUsageSink.latestUsageIndex, recorded in the on_context_usage handler) instead of a brittle snapshot-vs-primary count. A summarization detour adds an extra snapshot whose only following usage is tagged 'summarization', which the count guard could miscount and drop. - Summary marker: whenever a turn compacts (summaryTokens > 0), persist a lightweight metadata.summaryUsedTokens (the pre-invoke compacted context size) UNCONDITIONALLY — so even when the full snapshot can't be saved (interrupted final call) or never reaches the client, the per-message estimate has a signal to cap the discarded history. Tests: client.contextMetadata.spec (guard + marker, incl. marker-survives-drop) and a real-pipeline summarization integration test. * 🪙 fix: Cap the Context Estimate at the Summary Marker When the gauge falls back to the per-message estimate (no usable snapshot on the branch), sumBranch summed the ENTIRE branch history — after a summarization that discarded most of it, this over-counts and pins the gauge at 100% in perpetuity. sumBranch now stops at the deepest summarized response (metadata.summaryUsedTokens) and records it as summaryBaseline; the walk counts only post-summary messages, and useTokenUsage adds the baseline. So the estimate reflects the compacted context (summary + recent turns), not the discarded history. USD/default behavior unchanged when no marker is present. Test: sumBranch caps a huge pre-summary history at the compacted baseline. * 🪙 fix: Address Codex Review on the Summarization Marker - Branch cost/usage is no longer truncated at the summary marker — sumBranch caps only the CONTEXT-window count there and keeps accumulating provider usage/cost to the root (cumulative spend isn't discarded by compaction). - findBranchSnapshotAnchor stops at a summarized response with no snapshot of its own, so it can't recover a stale PRE-summary snapshot and show discarded history; the summary-baseline estimate is used instead. - Abort path: buildAbortedResponseMetadata now persists the summaryUsedTokens marker (pre-invoke, no completedOutputTokens ambiguity, so safe on abort) so a STOPPED summarized turn isn't re-summed on reload. - Marker baseline fallback now includes summaryTokens (a separate breakdown field) so it doesn't under-report the compacted size. DRY'd into a shared computeSummaryUsedTokens used by the completion and abort paths. - Estimate popover surfaces the summary baseline as a row so the displayed rows reconcile with the header total. Tests: sumBranch cost-not-truncated + anchor-stops-at-marker (client); computeSummaryUsedTokens fallback + abort marker (packages/api). * 🪙 fix: Attribute Persisted Context Usage to the Snapshot Run Match the post-snapshot primary usage to the latest snapshot's runId before persisting metadata.contextUsage. Parallel/direct runs interleave snapshots and usage (A snapshot → B snapshot → A usage → B no-usage); the prior index-only guard persisted B's snapshot with A's output. finalCallOutputTokens now filters completedOutputTokens to the snapshot's run. Untagged events (older lib/resume) match any run for back-compat. * 🪙 fix: Harden Summary Marker Against Tool-Loops, Stale Anchors, and Emit Races Codex round on the summarization marker: - Avoid double-counting earlier tool-loop outputs in the summary marker: those outputs sit in BOTH the latest snapshot's pre-invoke baseline AND the response message's tokenCount the client estimate adds on top. computeSummaryUsedTokens now subtracts the run's prior primary outputs (priorRunOutputTokens) — the live path bounds them by the snapshot's usage index, the abort path by all primaries (an interrupted final call emits none). Single-call turns subtract 0. - Stop treating pre-summary anchors as active: sumBranch no longer sets containsAnchor once the context is capped at a summary marker, so a stale pre-summary snapshot can't override the summary-baseline estimate. - Capture latestUsageIndex BEFORE awaiting emitEvent: a yield (resumable SSE / Redis) during parallel runs could let this call's own usage advance the index past the event that proves the snapshot completed, dropping a valid breakdown. * 🪙 fix: Subtract Summarization Output from the Summary Marker recordCollectedUsage folds the summarization call's completion into the response message's tokenCount, while the generated summary is also in the snapshot baseline as summaryTokens. The client estimate (summaryBaseline + responseTokenCount) thus counted the summary twice — inflating the gauge after compaction even on a single-call turn whenever the full snapshot is unavailable. priorRunOutputTokens now also counts summarization-tagged output (still excluding subagent/sequential, which recordCollectedUsage keeps out of the reported total), so the marker subtracts it. Updated unit + guard tests. * 🪙 fix: Refine Marker Subtraction for Summarization RunId and Abort Boundary Two Codex follow-ups on the marker-subtraction logic: - Subtract summarization output regardless of runId: the summarize detour is its own model-end call that may carry a distinct runId, but its output still lands in this response's tokenCount AND the snapshot baseline (summaryTokens). It is now counted unconditionally (still within the response's own usageEmitSink), while primaries keep the parallel-run runId filter. - Don't subtract primaries on the abort path: the job stores no snapshot/usage boundary, so a primary that completed AFTER the latest snapshot is NOT in the baseline; subtracting it would cancel real output and under-report. priorRun- OutputTokens gains an includePrimary flag (false for abort) — abort subtracts only the always-pre-snapshot summarization output. * 🪙 fix: Run-Scope Summary Subtraction and Stop Subtracting on Abort Two Codex follow-ups, resolved by reverting the round-4 detour: - Run-scope the summarization subtraction: the summarize detour inherits the graph run id (traceConfig spreads config.metadata.run_id), so its usage shares the answer snapshot's runId — it is NOT a distinct run. priorRunOutputTokens now filters summarization by runId like primaries, so a parallel sibling run's summary (different runId, in the sibling's baseline) is no longer subtracted from this branch's marker. Drops the includePrimary flag added last round. - Stop subtracting on the abort path: abort tokenCount is countTokens(text) (abortMiddleware) or absent (agents route) — it does not fold in summarization or earlier-call output the way recordCollectedUsage does, so the marker must keep the full baseline. buildAbortedResponseMetadata now subtracts nothing.	2026-06-14 18:23:30 -04:00
Danny Avila	2350ebb24a	📨 feat: Custom Headers on Built-in Provider Endpoints (#13742) ... * 📨 feat: Custom Headers on Built-in Provider Endpoints Add a `headers` config option to the built-in `openAI`, `anthropic`, and `google` endpoints (incl. Anthropic/Google Vertex), mirroring the custom endpoint header mechanism. Values support the same placeholder resolution (env vars, `{{LIBRECHAT_USER_*}}`, `{{LIBRECHAT_BODY_CONVERSATIONID}}`) and are resolved at request time so dynamic values like conversationId resolve against the live request — without losing provider-native request shaping. Closes #13082. Covers #13713: forwarding conversationId to a reverse proxy is now `X-Conversation-Id: '{{LIBRECHAT_BODY_CONVERSATIONID}}'` — an unknown header is ignored by the native Anthropic API, so no 400 and no metadata gating needed. - Schema: `headers` on `baseEndpointSchema` (openAI/google/anthropic/all). - New `mergeHeaders`/`resolveConfigHeaders` utils centralize the per-provider header locations (`configuration.defaultHeaders`, Anthropic `clientOptions.defaultHeaders`, Google `customHeaders`); provider-managed headers (auth, `anthropic-beta`) always win on collision. - Each initializer threads configured headers (endpoint over `all`) into the right place; request-time resolution runs across all locations in the main and title flows. * 🩹 fix: Cast endpoints.all to TEndpoint for headers DeepPartial widening Adding `headers` (a Record) to `baseEndpointSchema` makes `DeepPartial<TCustomConfig>` widen its value type to `string | undefined`, which is not assignable to the concrete `TEndpoint['headers']: Record<string, string>` at the `loadedEndpoints.all` assignment. Cast at the assignment site, mirroring the existing `anthropicConfig as TAnthropicEndpoint` cast in the same function. * 🛡️ fix: Harden built-in endpoint custom headers (Codex review) Address Codex P2 findings on the custom-headers feature: - Anthropic title requests: `omitTitleOptions` strips the `clientOptions` carrier, which dropped its `defaultHeaders`. Preserve just the header carrier so gateway/reverse-proxy metadata still reaches title generation. - mergeHeaders: match header names case-insensitively so an override (e.g. a provider-managed `Authorization`/`anthropic-beta`) replaces/uniones a case-variant from the base instead of emitting two names a client may collapse. - OpenAI: withhold admin-configured headers when the user supplies the base URL (`user_provided`), since values may carry `${SECRET}`/token placeholders that must not reach a user-controlled endpoint — mirrors the custom-endpoint guard. - Azure: honor global `endpoints.all` headers (same OpenAI carrier) while keeping Azure-managed `api-key`/version headers authoritative. Adds tests for each. * 🔐 fix: Resolve-once + provider-managed header safety (Codex review round 2) Address Codex P2 findings: - Azure: keep global `endpoints.all` headers unresolved at init and let request-time `resolveConfigHeaders` resolve them once, avoiding a second-order env expansion of already-substituted user values. - Google: `resolveConfigHeaders` no longer template-resolves the provider-managed `Authorization` header (built from a possibly user-provided key), so a user key like `${ENV}` can't leak server environment values. - Model fetches: thread configured headers (endpoint over `all`) + user object through `getOpenAIModels`/`getAnthropicModels` → `fetchModels`, so a gateway-fronted built-in provider receives the header on `/models` too. Fixed `fetchModels` to merge custom headers for Anthropic instead of overwriting them (managed `x-api-key`/version still win). Adds/updates tests for each. * 🧯 fix: Header provenance, memory/title coverage, idempotency (Codex round 3) Address Codex P2 findings, including two regressions from the prior round: - Google auth (findings 6 & 8): move native Google header resolution to init (`initializeGoogle`), resolving admin templates BEFORE the key-derived auth header is built. resolveConfigHeaders no longer touches Google `customHeaders`, so admin `Authorization` templates resolve again (fixes the round-2 regression) while the SDK auth header (possibly a user-provided key) is never env-expanded. - Memory runs: memory extraction now calls `resolveConfigHeaders`, so native Anthropic (and OpenAI) headers resolve for memory requests too. - Vertex titles: restore the ORIGINAL `clientOptions` object reference (not a copy) when preserving headers across `omitTitleOptions`, so the Vertex `createClient` closure and the resolved headers stay on the same object. - Reuse: `resolveConfigHeaders` is now idempotent (resolve-once per header map), preventing a second pass from env-expanding values already substituted with user/body data when an agent object flows through buildAgentInput twice. Adds/updates tests for each.	2026-06-14 17:02:04 -04:00
Danny Avila	4ee68d5240	💸 feat: Per-Agent Endpoint Token Config in Multi-Endpoint Billing (#13738) ... * 💸 feat: Per-Agent Endpoint Token Config in Multi-Endpoint Billing Price each collected/emitted usage item with the producing agent's resolved endpoint token config, instead of the primary agent's for the whole graph. Previously AgentClient.recordCollectedUsage and the subagent usage emitter used a single this.options.endpointTokenConfig (the primary's) for every usage item. A connected agent or subagent on a different custom endpoint that shares a model id with an entry in the primary's tokenConfig was therefore mis-priced (a model absent from it already fell back to the built-in rate map — no regression). - Tag each usage with its producing agent: ModelEndHandler stamps usage.agentId = agentContext.agentId; createSubagentUsageSink stamps the child's subagentAgentId (UsageMetadata gains an optional agentId). - buildAgentToolContext retains endpointTokenConfig so initialize.js can build an agentId -> endpointTokenConfig map from agentToolContexts (the one map that holds every agent, including pure subagents pruned from agentConfigs). - AgentClient.resolveAgentEndpointTokenConfig(usage) looks up that map by agentId, falling back to the primary config; used by both the billing path (new optional resolveEndpointTokenConfig on recordCollectedUsage) and the subagent cost emitter. - recordCollectedUsage's resolver is optional and falls back to the batch endpointTokenConfig, so the shared responses.js/openai.js call sites are unchanged. - Tests: two-endpoint graph with a colliding model id prices per-agent; resolver nullish falls back to batch; subagent sink tags the child agent id. * fix: Align emit-path cost with per-agent billing; honor known-agent built-in pricing Addresses Codex review on the per-agent endpoint token config: - Emit path (callbacks.js) now prices each on_token_usage event with the producing agent's config (resolved via usageCost.resolveEndpointTokenConfig), so streamed/persisted metadata.usage.cost matches the per-agent balance transaction. The agentId tag is resolved server-side and stripped from the emitted/persisted payload. - Resolver (resolveAgentTokenConfig) now treats a known agent's config as authoritative, including undefined → built-in pricing, so a known non-custom agent in a custom-primary graph is no longer charged the primary's rates. Only untagged/unknown usage falls back to the primary config. - endpointTokenConfigByAgentId records every known agent (value may be undefined) so the resolver distinguishes known-no-rates from unknown.	2026-06-14 12:00:32 -04:00
Danny Avila	b03b2a0a29	💾 feat: Persist Context Breakdown & Branch/Total Usage Cost (#13734) ... * 💾 feat: Persist Context Breakdown & Branch/Total Usage Cost Persist the granular context breakdown and per-response usage/cost on the response message metadata, and re-derive branch + total usage/cost from a per-message index so the popover survives reloads and is branch-aware live. - Add aggregateEmittedUsage + buildPersistedContextUsage helpers in packages/api; capture the latest visible snapshot and every emitted on_token_usage payload via contextUsageSink/usageEmitSink. - Attach metadata.contextUsage (Part A) and metadata.usage (Part B) on the agents response message in sendCompletion. - Carry per-message usage on the token index; add sumTotalUsage/setEntryUsage and branch-scoped usage on sumBranch. - Repurpose the session accumulator into a single in-flight pending holder; flush it into the index at finalize; hydrate breakdowns on load. - Render branch cost with a conditional all-branches total in the breakdown. * 🧹 chore: Remove orphaned com_ui_session_cost i18n key * 🩹 fix: Address Codex review — normalize usage server-side, fix reload deltas - Persist per-event-normalized display units in metadata.usage (TResponseUsage) so reloaded mixed-provider turns match the live session; client reads them directly instead of re-normalizing with a single stamped provider (P2). - Persist completedOutputTokens (final call output) on metadata.contextUsage so a reloaded multi-call turn adds the post-snapshot delta, not the full tokenCount the snapshot already counts (P2). - buildIndex preserves a prior entry's immutable usage when a rebuilt cache message lacks metadata.usage, so a mid-session rebuild (regenerate) keeps a sibling branch's flushed cost (fixes the e2e regenerate failure). - Track costKnown so turns saved with contextCost off don't render $0.00 when cost display is later enabled (P3). - Use an epsilon for the all-branches cost comparison to avoid a spurious total row from float summation order (P3). - Update unit/integration/e2e tests for the new shapes; regenerate e2e asserts the all-branches total after reload (deterministic via persisted metadata). * 🩹 fix: Address Codex round 2 — pending leak, cost coverage, reload delta - Clear the in-flight pending usage on terminal abort/error (resetLive), so a stopped generation's tokens no longer merge into the next response (P2). - costKnown now means COMPLETE coverage (ANDed): a branch mixing cost-bearing and cost-less turns is flagged incomplete and the cost row is hidden rather than rendering an under-reported total (P2). - Drop the tokenCount fallback for completedOutputTokens on reload: only the persisted post-snapshot delta is used, so a multi-call turn whose provider emitted no usage_metadata no longer double-counts earlier output (P2). - Update tokens.spec for AND coverage semantics + incomplete-cost case. * 🩹 fix: Address Codex round 3 — no-usage snapshots, total coverage, provider-less cache - Skip persisting metadata.contextUsage when the response emitted no primary usage event: without a known post-snapshot output the granular gauge would undercount the reply on reload, so fall back to the coarse per-message estimate instead (P2). - Gate the all-branches cost row on totalUsage.costKnown so an incomplete total (a sibling saved without cost) never renders an under-reported figure (P2). - aggregateEmittedUsage/finalCallOutputTokens now normalize per-event with the client's magnitude fallback (normalizeEventUnits) instead of billing splitUsage, so provider-less cached events match live on reload (P2). - Add backend test for the provider-less cached case. * 🩹 fix: Address Codex round 4 — abort attribution, complete cost coverage - aggregateEmittedUsage persists cost only when EVERY call was priced; a partial pricing failure now omits cost so the client treats coverage as unknown rather than reading an under-reported sum as authoritative (P2). - finalizeUsage flushes pending into the response entry only when events were folded this session (eventCount > 0), so a late/second resumable subscriber carrying persisted metadata.usage keeps it instead of being overwritten with an empty pending record (P2). - On user stop, attribute the in-flight pending usage to the partial response (new attributePending handler) instead of discarding it in resetLive — the stopped reply's billed tokens are kept and still can't leak into the next response; resetLive's discard remains for the error path (P2). * 🐛 fix: Persist branch cost across branch switches via sticky usage history Branch cost vanished on switching to a sibling branch (until a new turn) — the cost analog of the granularity bug. buildIndex rebuilds the token index from the messages cache; a sibling generated this session whose cache message lacks metadata.usage (and is transiently dropped from the cache during regenerate) lost its live-flushed usage, so sumBranch found none and the cost row hid. Fix: a sticky per-response usage map (conversationId → messageId → usage), written by setEntryUsage and never rebuilt from the cache — the usage counterpart of snapshotsByAnchorFamily for the breakdown. buildIndex/upsertEntries restore an entry's usage from it when the message carries none; cleared on convo switch and migrated with the index. Add unit coverage for the drop-then-readd regression and an e2e assertion that branch cost survives a branch switch. * 🐛 fix: Re-index on branch switch so branch cost survives the switch The sticky usage history alone didn't fix the reported branch-switch cost drop: on a branch switch no cache `updated` event fires, so the index subscriber never re-ran, and the post-regenerate rebuild was skipped while `isSubmitting` was still true — leaving the index stale and missing the now-viewed branch's response entirely (sticky can only restore entries present in a rebuild). Re-index from the messages cache on every tail change (created/finalize AND branch switch), not just while submitting. The cache holds the full message set at switch time, so the viewed branch's response is re-added and its usage restored from metadata.usage or the sticky history → sumBranch finds it and the branch cost renders. Verified locally: the branch-switch e2e now passes (the cost section shows both the branch row and the all-branches total). Also fixed that e2e assertion to target a single cost value (strict-mode safe). * 🩹 fix: Handle stopped-stream usage — reset pending + persist abort metadata Codex round (stop/abort edges): - Resumable explicit-stop (intentional SSE close) reset UI state but never cleared pendingUsageFamily, so usage folded before the stop leaked into the next response in the conversation. Discard pending on intentional close (resetLive); a resume re-folds via backfillUsage, so nothing is lost. - The abort save path (abortMiddleware) persisted the stopped response without metadata.usage/contextUsage, so its cost + breakdown vanished on reload. Rebuild both from the job's persisted tokenUsage (emitted payloads incl. cost) and contextUsage snapshot — parity with the normal sendCompletion path; breakdown gated on a primary usage event like buildResponseMetadata. Deferred (per scope decision): mid-stream branch-switch transiently shows the streaming branch's pending on the viewed sibling (cosmetic, until finalize). * 🩹 fix: Persist abort metadata on the real agents route + tighten snapshot gate Codex round (corrects last round's wrong-path fixes): - Stopped AGENTS responses are saved by routes/agents/index.js (/chat/abort), not abortMiddleware — so last round's metadata fix never ran for them. Moved the rollup/snapshot builder into packages/api as buildAbortedResponseMetadata (shared, unit-tested) and applied it in BOTH abort save paths, so a stopped agent reply keeps its cost + breakdown on reload. - Persist the breakdown only when the FINAL visible call emitted usage: track a per-response snapshot count and require primaryUsageCount >= snapshotCount. Previously any earlier primary usage event passed the gate, so a multi-call turn whose final call emitted no usage_metadata used an earlier call's output as completedOutputTokens (already counted by the latest snapshot) → reload over-reported. Now it falls back to the coarse estimate. Resumable stop pending-reset (prior round, 3cde6fe035) already flows through clearAllSubmissions → SSE close → the intentional-close handler's resetLive. Deferred per scope: mid-stream branch-switch pending attribution (tracked). * 🩹 fix: Abort breakdown over-count + resume re-fold after pending discard Codex round (on the re-applied abort/snapshot work): - buildAbortedResponseMetadata now persists ONLY the usage/cost rollup, not the context breakdown. The abort path can't tell whether the final call emitted usage (the job stores only the latest snapshot, not a count), so persisting the breakdown risked reusing an earlier call's output as completedOutputTokens (already in the snapshot) → reload over-count. Stopped/incomplete responses now fall back to the coarse gauge estimate, which is safe and apt. - resetLive now also forgets the conversation's folded usage-event identities (clearUsageFolded). Discarding pending on a terminal/intentional close left the folded keys set, so a later resume's backfillUsage saw the persisted events as duplicates and never rebuilt pending — leaving the response's usage missing until a full reload. Clearing them lets the resume re-fold.	2026-06-14 10:48:07 -04:00