927 commits

Author	SHA1	Message	Date
Danny Avila	104bbc8633	🔓 fix: Don't Lock Skills Out of Saving Over Pre-Existing Malformed Flags ... Three findings from the self-review's completeness pass. A skill whose STORED SKILL.md already carried a malformed flag became unsavable. `validateBodyDerivedColumns` ran on every body-carrying update, and the skill editor resubmits the whole file on every save, so a body containing `user-invocable: yes` rejected edits to unrelated fields — the user could not even fix the description. Those documents exist precisely because pre-fix import returned 201 for that value, and `yes`/`no`/`on`/`off` read as booleans in YAML 1.1, so it is an ordinary authoring shape. The stored-body scan is now taken before validation and a flag that was already malformed no longer blocks the save; a value this edit introduces is still rejected. The end-to-end spec never ran in CI: `test:ci` ignores `\.*integration\.`, and the named integration scripts are scoped to cache, s3 and agents, so the only test asserting the issue's reproduction against a real `createSkill` was skipped everywhere. Renamed to `import.db.spec.ts`, matching the several packages/api specs that already boot MongoMemoryServer in the default run. Parity between the two frontmatter readers was asserted only in a commit message. `parity.db.spec.ts` now feeds 23 shapes through both the upload path and the inline-body path and requires identical columns, with the two known asymmetries pinned as their own cases: tab indentation, which only the YAML parser rejects, and duplicate keys differing in case, which the file leaves ambiguous. Neither can release a restriction. Skill validation failures now carry `message`, as the import handler already did. The client falls back to a generic string when it is absent, so a rejected flag line was previously undiagnosable from the UI.	2026-08-04 09:52:39 -04:00
Danny Avila	1d139666ae	🧯 fix: Let Only the Body's Own Prior Declaration Release a Restriction ... Self-review found the previous guard was the wrong shape. Gating on "the new body has a frontmatter block" still released a bag-only restriction whenever the edited body happened to carry a block, and any key the body reader cannot see (a quoted `"user-invocable":`) looked like a removal too. Both are instances of one class: treating the reader's silence as a declaration. A body edit may now only remove a flag that the STORED body declared, read back under the same version guard that protects the write. An edit can therefore release what the author wrote into the file, a skill whose flags were never in the text keeps them, and a key the reader misses is invisible on both sides of the comparison — so its blind spots degrade to no-ops instead of silent releases. The structured-bag contract is untouched: a bag that omits a key still removes it, which stays the escape hatch for flags the body never had. Two reader divergences fixed with it, both confirmed against the real modules: the body scanner now unquotes keys, so `"user-invocable": false` is honored the way the importer already honored it; and an empty flag value is a placeholder rather than a malformed boolean even when the line scan finds nothing, which an indented mapping or a quoted key can cause. A corpus of 26 frontmatter shapes now runs through both the import path and the inline-body path with identical columns in 25 — the residual is duplicate keys differing only in case, where the file is ambiguous by construction (js-yaml takes the last, the line reader the first) and neither answer can release a restriction.	2026-08-04 09:23:33 -04:00
Danny Avila	286758449d	🩹 fix: Keep Skill Parser Mock-Safe and Read Continued Body Flags ... Three follow-ups on the invocation-mode work. CI: parse.ts built its key lookup at module scope from a `SKILL_BOOLEAN_FLAGS` value imported out of data-schemas. Suites that replace that module with a partial mock (agents/openai/service.spec.ts mocks it as `logger` alone) left the import undefined, so the map construction threw before any test ran and took six `api` suites plus one `@librechat/api` shard down with it. The table is declared locally again — this module is pure text parsing and must load without the DB package initialized — and parse.test.ts asserts it still matches data-schemas. Codex P1: a body-only edit unset the derived column but left the stored frontmatter bag's copy in place, so `backfillDerivedFromFrontmatter` read the restriction back on the next `getSkillByName` and the release undid itself. A body-driven update now clears the bag's flag keys, handing authority to the columns; the SKILL.md body still carries the declarations. Codex P2 / Copilot: the body scanner treated `user-invocable:` with its value on the following line as an unwritten placeholder, and skipping every indented line also blinded it to a frontmatter block indented as a whole. It now reads keys at the mapping's own indentation and follows a lone indented scalar as a continuation value, matching what the import/sync parser already accepted.	2026-08-04 08:39:17 -04:00
Danny Avila	1e1f751e92	🎛️ fix: Honor All Invocation-Mode Frontmatter Fields on Skill Import and Inline Edits ... `POST /api/skills/import` silently discarded `user-invocable` and `disable-model-invocation`, returning 201 with both columns at their schema defaults. Those columns derive only from the structured `frontmatter` bag, and import never passed one, so the flags had no channel to reach the document. `always-apply` survived because it also travels an explicit column param and a body-parse fallback. Import now passes a sanitized bag, and the body-level cascade that previously served only `always-apply` covers all three flags, so a flag declared inline is honored on `POST`/`PATCH /api/skills` too — the create/edit forms send `body` with no `frontmatter`, so that was the same defect on another endpoint, and without it an imported restriction could never be released from the UI. - parse.ts: one shared flag table drives parsing and the new `toCleanFrontmatter`, which rewrites each flag from its resolved value under the canonical key - data-schemas: `checkFrontmatterEntry` shared by `validateSkillFrontmatter` and the new `pickValidFrontmatter`; body scanner generalized to all three flags - sync/github.ts: local cleaner replaced by the shared one - both frontmatter readers stop matching indented lines, and trust a resolved boolean when the key's line carries no inline text to contradict it `allowedTools` stays bag-only: the body scan reads booleans, not YAML sequences, so a body-only edit must not drop a list it cannot re-read.	2026-08-04 07:37:36 -04:00
Danny Avila	120ee2afa6	🚦 fix: Bound, Single-Flight, and Retry Skill File Priming Uploads (#14611) ... Skill priming fanned out one unbounded batch upload per cold skill, bursting through codeapi's per-user upload limiter (30 per 5 min). Failures degraded silently: nothing persisted, every turn re-burned budget, and handle_skill reported success with no files mounted. - Bound batch uploads to 3 process-wide slots across both prime paths - Single-flight primeSkillFiles per (skill id, version) - Retry a 429 once per Retry-After, capped at 15s, fresh streams - handle_skill now tells the model when bundled files are unavailable - Warn on fulfilled-null primes in primeInvokedSkills	2026-08-03 13:21:47 -04:00
Danny Avila	b11978017d	🧱 fix: Enforce Agent Runtime File Trust Boundaries (#14577) ... * fix: secure agent runtime file metadata * chore: sort agent resource test imports * fix: Align Agent Tool Resource Types * fix: Rehydrate Agent Image Resources * fix: preserve remote agent file authorization	2026-08-02 14:18:28 -04:00
Danny Avila	db6ba5392a	🪢 fix: Bind MCP OAuth Secrets to Trusted Endpoints (#14578) ... * fix: bind MCP OAuth secrets to trusted endpoints * fix: bind stored MCP OAuth clients during refresh * fix: address MCP OAuth review findings * fix: bind stored MCP OAuth credentials * fix: make MCP OAuth credentials generation-safe * test: update MCP OAuth uninstall binding fixtures * fix: harden MCP OAuth credential persistence * fix: scope MCP OAuth refresh single-flight * style: sort MCP OAuth token imports	2026-08-02 13:38:58 -04:00
Danny Avila	928b14f5bc	🔒 fix: Single-Flight MCP OAuth Token Refresh per User/Server (#14596) ... * 🔒 fix: Single-Flight MCP OAuth Token Refresh per User/Server Concurrent refresh-token redemptions (tool-call 401, ping, reconnect retries, expired-token reads) each replayed the same stored refresh token at the OAuth token endpoint. RFC 9700 reuse detection treats the replay as theft and revokes the entire grant family, forcing manual re-consent every access-token expiry. MCPTokenStorage.forceRefreshTokens is the choke point every refresh path converges on; it now single-flights redemptions per (tenantId, userId, serverName) so concurrent callers share one wire call and receive the same rotated result. The refresh token is re-read from storage inside the locked execution — never from a caller snapshot — so a redemption starting after another refresh completed uses the rotated token instead of replaying the consumed one. Fixes #14583 * 🧪 test: Isolate Single-Flight Keys per Test via Unique Server Names * 🔒 fix: Evict Stalled Refresh Slots, Decouple Waiter Aborts from Shared Redemption Codex review round 1: - A redemption that never settles no longer wedges the single-flight slot until process restart: a stale-entry timer evicts the map entry so later refreshes start fresh, while existing waiters keep their promise. - Caller AbortSignals no longer thread into the shared redemption. An impatient waiter (silent refresh's short timeout) resolves its own wait with null via a per-waiter race; the shared wire call proceeds for everyone else, bounded by transport timeouts plus eviction. * 🔒 fix: Abort Stalled Refreshes Before Slot Release, Hook Cache Invalidation to Redemption Codex review round 2: - The stale timer now aborts the wedged execution instead of deleting its slot; the slot frees only once the execution has settled, and an abort guard before the token-endpoint call stops a woken pre-wire stall from replaying a refresh token a successor already rotated. - New onRefreshSuccess hook runs inside the shared redemption after rotated tokens persist, so the silent-refresh path's mcp_get_tokens cache invalidation fires even when the initiating waiter timed out before the redemption completed. * 📝 docs: Record Post-Dispatch Abort Recovery Rationale on Stale-Refresh Valve	2026-08-02 06:39:44 -04:00
Danny Avila	7e74f8eb8c	🪪 fix: Strip Unresolved Header Placeholders at Final Resolution (#14595) ... Unresolved {{LIBRECHAT_USER_*}} header templates leaked literally to upstream providers when user context was missing at resolution time (e.g. async title generation racing client disposal), letting a gateway trust LibreChat's own template syntax as an account identity. resolveHeaders now takes an opt-in stripUnresolved flag that blanks any resolvable-but-unresolved LIBRECHAT_USER/BODY/OPENID placeholder, enabled at every final resolution boundary (resolveConfigHeaders, model fetches, Google init, summarization overrides, azureAssistants init). Staged passes that resolve again later with more context are left untouched, as is the async-resolved {{LIBRECHAT_GRAPH_ACCESS_TOKEN}} and unknown names. titleConvo now resolves headers from the req captured at entry instead of re-reading this.options.req, which disposeClient nulls concurrently. Fixes #14580	2026-08-02 06:38:06 -04:00
Danny Avila	59395a6bf0	🪢 refactor: Move Agent Execution Seam Before Initialization (#14581) ... * refactor: move agent execution seam before initialization * refactor: type librechat agent request extensions * refactor: read envelope values from descriptors * fix: preserve envelope types and validation errors * fix: bound agent envelope traversal	2026-08-01 18:25:34 -04:00
Danny Avila	ad0f72dede	🌀 ci: Deterministic Circular Dependency Checks (#14579) ... * 🌀 ci: Deterministic Circular Dependency Checks * 🌀 ci: Enforce Type-Level Edges in Circular Dependency Scan * 🌀 ci: Materialize Import-Type Expression Edges in Cycle Scan * 🌀 ci: Collect Inline Type-Only Specifier Edges in Cycle Scan	2026-08-01 14:43:26 -04:00
Danny Avila	b253b623fe	📦 chore: update sanitize-html to latest (#14573) ... * 📦 chore: update `sanitize-html` to latest * chore: add additional modules to esModules for Jest configuration	2026-08-01 09:35:57 -04:00
Danny Avila	9fbea04d46	🦗 fix: Deliver Abort Acknowledgements on Zero-Subscriber Replicas (#14569)	2026-08-01 09:00:41 -04:00
Danny Avila	6f45a9e32e	🔗 fix: Normalize MCP Tool Keys at Every Producer, Resolve Raw Names via Aliases (#14553) ... * 🔗 fix: Normalize MCP Tool Keys at Every Producer, Resolve Raw Names via Aliases Tool keys had two spellings that could diverge for any server whose name contains characters outside [a-zA-Z0-9_.-]: the tool cache (and registry inspector) built keys with the RAW server name, while runtime instances are named with normalizeServerName(serverName). Three code comments already asserted "tool keys embed the normalized server name" - no producer honored it. For a special-character server that meant: - definitions-only mode shipped raw def names the model echoed back, but the executor's tool map held the normalized instance name, so every call failed with "Tool not found"; - per-tool tool_options (defer_loading / allowed_callers / run_in_background / describe_intent) were persisted under raw keys that never matched the definition names the option passes resolve against, so builder settings were silently inert; - tool-key parsing against normalized candidate lists silently fell back to last-delimiter splitting, which mis-parses delimiter-bearing tool names. The reconciliation is one contract enforced in three moves: 1. PRODUCERS NORMALIZE. The tool cache (packages/api/src/mcp/tools.ts) and the registry inspector build keys with the normalized server name, matching the instance names MCP.js has always assigned. The builder's tool ids, agent.tools entries, tool_options keys, and definition names all flow from these keys, so every model-facing name now agrees. The cache STORE stays keyed by the raw config name. 2. CONFIG LOOKUPS RESOLVE ALIASES. New shared helpers in data-provider (buildServerNameAliases, normalizeMCPToolKey) map a parsed normalized name back to the raw config name that the registry, config maps, tool cache, and plugin-auth rows are keyed by. Applied in the definitions loader closure, handleTools grouping, createMCPTool's parsing fallback, getUserMCPAuthMap, and the MCP tools endpoint - matching both spellings so legacy raw keys keep resolving. 3. LEGACY DATA HEALS AT ONE BOUNDARY. initializeAgent rewrites raw-keyed agent.tools entries and tool_options keys to the normalized form (normalizeAgentToolKeys) before anything consumes them, so agents persisted under the old convention load their tools AND have all four per-tool options honored. Placeholder and server-pin tokens stay raw - they are config-identity references, not model-facing names. Servers whose names are already in the safe character set (the common case) produce byte-identical keys before and after; the fast path allocates nothing. Stale Redis-cached raw keys self-heal via the existing reconnect-on-missing path within one cache cycle. * 🧯 fix: Deterministic Alias Collisions + Raw Names in Definition Metadata Two review findings on the normalization contract: - Two configured server names that normalize to the same segment (e.g. 'Sales Force' and 'Sales:Force' -> 'Sales_Force') produce inherently ambiguous tool keys; the alias map silently resolved last-wins, so a tool selected from one server could execute against the other's config. buildServerNameAliases now resolves collisions to the FIRST configured name deterministically, and resolveMCPServerContext warns once per colliding pair per process so the operator can rename one server. A collision-resistant identifier would change every existing tool key, so detection + stable routing is the right treatment here; startup-time config validation can follow separately. - The definitions loader resolved parsed (normalized) server names to raw only inside the ToolService closure, while the definition metadata (serverName -> mcpRawServerName) kept the normalized value. Server instructions are keyed by raw config names, so a special-character server's instructions were silently omitted in definitions-only mode. loadToolDefinitions now takes rawServerNames, resolves the boundary against both spellings, and stores the RAW name in definition metadata - consistent with the instance path. * 🧯 fix: Heal Stale Caches, Skill Allowed-Tools, and Builder Selectors Three review findings on the normalization rollout, all in the transition class: - Stale cache entries (P1): the definitions-only loader treats the per-server tool map as authoritative and never reconnects on a per-key miss, so a pre-change raw-keyed Redis entry would make a special-character server's tools vanish for up to the cache TTL. getMCPServerTools now heals legacy raw-keyed entries to the normalized format at read time (keys and function names), covering every consumer with no coordinated invalidation; safe names return the map untouched. - Skill allowed-tools: a skill declaring a raw MCP key in allowed-tools bypassed the initialize-boundary heal (the union runs after it) and would neither dedupe against healed agent tools nor match the normalized tool map. The primes' allowedTools now pass through the same normalizeAgentToolKeys heal before unioning. - Builder selectors: matchesMcpServer and useVisibleTools parsed tool ids against raw server names only, so an attached special-character server rendered as an unselected orphan card. Both now accept the normalized spelling and resolve it back to the raw map key, keeping legacy raw ids working. * 🧯 fix: Fail Closed on Normalized Server-Name Collisions Escalation of the collision finding: a deterministic first-wins alias plus a warning still let the tools listing publish BOTH colliding servers, so a tool selected under the shadowed second server would silently execute against the first server's configuration (their model-facing keys are identical, so routing cannot ever distinguish them). - findShadowedServerNames identifies later-configured names whose normalized form an earlier different name claimed. - getMCPTools excludes shadowed servers from the published listing entirely (with a warn naming the collision), so their tools are never selectable - nothing ambiguous can be picked. - Server creation reserves both spellings: a generated slug may not collide with a raw config name OR the normalized form its tool keys would carry. Collision-resistant model-facing IDs remain out of scope: changing normalizeServerName's output would rewrite every existing tool key (agent documents, caches, instance names) for ALL servers to handle a misconfiguration that is now blocked from exposure instead. * ✅ fix: Dedupe Reserved Server-Name Spellings at Creation The reservation list appended normalized forms unconditionally, which duplicated every safe name (raw === normalized) and broke the route-level contract test pinning the exact list. Dedupe via a Set so safe names contribute one entry, while special-character names still reserve both spellings; adds the special-character reservation case. * 🧯 fix: Never Heal a Shadowed Server's Keys; Align Authorization Tie-Break Persisted references were the remaining collision vector: an agent or skill saved with the shadowed later server's raw key was HEALED into the shared normalized key, authorized through a last-wins map, and routed first-wins - authorized as one server, executed as another. - normalizeAgentToolKeys now refuses to rewrite keys of shadowed servers (findShadowedServerNames): rewriting would produce exactly the first server's key. Left raw, the key cannot match the normalized-keyed tool map and the tool fails visibly - broken beats misrouted. Covers agent.tools, tool_options, and skill allowed-tools through the shared heal. - filterAuthorizedTools (agents/v1.js) builds its normalized-to-raw map via the shared buildServerNameAliases instead of a last-wins Map constructor, so authorization resolves a colliding key to the SAME first server execution routes to. * 🧯 fix: Direct Identity Wins Over Aliases; Heal Client Forms and Degraded Contexts Four review findings on the normalization edges: - Alias hijack (P1): a user-DB server named exactly like an operator server's normalized form ('foo' vs YAML 'foo!') had its tools rerouted to the operator server by unconditional alias resolution. Resolution is now DIRECT-FIRST everywhere: the parsed name is tried as-is, and only when nothing resolves is it treated as a normalized spelling (definitions loader, handleTools grouping, createMCPTool fallback). buildServerNameAliases seats identity entries before derived ones so a literal name owns its slot regardless of config order, findShadowedServerNames and the collision warning derive from the same construction, and getUserMCPAuthMap fetches auth under both spellings so either owner finds its rows. - Builder double-match: a normalized name containing the delimiter ('foo mcp bar' -> 'foo_mcp_bar') also suffix-matched a server named 'bar', selecting both cards and making removal strip the wrong tool. matchesMcpServer now resolves the token ONCE against the full configured list (longest boundary, both spellings) when the caller supplies it; selection and removal share the resolution. - Builder legacy ids: an agent saved with raw-keyed ids showed its tools unchecked while the runtime heal kept them active, and selection updates never replaced the legacy entries. McpSection maps legacy raw ids to their current normalized ids when deriving and rewriting this server's selection. - Degraded context: a transient ensureConfigServers failure returned an entirely empty context, leaving normalized keys unresolvable for the request. resolveMCPServerContext now keeps the name lists (they derive from the config snapshot alone) and degrades only the lazy-init configs. * 🧯 fix: Collision Detection Sees Accessible Servers; Shadowed Refs Fail Closed End to End Round follow-ups on the collision design, all in the DB-server-visibility class: - The legacy-key heal detected collisions against operator-config names only, so healing could still produce a key that direct-first resolution routes to an invisible user-DB server. initializeAgent gains an optional getAccessibleMcpServerNames dep (wired through ToolService for controllers that mock it, directly elsewhere), consulted ONLY when a configured name needs normalization - zero cost for safe-name deployments. The heal then sees the full accessible set and skips shadowed servers' keys. - Wildcard and legacy raw tokens bypassed catalog filtering, letting a shadowed server's instances join a run under the same normalized names as the winner's. filterAuthorizedTools rejects tools of shadowed servers at authorization (its merged map sees DB + config), and handleTools skips them at execution. - The builder migrated only tool selection, not tool_options: legacy raw option keys showed disabled while the runtime honored them, and toggles could not clear them. McpSection now migrates option keys to the current normalized ids (existing normalized entries win). - A transient ensureConfigServers failure degraded to an EMPTY server context, leaving normalized keys unresolvable for the request. resolveMCPServerContext keeps the name lists (derived from the config snapshot alone) and degrades only the lazy-init configs. * 🧯 fix: Complete the Collision Audit at Every Gate; Safer Heal Semantics Round follow-ups hardening the collision audit: - Execution guards now consult the FULL accessible set: the caller's heal threads its already-fetched names through loadTools, and handleTools fetches them itself when a configured name needs normalization (never for safe-name deployments) - so a cross-tier collision (user-DB 'foo' vs operator 'foo!') fails closed at eager execution instead of joining the run under one normalized name. - Healing is SKIPPED when the collision audit cannot complete (transient lookup failure, or no dep): un-healed raw keys still resolve through the direct-first candidates, so skipping is safe while rewriting against an incomplete audit is not. - The audit lookup is gated on the agent actually carrying delimiter-bearing keys (tools, tool_options, or skill allowed-tools), so non-MCP agents never pay a registry round-trip even on specially named deployments. - normalizeAgentToolKeys gives the CURRENT (normalized) entry precedence when both spellings carry options, matching the builder's migration semantics instead of letting insertion order decide. - The builder's toCurrentToolId resolves entries boundary-exactly against every configured server (longest match, both spellings), so a raw suffix shared with a LONGER server name can no longer reassign that server's selection or options while another dialog is open. * 🧯 fix: Shared Collision Audit for Definitions Loading; Fail Closed on Audit Failure Round follow-ups closing the remaining audit gaps: - The definitions-only loader now consumes the same collision audit as eager loading: shadowed servers' entries (wildcards included) are dropped before definitions are emitted, so the default execution path can never resolve a shadowed server's normalized function name to another server. The audit names thread from initializeAgent's heal; the loader self-fetches only when a configured name needs normalization. - resolveCollisionAuditNames centralizes the audit-resolution policy (threaded set > self-fetch when needed > incomplete on failure), and BOTH loaders now fail closed under an incomplete audit: any normalization-sensitive reference (its own name needs normalizing, or it equals the normalized form of a configured special-character name) is skipped with a warning instead of being audited against operator names alone. isNormalizationSensitiveName lives in packages/api as a pure helper so test mocks use the real predicate. - normalizeAgentToolKeys collapses duplicate ids after healing (order-preserving): a document carrying both spellings converges on one key, never two instances with the same function name. * 🧯 fix: Thread the Audit Everywhere; Identity-Aware Alias Fallback Round follow-ups on audit plumbing: - The OpenAI-compatible and Responses tool loaders now forward the already-resolved accessibleMcpServerNames instead of discarding it, so the definitions loader neither repeats the registry lookup nor fails closed on a transient second lookup after the first succeeded. - The skill-only path threads its audit: when the baseline agent has no MCP keys but a primed skill's allowed-tools fetched the complete set, that set (not the operator-only list) reaches the loader, so the collision remains visible and the shadowed reference stays rejected end to end. - OAuth discovery iterates the collision-FILTERED tool list, so a request can no longer emit an OAuth prompt, wait out the connection timeout, and reconnect a server whose definitions were deliberately rejected. - The definitions loader's alias fallback is identity-aware: when the parsed name IS a known accessible server, a null tool fetch means temporarily unavailable (OAuth pending, missing user variables, disconnected) and no longer reroutes to the raw alias - previously the aliased operator server's definitions could be emitted under the unavailable DB server's names. * 🧯 fix: Legacy-Key Definition Lookup; Retain Audit for Deferred Execution - createMCPTool resolves tool definitions by BOTH spellings: the key as persisted plus the canonical normalized key built from the resolved server name. Assistants and direct tool calls persisted before the rollout bypass the agent-boundary heal and arrive with raw keys, while availableTools is now indexed canonically - previously every such call missed the index, burned a reconnect, and returned the unavailable stub permanently via the negative cache. - The initialized agent retains accessibleMcpServerNames (the COMPLETE collision audit this initialization resolved), buildAgentToolContext copies it into every per-agent tool context, and loadToolsForExecution threads it into the eager loader as bare options. Deferred/event-driven execution therefore reuses the snapshot instead of repeating the merged registry read - a transient failure there could fail-closed a tool the same turn already advertised from the successful first audit. - MCP.spec.js keeps @librechat/api pure helpers REAL (requireActual spread) so normalization paths are exercised rather than mirrored. * 🧯 fix: Parse Legacy Keys Against Both Server-Name Spellings createMCPTool's boundary candidates were normalized-only, so a legacy raw key whose server name contains the delimiter (foo_mcp_bar!) missed the suffix match and fell to the generic last-delimiter split - the canonical rebuild then produced a key that could never hit the index and the persisted call stubbed out. The candidate list now carries the RAW resolved name (and raw config names on the parse-only path) next to the normalized spellings. * 🧯 fix: Honest Audit Completeness; Shadowed-Server Form-Key Guard - resolveAllMcpConfigs tolerates ensureConfigServers failures, so the merged registry read can silently omit config-only servers while the audit still reported complete: true - a foo/foo! collision would go unseen and a persisted key could route to the wrong server. Both audit consumers now union the snapshot-derived raw config names back in (resolveCollisionAuditNames unions the caller's rawServerNames; the initializeAgent heal unions configRawServerNames), keeping the completeness label honest without an extra read: operator names come from the registry-independent config snapshot, user-DB names from the merged read that fails loudly into the existing incomplete path. - The client tool_options migration now mirrors the runtime heal's fail-closed rule for SHADOWED servers: when the dialog's server has lost its normalized slot to another catalog name, legacy raw keys stay raw instead of being rewritten onto the winning server's key, where a later save would apply the wrong server's per-tool settings. The dialog's own server joins the alias construction so a stale catalog map can't misread as a collision. * 🧯 fix: Heal Legacy Assistant MCP Tool Names on Save The assistants create/update controllers look tools up in the cached definitions by exact key, and the cache is now normalized-keyed - an assistant saved before the convention resubmits its raw-suffixed MCP name on every edit, so any save silently removed the tool. healMcpToolNames pre-heals the payload's tool list: a delimiter-bearing string that misses the cache resolves through the configured raw names (longest-suffix, boundary-exact) and rewrites to the normalized key only when that key actually exists in the cache. SHADOWED raw names stay raw and fail closed, mirroring the runtime heal; the config read happens only when a delimiter-bearing name actually misses, and read failures propagate (write path) rather than silently dropping tools. v2's update loop also stops re-reading the tool cache per iteration. * 🧯 fix: Full-Audit Shadow Set + Dedupe in the Assistant Key Heal - The assistant-save heal built its shadow set from operator config names alone, so a cross-tier collision (user-DB `foo` owning the normalized slot of operator `foo!`) looked unshadowed and the legacy key healed into the shared normalized name - which direct-first execution then binds to the DB server. The shadow set now comes from resolveCollisionAuditNames' full accessible audit, and an incomplete audit skips healing outright (every rewrite candidate is normalization-sensitive by construction, so raw-and-fail-closed is the only safe answer). - Healed string entries dedupe order-preserving: a payload carrying both spellings of the same tool collapses to one entry instead of expanding into duplicate function definitions the provider rejects.	2026-08-01 07:39:24 -04:00
Danny Avila	de033b7dbd	🦗 fix: Ignore Sequenced Redis Events Without SSE Subscribers (#14557)	2026-08-01 07:22:05 -04:00
Danny Avila	b9ca391b84	📦 chore: bump @librechat/agents to v3.3.11 (#14562)	2026-08-01 02:39:24 -04:00
Danny Avila	e7f1838515	⚡ feat: Reliable Interrupt & Steer Escalation and Recovery (#14558) ... * feat: surface interrupt-steer escalation on waiting messages The interrupt & steer feature shipped reachable only through the composer chord, the send-button hovercard, and the composer button; a message already waiting (queued for after the run, or steered and parked at the next tool boundary) had no path to it. Both waiting surfaces now carry one: - Queued rows get an icon-only ZapOff escalation button beside the existing Steer primary. It routes through sendQueuedNow, which now takes a preempt option on its live-run path. The tooltip teaches the composer chord, derived through resolveComposerKeyDown so a rebound or yielded chord is never advertised. - In-flight steer bubbles get an "Interrupt now" overflow entry with the same race rules as Edit: reclaim first, and only a `reclaimed` outcome resubmits (via retrySteer with preempt, swapping the chip for an interrupting one). `applied` and run-ended-mid-reclaim outcomes stop at the existing informational toasts, so the words can never land twice. Not offered on a steer already preempting. - Every during-run overflow menu gains an "Always interrupt instead" toggle for steerInterruptsByDefault, next to the existing steer/queue default toggle. MenuEntry supports disabled for the new entries. Only one interrupt can be unresolved at a time: while one preempt is pending (or the run is paused on approval, where the server 409s), every escalation control disables instead of racing the same seal. Ten new tests across both surfaces; 381 green in the affected suites. * fix: lock escalation across its reclaim window, keep the paused control visible, label as steer Codex round 1, all three findings. P2, escalation race. The single-interrupt invariant had a window between clicking "Interrupt now" and the reclaim resolving, where no preempt chip existed for the chip-derived gate to see: two bubbles escalated back-to-back could both resubmit. A shared escalating flag (Jotai, per-conversation) now covers the window and disables every escalation control on both surfaces, and a fresh recheck before resubmitting catches an interrupt armed elsewhere meanwhile (composer chord, queued row); those words re-home to the queue with an informational toast instead of breaking the invariant. P2, unreachable paused state. canSteer is defined as hasRealConvoId && !pausedOnApproval, so gating the button on canSteer removed it exactly when it was meant to render disabled; the test only passed on an impossible stub combination. The render gate is now duringRunActive && (canSteer || pausedOnApproval), and the test uses the real invariant. P2, label semantics. "Interrupt & send now" borrowed the name of the hard-abort action; this one preserves the partial answer and steers. Renamed to "Interrupt & steer now" (com_ui_interrupt_steer_now). Both behavior fixes counterfactually verified; 384 tests green across the affected suites. * fix: disable bubble escalation while the run cannot accept a steer Codex round 2, one P2. Answer mode (ask_user_question) sets duringRunActive false while pausedOnApproval stays false, since that flag only detects approval-bearing tool calls. The bubble's escalation entry stayed enabled there, so clicking it cancelled a healthy waiting steer and the preempt resubmission bounced off RUN_PAUSED, degrading the words to the queue. The entry now also disables on !duringRunActive, matching the queued-row control's gate. Counterfactually verified: reverting the gate fails the new answer-mode test. * fix: recheck live run state after the reclaim, not just at the click Codex round 3, one P2, and it is the round-1 recheck principle applied one level deeper: the entry-time disable cannot see a run that pauses (tool approval, answer mode) while the reclaim round-trip is in flight, and the .then closure held the render's stale steering controls, so the resubmit would fire into a RUN_PAUSED rejection after the reclaim had already surrendered the steer's boundary slot. The escalation continuation now reads the LIVE controls through a latest-ref: if the run can no longer accept a steer, the words re-home to the queue with an informational toast instead of resubmitting, and the resubmit itself also goes through the live controls. Counterfactually verified: reading the stale closure instead of the ref fails the new mid-reclaim pause test. * refactor: make escalation one atomic server-side arm, in place Codex round 4: four P2s, every one an interleaving of the same window — escalation as reclaim-then-repost is a compound, non-atomic operation whose continuation must revalidate the world (FIFO position lost, ref assigned too late, no run fence, competing bubble actions). Rounds 1-3 patched that window with a lock and rechecks; round 4 shows the window itself is the defect, so this removes it instead of guarding it again. Escalation is now POST /chat/steer/arm: the server flips preempt on the EXISTING queued item in one atomic store op (new IJobStore.armSteer; a decode-patch-encode LSET Lua on Redis, an in-place mutation in memory), fenced to the validated generation and refused once the queue closes. The handler mirrors the steer POST's preempt contract exactly: durable flag gated on the owner's recorded capability, volatile requestPreempt fire-and-forget because the durable flag is the truth resume/handover re-arm from. By construction this resolves all four findings: FIFO survives (the item never moves; the whole queue still drains in instruction order at the seal), there is no continuation to hold stale controls, the store op is fenced to the original run, and a competing Edit/Queue/Cancel either beats the arm (armed:false, chip untouched) or operates on the armed item, whose cancel already disarms. The client escalation entry becomes one mutation: armed:true relabels the chip in place (same steerId, same position), PREEMPT_UNSUPPORTED and lost races toast honestly, and the round 1-3 machinery — the escalating lock atom, the latest-ref, the post-reclaim rechecks and their two toast strings — is deleted rather than extended. Verified: 7 new handler tests on the real in-memory manager (including FIFO preservation and the stale-generation fence), 2 Redis integration tests against real Redis (in-place arm keeps order and every field; missing/stale/closed all refuse), client suites 396 green. * fix: decide capability inside the atomic arm, neutralize the lost-race toast Codex round 5, both findings, both edges of the new arm design rather than its mechanism. P2, capability TOCTOU. A HITL resume on a rolling deploy rewrites preemptCapable for the SAME generation, so the handler's read could go stale between validation and the flag flip, arming a steer the live owner cannot seal. armSteer now returns armed | missing | incapable, with the owner's live capability part of the same atomic predicate as the generation fence (HGET preemptCapable inside the Lua; the flat job field, not a metadata blob — the in-memory store reads the same field). The handler's pre-check is deleted rather than kept alongside; the store predicate is the single source. New handler test rewrites the capability after queueing and expects PREEMPT_UNSUPPORTED with the item left unflagged; the Redis guards test now asserts the incapable refusal against real Redis. P2, ambiguous toast. armed:false covers injected, cancelled, re-homed, and run-over alike, so telling the user the message "already reached the agent" claimed one specific outcome. The lost-race branch now uses a neutral message (com_ui_steer_arm_lost_race) and defers to the events for what actually happened. * fix: flip the escalation lock synchronously before the arm request Codex round 6, one P2. Round 4 deleted the escalating flag along with the reclaim continuation it guarded, but that left the one-interrupt gate blind during the arm request's own round trip: the chip-derived check cannot see an arm until its response relabels the chip, so on a slow connection two bubbles could both arm before either response landed. Double-arm is harmless server-side now (the run seals once and drains the whole queue in order), but every escalation control advertises "one interrupt at a time" by disabling, and the controls must tell the truth. The per-conversation escalating flag returns as a pure UX gate: set synchronously at click, before the mutation, cleared on settlement, and folded into interruptPending on both surfaces. Unlike its round 1-3 ancestor there is no continuation behind it to guard and no recheck to pair with it. Counterfactually verified: without the synchronous set, the two-bubble race test arms twice. 207 tests green across the Chat Input suites. * test(e2e): cover escalation of waiting messages through the real seal Three mock-harness tests on E2E_SLOW_REPLY, a 160-chunk stream with no tool boundary, so an in-thread steer part can ONLY come from a genuine mid-stream seal — which makes each test a behavioral proof rather than a UI check: - Queued row escalation: the ZapOff button turns a waiting queued message into a preempt-armed steer (202 echoes preempt: true) that seals and injects, where the sibling steering.spec test proves the unescalated path waits for run end instead. - Bubble in-place arm: an ordinary steer (202 with no preempt echo) waits as a bubble, POST /chat/steer/arm answers armed: true, the bubble relabels in place (same single bubble, same text, escalation no longer offered on reopen), and the armed steer seals mid-stream. - Always-interrupt toggle: flipped from a waiting row's overflow menu, plain Enter now produces a preempt: true steer that seals in the SAME run, and the menu offers the way back. An afterEach clears the localStorage preference so a mid-test failure cannot leak preempt-by-default into the rest of the serial suite. All three verified locally through the full harness (real backend, mock LLM, seeded DB): 3 passed in 27s. * feat: dedicated escalation arrow + shortcut, menu split into actions and preferences The escalation was still half-hidden: the bubble only offered it inside the overflow menu, and the tooltip taught the composer chord, which does a different thing (interrupts with typed text, not this chip). Three changes make it a first-class command: - A shared EscalateNowButton (circular arrow, ghost-bordered like the composer's interrupt control) is always visible on BOTH surfaces: beside each queued row's Steer primary and on every waiting steer bubble next to its menu. It disappears once a steer is interrupting. - A dedicated registry shortcut, escalateSteer (Cmd/Ctrl+Shift+.), editing-allowed and rebindable like every other action. Deliberately NOT an Enter chord: the composer owns every Enter chord, and the yield design rests on no default binding using Enter besides submit. Its handler clicks the newest enabled arrow control (bubbles beat queued rows), so the shortcut can never diverge from the button, and the arrow's tooltip teaches THIS command via the registry display. - The overflow menus separate one-off actions from sticky behavior changes: Edit, Cancel, Queue, then a smaller "Preferences" section holding the queueing and always-interrupt toggles, each with the standard InfoHoverCard reusing the Settings panel's descriptions. "Interrupt & steer now" leaves the menu entirely. 386 client tests green, including a menu-structure test locking the order and the absence of the escalation entry; bubble escalation tests drive the visible arrow. The e2e spec's bubble test now clicks the arrow, and a fourth test drives the dedicated shortcut end to end through a real mid-stream seal. * style: bind the escalation arrow to its message (variant A anatomy) Two same-weight circles in a row read as one control group, leaving the arrow's ownership ambiguous, and a floating arrow stops meaning anything once several messages stack. The shared control now carries variant A's anatomy: a thin divider binds a small SOLID arrow (filled, inverted) to the message region on its left, and the menu ellipsis stays a bare glyph, so the two affordances can no longer blur together — and the divider+arrow pairing repeats cleanly per chip at N messages. * chore: drop the unused within import CI lint caught * fix: advertise the escalation shortcut only while the control is live Codex on the e2e head, one P2: the tooltip appended the chord hint even while the button was disabled, advertising a shortcut that does nothing during an approval pause. The flagged control (InterruptNowButton) was since replaced by the shared EscalateNowButton, which inherited the pattern; the successor now omits the chord whenever the control is disabled, matching the rule the during-run hovercard already follows. * fix: harden steer escalation lifecycle and recovery * test(e2e): disambiguate accessible steer preferences * test: align abort persistence coverage with prerequisites * chore(i18n): remove obsolete steer race message * chore: normalize imports across steering changes * test: exercise stream integration on Redis Cluster * test: scope HITL checkpoints to generation * test: fix cluster cleanup and locale policy * fix: keep escalation visible during ask pauses * fix: fence recovery downgrade and stale predecessors * fix: require generation owner abort acknowledgement * fix: validate delayed preempt arms * test: align final escalation fixtures * fix: preserve in-memory predecessor abort handoff * fix: restore controls for recovered queued messages * test: cover recovered queue controls * fix: close final steering review gaps	2026-07-31 20:07:56 -04:00
Danny Avila	60ca751a7f	🧠 fix: Preserve Deferred Tool Schemas Across HITL Resume (#14552) ... * 🧠 fix: Preserve deferred tool schemas across HITL resume * 🧪 test: Harden deferred tool resume regression * 📦 chore: bump @librechat/agents to v3.3.10	2026-07-31 14:06:13 -04:00
Danny Avila	78ec1940a2	🧹 fix: Clean Up MCP OAuth State Mappings on Uninstall + Reject Superseded Callbacks (#14549) ... * 🧹 fix: Clean Up MCP OAuth State Mappings on Uninstall + Reject Superseded Callbacks Disconnecting an OAuth MCP server deleted its mcp_oauth flows but left the mcp_oauth_state:{state} mappings behind for the full TTL. Because flow ids are deterministic (userId:serverName) and the CSRF token is HMAC(flowId), a stale browser tab's callback could resolve its orphaned state to the NEXT flow for the same server, pass CSRF, burn the fresh flow's one-shot CSRF cookie, and fail the PKCE exchange, sabotaging the legitimate retry. - Add MCPOAuthHandler.deleteFlowAndStateMapping: reads the flow's stored state and deletes the mapping before the flow (mapping-first so a crash between deletes fails closed instead of recreating the orphan) - Route mcp_oauth deletions in clearStoredMCPOAuthState through the helper for both tenant-scoped and legacy flow ids - Reject callbacks whose state does not match the resolved flow's stored state: the only control distinguishing a superseded attempt from the current one on a deterministic flow id Fixes #14534 * fix: gate failFlow on state match in the OAuth error branch (Codex P1) The provider-error branch failed the resolved flow on CSRF/session alone, so a superseded error callback resolved through an orphaned mapping could mark the current flow FAILED. Apply the same stored-state equality gate before failFlow. * fix: leave the flow in place when the state-mapping delete fails (Codex P2) deleteFlow swallows storage errors and returns false, and deleteStateMapping discarded that result, so a failed mapping delete followed by a successful flow delete would silently recreate the orphan. Surface the boolean from deleteStateMapping and throw from deleteFlowAndStateMapping before touching the flow, so the caller's allSettled warn branch fires and the next replacement retries both. * fix: restore the state mapping when the flow delete fails (Codex P2) The inverse partial failure of the round-3 fix: a successful mapping delete followed by a silently failed flow delete left a PENDING flow whose reused authorization URL could never resolve, dead-ending every callback in invalid_state until the flow went stale. Check deleteFlow's result, re-store the mapping on failure, and throw so the caller's allSettled warn branch fires. * fix: never leave a callback-capable flow behind on uninstall (Codex round 6) Teardown runs after the server's tokens are deleted, so a preserved flow+mapping pair (the round-3 early-throw path) let a lingering consent tab complete the callback and recreate credentials post-uninstall. Now that both callback branches gate on stored-state equality, an orphaned mapping is the benign failure mode, so invert the order: delete the flow first, attempt the mapping delete regardless, and reject when either reports a storage failure. This supersedes the round-4 mapping restore, which also preserved a callback-capable pair. * fix: delete the flow even when its metadata read fails (Codex round 7) A storage error on the initial getFlowState aborted teardown before any delete ran, preserving the callback-capable flow after token deletion. Tolerate the read failure, delete the flow blindly, skip the mapping it could not identify (the callback gates neutralize the possible orphan), and reject so the caller's warn branch fires.	2026-07-31 12:11:36 -04:00
Danny Avila	f5e8feba80	📦 chore: bump @librechat/agents to v3.3.9 (#14548)	2026-07-31 10:43:00 -04:00
Danny Avila	1e1de6eff9	🎯 fix: Exact Ask-Question Attribution via Interrupt tool_call_id (#14539) ... The ask_user_question pause/answer stamps (server pause-time args stamp, resume-time answer stamp, and the client mirror) targeted the newest unanswered ask part by position. When a model emits several ask calls in one turn, the interrupt's question and the user's answer land on the wrong card. @librechat/agents > 3.3.8 surfaces the interrupting call's tool_call_id on the ask interrupt payload. All three stamps now target that id exactly when present, keeping the positional fallback for older payloads. The tool body passes config.toolCall.id through to askUserQuestion via a typed alias that is a no-op on the pinned SDK and lights up on the next dependency bump. Companion to danny-avila/agents#366, which also fixes the underlying dangling tool_use 400 (INVALID_TOOL_RESULTS) when one of the parallel ask calls streams malformed args.	2026-07-31 10:11:08 -04:00
Danny Avila	f0d3bcb622	📄 fix: Filter Non-PDF Documents on the Anthropic Encode Path (#14535) ... Anthropic's Messages API only accepts application/pdf for base64 document sources, but encodeAndFormatDocuments sent every allowlisted file (docx, xlsx, csv, html) through the base64 branch unfiltered. The provider 400 recurs on every retry because attachments are re-encoded each request, permanently breaking the conversation. - Add isAnthropicDocumentType / isAnthropicTextDocumentType to data-provider, mirroring isBedrockDocumentType - Filter unsupported types before encoding (matching Bedrock semantics) and log the skipped attachments - Send textual types as plain-text document sources (source.type 'text'), which Anthropic accepts and supports citations for, instead of invalid base64 blocks Fixes #14485	2026-07-31 09:57:02 -04:00
Danny Avila	8e165eb451	🔒 fix: Remove Owner Email from Agent owner_contact Fallback (#14541) ... * 🔒 fix: Remove Owner Email from Agent `owner_contact` Fallback The owner-contact fallback for agents without an explicit support_contact exposed the owner's private account email to any VIEW-level caller via GET /agents/:id and GET /agents. The fallback now resolves a display name only (name/username/authorName): the User query no longer projects email, the resolver never returns one, and the shared AgentOwnerContact type drops the field. Emails are only served when the owner opts in via support_contact. * 🔒 fix: Reject Email-Shaped Owner Display Names in Contact Fallback OpenID and SAML strategies fall back to the account email for the user's name and username when no display-name claims exist, so the name-only owner fallback could still surface the email through those fields. The resolver now rejects email-shaped display-name candidates entirely. * 🔒 fix: Treat Any @-Containing Display Name as Email-Derived RFC-5321 quoted local parts may contain whitespace and the User schema email validator is an unanchored substring match, so such addresses can reach the name/username fields via SSO fallbacks. Rejecting on '@' presence covers every legal email form without re-fetching the account email.	2026-07-30 23:46:22 -04:00
Jens Schumann	ad74a282d1	🪃 fix: Resolve User Vars Before the First Post-OAuth Reconnect (#14538) ... * fix: resolve customUserVars before first post-OAuth-callback MCP reconnect The OAuth callback route reconnects the user's MCP connection immediately after storing new tokens, but never resolves customUserVars before doing so - unlike the /reinitialize route a few hundred lines below, which does. As a result, headers/oauth_headers templates like `{{MY_KEY}}` are sent to the MCP server literally, unsubstituted, on this first connection attempt, even though the user's value is already saved. The upstream server rejects it as an invalid credential. Fixes #14537 * refactor: share getServerCustomUserVars reader from @librechat/api The mcp_-prefixed key shape was built by getUserMCPAuthMap but re-derived by hand at each read site (a private helper in services/MCP.js, and the new callback-route extraction). Export a reader from the same module that owns the writer and reuse it at both sites, so the key shape has a single source of truth. * chore: sort destructured require members in routes/mcp.js --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-07-30 22:17:06 -04:00
Danny Avila	d6c2dc5d8e	🧵 feat: Background-Native Code Execution Tools (#14532) ... The code-execution pair (execute_code/bash_tool) now defaults INTO background dispatch whenever the run_in_background capability is enabled, the same way the SDK's coding tools carry `intent` natively: enabling the capability is enough, with no per-tool or per-spec flag required. An explicit run_in_background: false opts the pair out (by definition name, marker projection, or a narrowing spec selection's wildcard), and the builder Code toggle flips to opt-out semantics: absent reads as on, and turning it off persists an explicit false. A spec's runInBackground: false now synthesizes an explicit wildcard opt-out instead of staying a silent no-op. Pre-native, false and absent were behaviorally identical, so a config that wrote false must not silently flip to backgrounding code. The ephemeral toggle's false stays no-policy: it is a badge default, not a decision.	2026-07-30 17:53:47 -04:00
Danny Avila	d5819becf2	🎯 feat: Per-Tool Intent Labels for Model Specs (#14526) ... * 🎯 feat: Per-Tool Intent Labels for Model Specs A model spec could only turn intent labels on for ALL of its eligible tools. Saved agents have had per-tool control since the capability landed (`tool_options[id].describe_intent`, with the builder toggle following in the UI slice), but a model spec is admin YAML that produces an ephemeral agent — there is no agent document to hold per-tool options, so `describeIntent: true` synthesized an entry for every eligible tool. `describeIntent` now accepts a string array alongside the boolean, matching the `skills` field already in the same schema: describeIntent: true # every eligible tool describeIntent: ['web_search', 'search_code_mcp_github'] This matters because the label costs schema tokens on every request, so an admin may want it on a handful of illegible calls rather than the whole toolset. - Named tools still pass eligibility, so an excluded tool cannot be forced on by listing it. - An empty array reads as disabled. - Names that are not eligible or not equipped on the spec are logged rather than silently skipped — a typo in a spec would otherwise be undiagnosable. - The ephemeral toggle stays boolean and stays global even when a spec list is present: it has no per-tool UI to drive it, so narrowing it would silently cover fewer tools than the user asked for. `runInBackground` has the same all-or-nothing limitation and could take the same shape; left alone here to keep this change reviewable. * 🎯 feat: Per-Tool Background Dispatch for Model Specs Gives `runInBackground` the same `boolean | string[]` shape as `describeIntent`, so the two per-tool capabilities are configured identically from a model spec: runInBackground: true # every eligible tool runInBackground: ['slow_report_mcp_analytics'] # only this one Selectivity matters more here than for intent labels. An intent label is inert — it costs tokens and nothing else. Backgrounding changes execution semantics: the model gets a synthetic handle and must poll. Letting an admin detach one slow MCP call without making every other tool in the spec detachable is the difference between a usable setting and an all-or-nothing one. Same guarantees as the intent equivalent: - Named tools still pass eligibility, so the exclusion list still holds — a list cannot force on web_search, file_search, image gen, the HITL tool, or anything whose attachments/artifact continuity would break. - Empty array reads as disabled. - Unmatched names are logged rather than silently skipped. - The ephemeral toggle stays boolean and global; it has no per-tool UI. Also fixes a latent no-op: this function did not skip the lazily-expanded `mcp_all` placeholder, so a spec with an overlay MCP server recorded an option under a name `applyBackgroundToolCalls` can never match. The intent equivalent already skipped it; now both do. * ♻️ refactor: One Definition of the mcp_all Placeholder Guard Fixing the background no-op left the placeholder prefix declared twice — once per capability synthesizer — which is the same duplicated-literal shape that made the intent label marker fragile: two copies that must agree, with drift producing a silent no-op rather than an error. `MCP_ALL_PLACEHOLDER_PREFIX` and `isMCPAllPlaceholder` now live beside `mcpToolPattern` in mcp/utils, so both synthesizers cannot disagree about which tool entries to ignore, and anything added later that keys per-tool config by exact name has an obvious guard to reach for. Audited the rest of the capability family while here: only background and intent synthesize per-tool options from a model spec. `defer_loading` and `allowed_callers` have no model-spec path at all, so neither can carry this bug. Both synthesizers now have an explicit regression test naming the placeholder. * 🧯 fix: Treat a describeIntent List as a Selection Policy, Not a Filter Two build/behavior defects from the previous commits. **Narrowing did not actually narrow.** Omitting a tool from the synthesized options is not the same as opting it out, because intent has two default-on paths background does not: `isIntentOptedIn` treats every NATIVE_INTENT_TOOL_NAMES member as enabled when it finds no entry, and `sanitizeIntentLabels` keeps an SDK-native label unless it sees an explicit `describe_intent: false`. So `describeIntent: ['web_search']` still labelled `set_memory`, and an empty list — the most explicit way to say "none" — disabled nothing at all. A list is now a selection policy: selected eligible tools get true, unselected get an explicit false. Ineligible tools still get no entry at all. Background needs no equivalent change: it opts in on `run_in_background === true` only, with no default-on set, so omission there genuinely means off. **Fixed the CI build break.** `MCP_ALL_PLACEHOLDER_PREFIX` was exported without a type annotation, which `tsc --noEmit` accepts but the package build rejects under `--isolatedDeclarations` (TS9010) — the same reason `mcpToolPattern` beside it is annotated `: RegExp`. Verified with an actual `npm run build` this time, not just a typecheck. * 🧯 fix: Propagate Intent Selection Through Capability Marker Expansion A model spec's `tools` carries capability MARKERS, not the definition names initialization actually registers, so an option recorded under a marker never matches the tool it becomes — the same silent no-op as an `mcp_all` placeholder entry. Harmless for an opt-IN (the tool keeps its default) but not for the opt-OUTs the previous commit introduced: `memory` becomes `set_memory` + `delete_memory`, both default-on natives, and `execute_code` becomes `bash_tool`, which carries an SDK-native label that survives unless sanitize sees an explicit false. So `describeIntent: ['web_search']` on a spec with `memory: true` still labelled both memory tools, and `[]` disabled neither. `expandIntentToolOptions` propagates a marker's value to the names it expands into, mirroring `expandCodeToolOptions` in background.ts which solves the same problem for the code marker. Applied at APPLY time rather than synthesis, so hand-edited saved agents that key options by marker benefit too, not just synthesized specs. An explicit per-tool entry always wins — expansion only fills names the caller did not already decide — and it runs in both `applyIntentLabels` and `sanitizeIntentLabels`, since the SDK-native strip reads the same options. Verified with an actual package build, not just a typecheck. * ♻️ refactor: Resolve Spec Tool Selections Against Final Definitions Three review rounds hit the same root cause from three directions: a per-tool option synthesized at load time is keyed by names that may not exist at injection time. Spec tools carry capability markers (execute_code, memory), skills never reach the tools array at all, and lazy MCP servers expand after synthesis - every mismatch was a silent no-op, and each fix added another hand-maintained marker map that the next case leaked past. This removes the name-space gap instead of bridging it per case: - Synthesis records the selection as a policy: a wildcard '*' entry carries the default (true for "every tool", false for "only the named ones") and listed names are recorded verbatim. No load-time enumeration, eligibility checks, or placeholder special-casing. - Resolution happens at injection time, per final definition: explicit name -> capability marker projection -> wildcard. The projection maps a marker onto the names its registration actually produced this run - the registrars report their own tool names and initializeAgent accumulates them - so the mapping cannot drift from what gets registered. - The unmatched-name diagnosis moves to the apply passes, the one place the real definitions are known, so a selection naming a marker whose runtime expansion is entirely ineligible (runInBackground: ['memory']) is now warned about instead of recorded as a dead success. Covers all four round-3 findings: code opt-outs now reach create_file/edit_file/read_file, skill definitions are governed by narrowing selections, the memory marker is rejected and diagnosed for backgrounding, and the mcp_all placeholder predicate is no longer consulted for selections at all (its one remaining consumer is the definitions loader that defines the convention). * 🧯 fix: Reject Dead Ask-Tool Selections and the Reserved Wildcard Two review findings on the selection policy, both fixed at the point where they are knowably invalid: - ask_user_question joins EXCLUDED_INTENT_TOOL_NAMES: createRun strips its provisional definition and rebuilds the graph tool from its own Zod schema, so definition-level injection never reaches the model. A describeIntent selection naming it now warns as ineligible instead of crediting a label that gets discarded. Real intent support for the ask tool lands with the HITL slice via the interrupt payload. - A literal '*' in a describeIntent/runInBackground list is dropped with a warning at synthesis: it would overwrite the wildcard opt-out default and silently enable the capability for every eligible tool instead of selecting one named tool. The wildcard is reserved for the internal policy; boolean true is the supported way to cover everything.	2026-07-30 13:32:04 -04:00
Danny Avila	7bb6651883	🛑 feat: Preemptive Steer - Backend Interrupt & Steer (#14518) ... * 🛑 feat: Preemptive Steer — server half (Interrupt & Steer, PR 2 of 3) Lets the steer route ask the generating replica to seal its live model stream at the next provider-safe boundary instead of waiting for a tool step. The run is never aborted, job status never changes, the partial answer is kept, and generation resumes in the same assistant message after the injected steer. Consumes the SDK seam in @librechat/agents (danny-avila/agents#335, #346). Transport: IEventTransport gains a fenced emitPreempt/onPreempt pair beside abort. RedisEventTransport fans PREEMPT out on the SAME events channel and subscription (no new connection, key, or subscribe call); onPreempt returns a registration-scoped unsubscribe with the same replacement-safe state-identity guard onAbort uses. InMemory implements neither — single-process preempt lives entirely in the runtime set. Runtime state: RuntimeJobState carries the per-generation request set, createdAt-fenced and capped at STEER_QUEUE_MAX_DEPTH, plus a bounded `cleared` tombstone so a late cross-replica arm cannot resurrect a request whose steer already drained. registerPreemptSubscription mirrors the abort registration's double fence (runtime identity + generation createdAt); releaseAbortSubscription retires BOTH listeners and the armed set, so every terminal path drops preempt state for free. Public surface: requestPreempt (arm + fenced publish, never a rejection surface, never touches job status), isPreemptRequested (O(1) level-triggered poll), noteSteersRemoved (drain/cancel bookkeeping + fenced clear), clearPreemptRequests (empty-boundary disarm). One drain body, two boundaries: createSteerDrainHook (PostToolBatch) and createSteerPreemptBoundaryHook (PreemptBoundary) share drainAndBuildInjections, so the two injection sites cannot drift — the SDK's provider-safety argument rests on identical HumanMessage shapes. The shared body builds injections incrementally under a swallow-all catch (a mid-loop throw still injects what was applied — those parts are already persisted), clears preempt requests in finally, and disarms the generation when a boundary drains nothing. Request path: POST /chat/steer accepts preempt: true. The guard ladder is unchanged in order and in every status code. A preempt request is NEVER a rejection reason — without the capability the steer still enqueues and the 202 echoes preempt: false. Armed strictly after a successful enqueue; cancel disarms. The capability is read from the OWNING replica's recorded `preemptCapable` rather than the route replica's own SDK probe, so a rolling deploy cannot label a steer "interrupting" that the older owner will only inject at a tool step. Durable label: SteerQueueItem.preempt → TPendingSteer.preempt, so a parked/claimed/replayed chip keeps its wording. Run wiring: createRun registers the PreemptBoundary hook and threads RunConfig.preemption, both gated on isSteerPreemptSupported() — a separate probe from isSteeringSupported(), so the client affordance can never arm against an SDK that only injects at tool boundaries. buildSteerWiring builds both hooks from one shared closures object, so preemption survives HITL pause/resume for free. Honest finalization: an empty preempt boundary persists and emits with unfinished: true — the same contract an abort gets — re-marked explicitly because BaseClient has already saved the row as unfinished: false by that point. Not changed: no new job status, store method, Lua, SSE event type, endpoint, or authorization surface. abortJob, completeJob, transitionStatus, closeAndDrainSteers, getResumeState, emitChunk, applySteerPart and the whole abort path are untouched. Tests: 120 packages/api steering specs (preempt lifecycle, tombstone, fences, caps, terminal release, both-boundary drain parity, level-triggered poll, request/cancel arming, owner-capability degradation) plus 5 in api for buildSteerWiring gating, and 2 Redis-gated cross-replica transport specs. * 🔒 fix: Codex round 2 — evict tombstones, scope the empty-boundary disarm, honest resumes All four server findings were fresh consequences of the round-1 fixes, which is the review doing exactly what it should. - Tombstone cap refused new entries instead of evicting. Every drained or cancelled steer is tombstoned, not just preempting ones, so a generation that processed 20 steers exhausted the set and the late-arm race resurfaced silently. Now evicts oldest-first (Set iteration is insertion-ordered), with the budget named PREEMPT_TOMBSTONE_MAX rather than an inline expression. - The empty-boundary disarm I added in round 1 wiped the generation's ENTIRE armed set. A second steer can enqueue and arm between the atomic drain returning empty and the disarm running — that arm is backed by a live, uninjected queue item and must survive. The drain now snapshots the armed ids BEFORE draining (getArmedPreemptIds) and clearPreemptRequests takes an explicit id list instead of clearing everything. - HITL resume finalized with a hardcoded unfinished: false. The boundary hook is re-registered on resume via buildSteerWiring, so a resumed segment can end on an empty preempt boundary exactly like a fresh one; finalizeResumedTurn now reads getPreemptStats() and the halt reason, matching the normal request path. - Ownership moves on resume, so the job's recorded preemptCapable must describe the replica that will actually generate. Refreshed before resumeCompletion; a job created on a capable replica that resumes on an older one during a rolling deploy no longer acknowledges steers as interrupting. Tests: +3 (scoped disarm sparing a post-snapshot arm, oldest-first tombstone eviction, id-list disarm). 122 packages/api steering specs green. * 🚨 fix: Codex round 3 — deserialize preemptCapable from Redis (feature was dead under Redis) The P1 here is the most consequential defect in the whole feature, and it was introduced by round 1's own capability fix. - `RedisJobStore.serializeJob` writes booleans generically, so `preemptCapable` reached Redis — but `deserializeJob` is an EXPLICIT field map and had no line for it. Every `getJob()` therefore dropped the flag, `job.metadata.preemptCapable` was always undefined, and `handleSteerRequest` computed `preemptArmed: false` unconditionally. Interrupt & steer would have silently degraded to ordinary tool-boundary steering in EVERY Redis deployment — i.e. the feature shipping as a no-op in production while passing every in-memory test. Now deserialized, with a round-trip assertion in the metadata spec that fails (`Received: undefined`) against the unfixed store. - The resume capability refresh moved from just-before `resumeCompletion` to immediately after `approvals.resolve` claims the run. That call already flips the job back to `running`, so the steer route accepts requests from that instant; leaving the refresh 135 lines later (across the whole client reconstruction) left a real window where a steer read the PREVIOUS owner's capability. Not the fully atomic transition Codex suggested — that reaches into the approvals Lua — but it shrinks the window from seconds to one await, which is proportionate for a label-accuracy issue. Refuted: "avoid triggering preemption inside subagents". The premise — that the run-wide poll can seal a subagent stream — does not hold against the shipped SDK. Child graphs are constructed with `subagentScope: true` (SubagentExecutor) and `preemption` is NOT propagated into child inputs, while `canClaimPreemptSeal()` requires `!subagentScope && preemption != null`. Both conditions fail independently, so a subagent can never claim a seal and the boundary cannot fire with `agentId` set. The `input.agentId != null` guard in the hook is defensive depth, not the thing standing between us and the described failure. 140 packages/api specs green. * 🔁 fix: Codex round 4 — re-arm durable interrupt steers when resume moves owners - An arm lives only in the owning replica's runtime plus a transient pub/sub message, while the steer's `preempt` flag is durable on the queue item. A HITL resume landing on a different replica therefore started with an empty armed set and a poll stuck false, so an interrupt the user had already been ACKed for silently waited for an ordinary tool boundary. New `GenerationJobManager.rearmQueuedPreempts` rebuilds the armed set by peeking the durable queue (fenced on the generation) and re-arming every item flagged `preempt`; resume calls it right after claiming. Safe by construction: every item peeked is still queued, so no drained steer can be resurrected. - Capability-refresh failure now logs at error rather than warn, but deliberately does NOT fail the resume — see the reply on that thread. Tests: +2 (rebuild from queue arms only the flagged item and reports the count; a stale generation arms nothing). 124 packages/api steering specs green. * 📡 fix: Codex round 5 — acknowledge only what was actually armed - A cross-replica arm was fire-and-forget: `emitPreempt` logged its own publish failure and `requestPreempt` returned void, so the route answered `preempt: true` even when the owner never armed a poll. The steer still injected at the next tool boundary, but the chip claimed an interrupt that could not happen — and unlike HITL resume, an ordinary running generation had no durable reconciliation to recover it. `emitPreempt` now resolves to the subscriber count and rejects on failure; `requestPreempt` is async and returns whether the arm truly landed (owned locally, or delivered to at least one subscriber). The 202 reports THAT rather than what was asked for, so the chip relabels to ordinary steering exactly as it does for a capability-degraded deployment. Errors are swallowed into `false` — an unarmed interrupt is a downgrade, never a failed steer. - The owner capability is re-read immediately before enqueue rather than reused from the top of the guard ladder. `checkAgentAccess` and file resolution are awaits, so a request can span an entire HITL pause/resume that moves ownership to a replica with different capability and rewrites that very flag. Only paid for by requests that actually asked to interrupt. Tests: +3 (not-armed when the publish reaches nobody; armed when this replica owns the generation; a throwing publish downgrades instead of propagating). 127 packages/api steering specs green. * 🎯 fix: Codex round 6 — real ownership, confirmed disarms, and a CI regression of my own Three review findings plus three CI failures the round-5 commit caused. Review: - Ownership came from `runtimeState`, which a cross-replica `getJob` populates with a FACADE runtime on any replica that merely read the job. Matching `createdAt` therefore proved only "we looked at this job", so a non-owner could arm nothing and report success. Ownership now comes from `ownedJobs`, the actual owner map. - `armPreemptIds` returns how many ids it accepted, and a local arm is only reported as armed when one was. A tombstoned id (its steer drained at an ordinary boundary mid-request) no longer answers `preempt: true` for an interrupt that cannot happen. - The cancel disarm is awaited. A dropped clear is worse than a dropped arm: the owner keeps a level-triggered request for a steer that no longer exists, seals its next chunk and truncates an unrelated answer. The boundary drain's own call stays non-blocking — there the owner is local, so the disarm is already effective and awaiting the informational publish would only delay injection. - Subscriber count is NOT read as proof of owner receipt: the count includes this replica's own facade subscription. A successful publish reports armed, a rejected one does not. Documented rather than papered over — see the acknowledgement-semantics note on the PR. CI regressions from round 5, all mine: - `registerPreemptSubscription` was AWAITED at both runtime-init sites, so job creation blocked on a second Redis channel subscription and hung when that subscribe was slow. Abort is awaited because a missed abort strands a run; a missed preempt only degrades that steer to the next tool boundary, so it now registers without gating createJob. - Two api specs mocked `@librechat/api` without the newly imported `isSteerPreemptSupported`, so the call threw before createJob; and one exact-match assertion needed the new `preemptCapable` metadata field. - My own Redis integration spec asserted arm-before-clear ordering, which two publishes carry no guarantee of — the receiving tombstone exists precisely because of that. Now asserts delivery and payload fidelity, order-independent. 158 packages/api specs, 27 api specs green. * 🧭 fix: Codex round 7 — settle the acknowledgement semantics (Option A) Round 7's second finding is the incoherence I flagged on the PR: the route persisted `preempt: true` on the durable queue item while returning `preempt: false` when delivery could not be confirmed. Those two then disagreed, and `rearmQueuedPreempts` trusts the DURABLE one — so a resumed owner would honour an interrupt the client had explicitly been told degraded to ordinary steering. Rather than patch the disagreement, this settles the meaning: `preempt` in the 202 means "queued as an interrupt request", NOT "a seal is guaranteed". It mirrors `SteerQueueItem.preempt` exactly, so the response, the durable record, and the resume-time re-arm can never disagree. The gates that ARE knowable stay — the owner's recorded capability and a successful enqueue. Everything past that degrades to the documented fallback of injecting at the next tool boundary. A route cannot synchronously know whether another replica will seal: proving it needs a correlated request/response over pub-sub, and even that only proves the owner heard, not that it is still streaming when the arm lands. Four rounds of tightening this boolean each surfaced a narrower case; the sequence does not converge, so the invariant is now "the flag describes the durable decision" and an unconfirmed arm logs a warning instead of rewriting the answer. Also from this round: a failed disarm publish is retried once and its outcome reported. `handleSteerCancel` keeps `removed: true` — the steer really did leave the queue, and saying otherwise would make the client re-show a chip for a steer that can never arrive — and adds `disarmed: false` so the residual risk is visible rather than swallowed. Damage stays bounded regardless: the empty-boundary self-clear disarms the generation after a single seal. Tests: +1 pinning the response/durable-flag invariant. 159 packages/api specs green. * 🧹 fix: Codex round 8 — remove the unverifiable disarm signal Round 8 found the same over-promise on the disarm side that round 7 corrected on the arm side, so this applies the same answer rather than patching around it. The `disarmed: false` field added in round 7 was both unreliable and unused: a resolved publish is not proof the owner heard it (the delivery count includes this replica's own facade subscription), and it was never threaded into `CancelSteerResponse` or read by any client. A signal that claims a certainty the transport cannot provide is worse than no signal — it invites callers to trust it. Removed from the response. The retry stays, because it genuinely reduces the failure rate, and `noteSteersRemoved` still returns whether the publish succeeded FOR LOGGING, now documented explicitly as "published without error", not "the owner disarmed". Disarm is best effort with a bounded, self-healing failure: if the clear is lost the owner seals once, the empty-boundary self-clear disarms the generation, and the turn is persisted `unfinished: true` rather than silently truncated. Tightening that further needs a correlated request/response over pub-sub with a timeout — noted on the PR as the deliberate boundary of this design rather than an oversight. 130 packages/api steering specs green. * 🧽 fix: Codex round 9 — spend snapshot arms on nonempty drains too The round-6 scoping fix only cleared the pre-drain snapshot when the drain came back EMPTY. On a nonempty drain the `finally` cleared just the drained ids, so a stale arm — typically a cancel whose cross-replica clear was lost — survived the boundary. It would then immediately seal the continuation meant to answer the steer that had just been injected, and land on an empty boundary as `preempt_incomplete`: the interrupt appears to work, and the answer to it is truncated. A boundary that runs has spent its seal, so everything armed at snapshot time is spent whether or not it came back from the drain. The `finally` now clears the union of the snapshot and the drained ids. Arms that land AFTER the snapshot are still spared — their queue items are live and uninjected, which is the property round 6 added. Also fixes an api-workspace CI failure of mine: `resume.spec.js` stubs `GenerationJobManager` wholesale, and the round-3/4 resume work added two calls (`updateMetadata`, `rearmQueuedPreempts`) the stub did not define, so 34 specs threw. Stub extended. Tests: +2 (a nonempty drain clears a stale snapshot arm; a nonempty drain spares an arm that landed mid-drain). Counterfactually verified — the stale-arm spec fails against the unfixed drain. 132 packages/api specs, 60 resume specs green. * fix: never let a failed preempt subscription reject into the void registerPreemptSubscription is called detached at both sites, so a rejected Redis SUBSCRIBE became an unhandled rejection — process-fatal under Node's default --unhandled-rejections=throw. The comment already promised this path merely degrades steering; it now does. Swallowed and logged inside the registration rather than at each call site, so a future third caller cannot reintroduce the trap. Losing the channel costs this generation's cross-replica preempts, not the server: same-replica arming is runtime state and still works, and remote arms fall back to the next tool boundary. Verified counterfactually — the new spec surfaces SUBSCRIBE failed as an unhandled rejection against the unfixed registration. * docs: state the real blast radius of a failed preempt subscription LibreChat's own entrypoints install a global unhandledRejection handler that logs and keeps serving, so the escaping rejection this guards was never fatal to this server — only to another consumer of @librechat/api that installs no handler. The fix stands either way; the comment just should not overstate what it prevents. * test: cover the cross-replica preempt hop with two manager instances Every other preempt test runs against a single manager, so the hop that actually carries an interrupt in production had no coverage: the steer POST lands on whichever replica the balancer picks, which is usually not the one generating. Non-owner publishes, owner arms, owner's level-triggered poll flips — none of that was exercised end to end. Two GenerationJobManagerClass instances are a faithful replica pair here. runtimeState and ownedJobs are private instance fields, there is no module-level mutable state between them, and createStreamServices duplicates a dedicated subscriber connection per call, so separate OS processes would exercise the same objects over the same Redis. Both assertions verified counterfactually against real Redis: - Deleting the preemptCapable deserialization in RedisJobStore fails this with 'Expected: true, Received: undefined' — the exact P1 that shipped past every in-memory test and would have made the feature a silent no-op on every Redis deployment. - Dropping the non-owner arm publish fails it with 'Received: false'. * test: remove the fixed sleeps and vacuity from the cross-replica preempt test Codex round 11, both findings, both on the test I added last commit. P2 — the 300ms waits were load-bearing. Redis pub/sub never replays and the owner's SUBSCRIBE is detached, so on a slow CI worker the publish could land before anyone was listening and the test would fail against correct code. Now it republishes until the owner's state converges, which is safe because arms and clears are idempotent set writes keyed by steerId. Side effect: the tests got ~10x faster (85ms/57ms vs 929ms/606ms) since they finish on delivery rather than on a timer. P3 — afterEach destroyed only the transports, leaving each manager alive in its own cleanup-interval closure, still working against a dead transport. Now tracks the managers and awaits destroy(), which disposes the job store and its timer too. Matches how the rest of this file cleans up. Fixing the sleeps exposed a third problem codex did not flag: the stale-arm test could pass vacuously, because an undelivered arm and a fenced one look identical. It now brackets the stale publish between two control arms — the first proves the owner is listening before the stale one is sent, the second proves it has had its chance to arrive. Verified counterfactually against real Redis, and stable over 5 runs: - dropping the preemptCapable deserialization fails with 'Received: undefined' - dropping the non-owner arm publish times out both tests - removing the generation fence fails the stale test with ["control-before", "steer-stale", "control-after"] — which also confirms the bracketing orders as intended rather than by luck * fix: gate interrupt on the OWNER's capability alone, not the route's Codex round 12. The comment above this gate already said 'the OWNER's recorded capability, not this replica's probe' — and then the code ANDed in isSteerPreemptSupported(), which is exactly this replica's probe. The contradiction dates to the original commit; round 6 made the gate owner-scoped and wrote that comment without removing the local conjunct. The route never seals. It enqueues and publishes an arm, neither of which touches the SDK, so during a rolling deploy a steer landing on an un-upgraded replica silently lost its interrupt even though the owner could seal. When the route IS the owner the probe is redundant anyway: the flag it would consult is the one this process wrote at createJob. The real degradation path is unchanged and still tested — an owner that recorded no capability relabels to an ordinary steer. The test that pinned the local probe asserted an impossible same-replica state (capable metadata plus an incapable local SDK, when the metadata is written from that probe); it now pins the mixed-SDK direction instead, and fails with 'Expected: true, Received: false' if the probe is put back. * fix: reconcile arms at handover, and stop holding the 202 on a publish Codex round 13, two of three findings. P2 — rearmQueuedPreempts only ever ADDED. A replica that merely read the job still installs a facade runtime and subscribes, so it can accept an arm and then miss the best-effort clear that follows the drain. HITL resume promotes that facade to owner, the union keeps the orphan, and the first resumed stream seals on a steer no longer in the queue, drains nothing, and truncates the resumed answer as preempt_incomplete. acquireResumedJobOwnership only sets ownedJobs, so nothing else was clearing it. The durable queue is the sole authority at a handover: arms it does not back are now disarmed and tombstoned, so an in-flight publish cannot revive them either. Worth recording that my own independent review raised this and my verifier refuted it. Codex found it separately; two reviewers converging should have outweighed one refutation. P2 — the route awaited the arm publish before answering. The 202 reports capability, not delivery, so the await could not change the response; it only exposed the caller to Redis latency after the queue item was already durable. A client that times out and retries mints a second steer while the first stays queued, injecting the same instruction twice, whereas a lost publish merely takes the tool-boundary fallback. Detached, with both outcomes logged. All three tests verified counterfactually: union-only rearm fails the two new handover specs, and re-awaiting the publish hangs the stalled-publish spec until jest kills it. * fix: snapshot arms before reading the queue at handover Codex round 14 — a regression from my own round-13 fix, and a worse failure than the one it corrected. Round 13 read the durable queue first, then tombstoned any armed id the snapshot did not back. But approvals.resolve reopens steering before reconciliation runs, so another replica can commit a preempt steer and publish its arm while the peek is in flight. That arm is then present locally but absent from a snapshot taken before the steer existed, so a LIVE interrupt the route already acknowledged got dropped — and tombstoned, which blocks the re-arm, making it unrecoverable rather than merely late. Fixed by inverting the two reads rather than by locking or paying a second round trip. A steer is durably enqueued BEFORE its arm is published, so any id in an arms-first snapshot was already queued when it was armed, and the later peek must observe it unless it has since drained — which is exactly the orphan this reconciliation exists to drop. Arms landing after the snapshot are simply not candidates. Also re-checks runtime identity across the await, since the generation can be replaced while the queue read is in flight. New spec injects a steer + arm during the peek and verifies it survives; against the round-13 ordering it fails with Received array: []. * fix: bound the cancel disarm wait and fence enqueue to its generation Codex round 15. P2 — the cancel awaited its disarm publish unbounded. ioredis queues commands during an outage rather than rejecting, so that await could hang for the length of the outage with the steer ALREADY durably cancelled; a client that gives up then treats the cancel as failed and restores a chip for a steer that can never produce an applied event. Every successful cancel publishes, so ordinary steers were exposed too, not only preemptive ones. Now bounded at 1s, with the publish continuing behind it — its retry and logging are unchanged, it is just no longer in front of the response. This is the sibling of round 13's arm-publish finding; I fixed one path and left this one. P3 — enqueue was not fenced to the generation the capability decision was made against. The access checks, file resolution and owner re-read are all awaits, so the run can be replaced before the enqueue: the item then lands on the REPLACEMENT queue while the durable preempt flag and the arm still name the previous epoch, the arm is fenced out at the owner, and the 202 promises an interrupt that cannot happen. enqueueSteer now takes an expected generation, mirroring drain/peek, and the Redis path enforces it inside STEER_ENQUEUE_LUA so the check is atomic with the push rather than racing it. All three new specs verified counterfactually, including the Lua guard against real Redis (removing it returns 1 where -1 is required). * fix: fence the steer to its authorized generation, bound resume setup, keep preempt when Redis parks Codex round 16, all three findings. P2 — round 15 fenced the enqueue to owner.createdAt, the RE-READ job. Every guard above it (ownership, tenant, paused-state, agent ACL) ran against the job read at the top, so if the run was replaced during those awaits the fence happily accepted the steer into a generation the request was never authorized against, carrying the wrong agent's metadata. Now rejects on any mismatch between the validated job and the re-read. P2 — resume awaited its steering bookkeeping unbounded, after approvals.resolve had consumed the action and flipped the job to running, and outside the resume lifecycle's own try/finally. .catch does not fire on a promise that never settles, which is what ioredis produces during an outage, so the client times out, its retry gets a 409 for a spent action, and no cleanup runs. Bounded at 1s with the writes finishing in the background. P3 — Redis parks leftover steers inside its terminal-transition Lua, which projects item fields one by one, so preempt was silently dropped and a steer recovered from /chat/status lost its interrupting label. Added to both projections. All three verified counterfactually, two against real Redis. Worth recording that my first version of the generation-mismatch test was VACUOUS — it faked a createdAt matching no live job, so the round-15 enqueue fence rejected it for the wrong reason and the test passed with the guard removed. Rewritten to replace the run for real; it now fails with 'Expected 404, Received 202'.	2026-07-30 12:36:13 -04:00
Danny Avila	e8a943c7e8	📦 chore: bump @librechat/agents to v3.3.8 (#14525) ... Activates activity-label continuity end to end. The host side landed with the activity-groups feature (#14391) — the per-run accumulator that reads committed headers at request-build time, the `previousLabels` payload field, resume seeding, and the bridge passthrough — but the SDK had no field to receive them, so the traced generation path ignored the context and only the direct fallback rendered it. v3.3.8 carries danny-avila/agents#356, which adds `previousLabels` to `RunActivityLabelOptions` and renders it as the label prompt's first section (capped at 3, each entry whitespace-collapsed and clipped at 200 chars so one malformed header cannot forge prompt sections or inflate every later request in the run). Effect: consecutive same-activity batches now extend the run's story instead of restating a line already on screen, and setup batches stop being labeled with conclusions their tools had not yet established. Also included between v3.3.7 and v3.3.8: danny-avila/agents#354, which anchors summary coverage to a source message id. Lockfile carries no transitive churn — 3.3.8's dependency tree is identical to 3.3.7's. Verified against the installed package: 73 activityLabels tests and 218 api agents-controller tests pass, `tsc --noEmit` clean on packages/api, and the published build renders the capped, sanitized header section (oversized labels clipped, embedded newlines flattened to inert text).	2026-07-30 08:23:09 -04:00
Dustin Healy	af795be0c2	🪢 feat: Langfuse Fanout Connection Setting (#14108) ... * feat: encrypt tenant Langfuse secret in admin config Add generic per-field secret encryption to the admin config layer: registered secret paths (langfuse.secretKey) are encrypted with encryptV3 on write and a non-secret fingerprint companion is stored. Admin config reads (base + per principal) redact registered secrets so they are never returned; the fingerprint is kept so the UI can show which key is configured. The Langfuse fanout read path decrypts the tenant secret before export. Adds secretKeyFingerprint to langfuseConfigSchema and tests for the encrypt/redact policy. * fix(api): secure admin config secret handling * fix(api): preserve encrypted langfuse config secrets * fix(api): couple config secret fingerprint deletion * fix(api): read langfuse fanout collector url from env * fix(api): display langfuse secret key hint * fix(api): remove langfuse secret fingerprint breadcrumbs * fix(api): use langfuse destination keys for tenant config * fix(api): remove langfuse config compatibility fallbacks * refactor(api): simplify langfuse secret helpers * refactor(api): simplify langfuse config secret handling * feat: in-app Langfuse connection settings panel Add a discoverable, admin-gated Langfuse connection panel inside LibreChat Settings (Dify-style): enable toggle, host, public key, masked write-only secret, configured-key fingerprint, and a test-connection action. Backed by a dedicated /api/admin/langfuse/connection endpoint that encrypts the secret at rest, returns metadata plus fingerprint on read, and validates credentials. Builds on the per-field encryption and fanout decrypt from the langfuse-config-encryption branch. * refactor: align Langfuse secret field to CustomUserVars pattern Use the established SecretInput plus Set/Unset state pill (com_ui_set/com_ui_unset) from the MCP CustomUserVars UI for the saved-secret state, instead of a bespoke masked input. * fix: drop em dash from saved-secret placeholder * feat: show loading state on Langfuse test connection button * feat: gate in-app Langfuse settings on fanout config and admin role * test: align Langfuse connection spec with SecretInput refactor * feat(langfuse): refine tenant connection controls * fix(admin): refine Langfuse connection verification * fix(langfuse): refine tenant connection settings * fix(langfuse): simplify export enablement controls * fix(langfuse): validate tenant export configuration * fix(langfuse): align startup fanout gate * fix(admin): time out Langfuse verification * fix(ui): rename Langfuse connection setting * fix(admin): enforce Langfuse config capability * feat(langfuse): require explicit tenant export activation * feat(langfuse): support single-tenant connection settings * fix(i18n): remove obsolete integrations label * fix(langfuse): authenticate ingestion verification * fix(langfuse): validate public key independently * fix(langfuse): localize connection errors * perf(config): skip Langfuse checks for non-admins * fix(langfuse): preserve trace sampling for feedback * test(langfuse): fix feedback sampling fixture * fix(langfuse): align secret preview field * fix(langfuse): harden connection settings state * fix(langfuse): preserve trace destination state * fix(langfuse): enforce tenant-wide routing invariants * fix(langfuse): preserve verified connection invariants * fix(langfuse): preserve stable project identity * fix(langfuse): warm project identity asynchronously --------- Co-authored-by: Ravi Kumar L <ravi.lazar@clickhouse.com> Co-authored-by: Danny Avila <danny@librechat.ai>	2026-07-29 18:33:10 -04:00
Danny Avila	cc813f430e	🎯 feat: Tool Intent Label Capability (tool_intents) (#14499) ... * 🎯 feat: Tool Intent Label Capability (tool_intents) Adds the fourth member of the per-tool capability family (defer_loading, allowed_callers, run_in_background): an admin capability AgentCapabilities.tool_intents plus a per-tool tool_options[name].describe_intent flag. Opted-in tools get an optional intent string injected as the FIRST property of their schema — one model-authored sentence per call, streamed to the client as the call's live status label (args already reach the client verbatim, so no new event plumbing). Native host tools (web_search, create_file/edit_file, set_memory/delete_memory, ask_user_question) default on while the capability is enabled; explicit false opts out. SDK-native intent schemas (@librechat/agents coding suite) are recognized and left alone. - packages/api/src/agents/intent.ts: structural sibling of background.ts — first-key non-mutating injection with registry parity (covers deferred/tool_search discovery), eligibility and PTC-only skips, arg read/strip helpers, self-spawn strip for defs and registry, ephemeral/model-spec synthesis with a tool_options merge so the background and intent toggles compose. - handlers.ts: intent runs BEFORE background injection so the label stays the first streamed key when a tool carries both (pinned by test); the arg is stripped before invocation unless the tool's own schema declares it, on both the foreground and background-dispatch paths; PTC target schemas are sanitized like background's. - Capability plumbing through all four routes (endpoint initialize, openai + responses controllers, the exported OpenAI-compatible service) plus handoff discovery and added-convo agents, and the intentToolNames execution channel via configurable. - describe_intent on toolOptionsSchema (all three written-out Zod annotations), ToolOptions, TEphemeralAgent, TModelSpec (+ zod), and data-schemas doc comments (tool_options is Mixed — no migration). - intent.spec.ts: 28 tests cloned from background.spec.ts structure, including the intent+background key-order composition. * 🧯 fix: Codex Review — Opt-Out Strips SDK-Native Intent, Skip mcp_all Placeholders - An explicit describe_intent: false now REMOVES an SDK-native intent property from the definition and registry entry, so the per-tool opt-out actually disables the arg's token cost for tools like web_search that carry the schema natively (SDK bodies tolerate its absence). Previously the early return left the property in place. - synthesizeIntentToolOptions skips lazily-expanded mcp_all placeholders instead of recording options under names that applyIntentLabels' exact-name matching can never match, and documents the limitation (parity with synthesizeBackgroundToolOptions). The P1 about the client not rendering the label is the documented slicing: the UI streaming-label PR follows once #14391's ToolCallGroup changes merge — args already reach the client, so that slice is purely rendering. * 🧯 fix: Codex Re-Review — Label Marker Guard, Capability Kill Switch, Late Defs, Service Threading - removeIntentParam is now marker-guarded (the label contract's opening instruction discriminates it), so an MCP/action tool's own business `intent` parameter is never stripped by an opt-out or the disabled path — previously an explicit false could remove a real, possibly required argument. - New sanitizeIntentLabels pass runs AFTER every registration step (the skill catalog appends its SDK definition post-injection): with tool_intents disabled it strips SDK-native intent labels from all definitions and registry entries, making the capability a real kill switch over their token cost; with it enabled it enforces explicit per-tool opt-outs on late-registered definitions. - ask_user_question removed from the native default-on set: its graph tool is rebuilt in run.ts from its own Zod schema (also the HITL card's wire shape), so definition-level injection never reached the model. Its intent support lands with the HITL slice, which threads the label into the interrupt payload deliberately. - The exported OpenAI-compatible service now threads intentToolNames into the run configurable, so the executor's PTC path can strip host-injected intent schemas on that route like the in-repo controllers do. * 🧯 fix: Codex Round 2 — Post-Skill Injection, PTC Native Strip, Service Boundary, Honest Docs - Intent injection now runs LAST in initializeAgent, after the skill catalog — which both appends its own definition and REPLACES upgraded ones (skill-aware read_file), clobbering an earlier injection while intentToolNames still listed the tool. Injection PREPENDS while background APPENDS, so intent stays the first schema property under the new ordering (pinned by a reverse-order composition test). - The PTC target-schema strip is now marker-guarded strip-ALL: SDK- native intent labels (which are deliberately never in intentToolNames) are removed from sandbox-advertised schemas alongside host-injected ones; business intent params survive. - toolIntentsAvailable on the exported service documents the loader boundary: a custom LoadToolsFn returning only structured instances bypasses definition/registry injection and sanitize by construction. - librechat.example.yaml describes tool_intents as backend groundwork with UI rendering in an upcoming release rather than promising a live label today. * 📦 chore: bump `@librechat/agents` to v3.3.6 Brings in the SDK half of tool intent labels (danny-avila/agents#347, #349): intent-first schemas on the coding suite across all three engines, plus web_search / subagent / skill / tool_search, and the outcome / outcome_patch result channel. Activates three host paths that were inert while no SDK tool shipped an `intent` property — verified against the real 3.3.6 schemas: - capability OFF now strips SDK-native labels (a real admin kill switch) - explicit `describe_intent: false` removes them per tool - host injection stays idempotent against an SDK schema, keeping `intent` first and never double-injecting * 🔬 test: Real-Provider Verification for Tool Intent Labels Adds the live check the unit tests structurally cannot perform: whether a real model actually authors the injected arg, places it FIRST, and gives sibling calls to one tool distinct labels. Reuses the existing real-provider harness (in-memory Mongo, seeded user, credential neutralizer) and the existing stdio MCP fixture as a genuine tool, so no external service is involved. - e2e/config/librechat.real.yaml: adds the e2e-memory MCP server and the tool_intents capability, giving the real model something to call. The sibling spec asserts only relative token growth, so the extra schemas do not perturb it. - e2e/playwright.config.real.ts: optional Langfuse passthrough. The LANGFUSE_* keys match the credential-neutralizer pattern and were being blanked before the server booted; they are preserved explicitly, read from the invoking environment only, and never written to the generated config. - e2e/specs/real/tool-intents.spec.ts: two facts stored in one turn, both through the same tool, asserting intent is the first key of each call and that the two labels differ. Args are read from persistence rather than the DOM deliberately — no UI renders the label yet, and persistence is what a reloaded conversation and the trace both read. First run against claude-haiku-4-5 produced 'Recording the location of the OAuth callback router' and 'Recording the location of the MCP connection pool configuration' — distinct, first-position, no tool name. Also updates tool-intent-spec.md: records the 3.3.7 removal of the tense verb map with the evidence that motivated it, the trimmed description and the marker's role as an API, and a new mandatory requirement that client-side label rendering be gated on a server-sent signal rather than the presence of an intent key (a tool's own business 'intent' parameter would otherwise render as a status label). * 📦 chore: bump `@librechat/agents` to v3.3.7 and dedupe the intent contract Picks up danny-avila/agents#353: the tense verb map is gone (a bare intent now displays unchanged, with completion carried by UI state), the model-facing description is trimmed 502 → 289 chars, and both the marker and the description are exported. Stops redeclaring the SDK contract here: - INTENT_LABEL_MARKER is imported instead of duplicated as a string literal. Every removal path in this module keys on it, and a local copy that drifted from the SDK's would make them all stop recognizing SDK-native labels — failing OPEN, with labels left in schemas and per-tool opt-outs silently inert. - INTENT_DESCRIPTION is imported too, so host-injected tools and SDK-native tools present the model with one identical instruction. Keeping the old local copy would also have meant host-injected tools still paying ~126 tokens per schema while SDK tools paid ~72. Verified live against real Anthropic after the trim: two sibling calls to one MCP tool produced 'Storing the OAuth callback router file location' and 'Storing the MCP connection pool configuration file location' — first-position and distinct, so the shorter description holds compliance.	2026-07-29 15:40:52 -04:00
Danny Avila	becfc5a373	🗝️ feat: Custom Endpoint API Key Encryption via Array Secret Registry (#14510) ... Extend the admin-config secret registry (#14509) with array-item secrets, registering endpoints.custom[*].apiKey: encryptV3 at rest with apiKeyPreview companions, redaction on admin reads (plaintext-legacy included, with verbatim-name omit-to-keep preservation, duplicate-identity skip, and legacy-plaintext self-healing), passthrough for user_provided/${ENV} refs, strict-payload runtime decryption at getCustomEndpointConfig, the custom model fetch, and the provider fallback, and rejection of named/indexed/ positional writes beneath the protected array path.	2026-07-29 15:37:46 -04:00
Danny Avila	7b6900d556	🏷️ feat: Activity Groups With Fast-Model Headers (#14391) ... * ✨ feat: Activity Groups with Fast-Model Labels Groups each contiguous block of reasoning + tool calls into a collapsible unit headed by a fast-model label (claude.ai-style hierarchy), off the critical path: a PostToolBatch hook claims a live content slot at the batch boundary (steering index-offset pattern), renders a deterministic counts phrase instantly, and swaps in the generated label ~1s later while the next model call streams. Labels are UI-only — stripped before the SDK formatter and skipped in the legacy formatter — and reach live clients via a dedicated on_activity_label SSE event (live/replay/pending paths). Grouping preserves legacy rendering byte-for-byte when no label part is present. Generation bridges to Run.generateActivityLabel() when the SDK ships it (session-grouped Langfuse tracing); falls back to a direct call today. Env-gated: ACTIVITY_LABELS_POC=true, ACTIVITY_LABEL_MODEL. * 🧷 fix: Address Codex and Copilot Review Findings for Activity Labels - Settle in-flight label fills (bounded 3s) before finalization on both the main and resume paths, so a label resolving during the final batch still reaches the durable log and saved message. - Overlay on_activity_label chunks in RedisJobStore content reconstruction (splice path last-wins per index; replay path chronological overwrite), matching steer handling. - Wire activity labels into the HITL resume createRun so post-resume batches keep claiming slots. - Guard against out-of-order publishes: fill() awaits the claim emit before emitting the resolved label, and the client applier ignores a stale pending placeholder once a resolved label is present. - Stamp the batch's groupId onto label parts so parallel-column runs place them inside their group instead of filtering them out. - Localize the counts fallback phrase (10 keys, singular/plural) through useLocalize across chat rendering and exports. - Type the hook with Providers/ClientOptions instead of stringly types; drop the unknown cast in the spec; add a dedicated rAF retry ref for label events with effect cleanup. * 🛡️ fix: Address Independent Review — Abort, Usage, Lane Context, Redis Test - Propagate the run abort signal into label generation (both wiring call sites; runtime combines host + dispatch signals with the timeout) so a user abort cancels in-flight label calls instead of paying to timeout. - Record label-call usage like titles: the SDK bridge aggregates via chainOptions callbacks, the fallback path via a per-generation callback factory; both feed recordCollectedUsage under context 'activity-label'. - Scope block-context capture: reasoning collection stops at the previous block's label part and filters by executingAgentId, so consecutive or parallel batches can no longer bleed another block's thinking into the payload; intent text still scans past labels (persists across batches). - Forward the effective charLimit to the SDK call so host and SDK prompts agree (SDK default aligned to 600 in agents#327). - Add a Redis integration test proving last-write-wins reconstruction of on_activity_label chunks per claimed index. - Rebased onto main: only the two activity commits replay (the nine steering commits belonged to the old base branch), zero conflicts, steering suites green. * 📐 refactor: Move Activity-Label Wiring to TypeScript, Address Codex Round 2 - [P1] Slot claiming, lane stamping, emit ordering, context capture, and settle tracking now live in packages/api (createActivityLabelWiring + captureActivityBlockContext); client.js is a thin closure wrapper. - Register the activity-label hook BEFORE the steer drain so a steer draining at the same batch boundary cannot flush the tool block and orphan the label outside its group. - Resolve request-based header placeholders in resolveActivityLabelLLM (titleConvo parity) so metadata-keyed proxies work on label calls. - Trim labels centrally before filling so whitespace-only output from either generation path keeps the deterministic counts fallback. * 🧭 fix: Codex Round 3 — Capture Order, Shared Strip, Token Estimator, Hide Filter - Capture block context BEFORE pushing the label part: the scan stops at ACTIVITY_LABEL parts, so post-push capture hit the just-inserted label and silently collected no reasoning excerpts (regression test added). - Share stripActivityLabelParts from packages/api and apply it in the Responses and OpenAI-compatible controllers, closing the replay leak for entry points still running SDKs without the formatter skip. - Exclude activity_label parts from the fallback response-token estimator (UI-only parts must not inflate no-usage provider billing). - Keep label parts explicitly under hide_sequential_outputs — they summarize exactly the outputs that mode hides. * 🔁 fix: Codex Round 4 — Resume Gap, Delta Flush, Agent-Scoped Intent, Token Counter - Synthesize on_activity_label events for labels claimed or filled in the snapshot→subscribe window (the publish is fire-and-forget, so Redis-mode reconnects missed them). Feature-gated so the default path adds no content re-read; the client applier already ignores duplicates. - Flush queued deltas before applying a label part, matching the pending- action and steer appliers — without it the handler read a stale message cache and syncStepMessage pushed a pre-delta copy back. - Skip another agent's tail text when resolving intent, so parallel runs cannot seed a label prompt with a sibling agent's narration. - Exclude activity_label parts from countFormattedMessageTokens (the agent-path counter), not just the legacy BaseClient one. * 🏗️ refactor: Codex Round 5 — Extract Label Host Logic, Report Usage, Icon Strip - Move provider/model resolution, usage-metadata mapping, and the settle loop into packages/api (activityLabels/host.ts); client.js keeps only thin delegations, per the repo's TypeScript-implementation convention. - Fold label usage into the response rollup with an 'activity-label' tag (subagent precedent) so metadata.usage and the live cost gauge account for it; tagged, so it stays out of PRIMARY usage/context pairing. - Narrow tool metadata once in ToolCallGroup so THINK parts in a labeled block no longer render phantom generic icons in the stacked strip. - Import the activity-label helpers by deep path in GenerationJobManager: the package barrel now reaches provider-config/cache modules that import back into the stream layer, and the cycle broke suite loading. Declined: resetting steerOffsetState before HITL resume — resume builds a FRESH AgentClient via initializeClient (initialize.js:978), so the offset is already zero; the seed wrapper alone accounts for pre-pause parts. * 🚦 fix: Codex Round 6 — Stream Label Usage, Close Late Fills - Emit an on_token_usage chunk for label calls (sink push alone left the live session gauge blind); retained in pendingSubagentEmits so job cleanup cannot race the persist, tagged 'activity-label' as before. - Close the label scope when settle times out: the wiring gates fill() on isClosed and the client fires a label-scoped AbortController, so a straggling generation can neither mutate a saved response nor emit into a job whose runtime is gone. The controller also chains to the run signal, so a user abort still cancels label work. * 🩹 fix: Repair CI — Package Typecheck and Module Mocks Local runs covered the client tsconfig and jest, but never packages/api's own tsconfig, so nine type errors in the extracted host module shipped. - Type host.ts against the real contracts: ServerRequest, EndpointDbMethods, AppConfig from @librechat/data-schemas, IUser for createSafeUser, and a MaybeAzureConfig view for the azure instance-name probe and configuration. - Widen resolveConfigHeaders' llmConfig to Partial<RunLLMConfig>: it only reads the three provider header carriers, so auxiliary generations with a bare ClientOptions can resolve headers without assembling a run config. Type-only widening; every existing caller still satisfies it. - Add stripActivityLabelParts to the @librechat/api mock in the OpenAI and Responses controller specs — those mocks enumerate exports, so a new import read as undefined and threw before the assertions ran. - Use the real activity-label helpers in the ToolCallGroup spec's ~/utils mock; stubbing them out would hide the header logic under test. * ⚙️ feat: Configure Activity Labels via librechat.yaml, Drop Env Vars Replaces the ACTIVITY_LABELS_POC / ACTIVITY_LABEL_MODEL env gate with per-endpoint settings, following the title options convention rather than a top-level block — each endpoint picks its own cheap label model. - Add activity, activityModel, activityEndpoint, activityPrompt, activityMaxPerRun, and activityCharLimit to the endpoint schema, and to the endpoints.all pick list (enumerated, so 'all:' would otherwise drop them silently). - resolveActivityConfig reads them with title-style precedence: endpoints.all > named endpoint > custom endpoint config. - Model precedence is now activityModel > titleModel > the agent's model. activityEndpoint runs labels on another endpoint's credentials, with titleConvo's fallback-on-unknown-name behavior. - Thread activityPrompt/MaxPerRun/CharLimit through the wiring into the hook and the SDK bridge; they were hardcoded defaults. - The resume gap-repair gate keyed on the env var; it now keys on the snapshot actually containing label parts, so deployments without the feature still perform no extra content read. - Document the fields in librechat.example.yaml; add host.spec.ts covering precedence, custom-endpoint fallback, and opt-out. * 📝 refactor: Rename Enable Flag to activityLabel, Document Schema Inheritance - Rename the boolean from `activity` to `activityLabel`, matching the titleConvo/titleModel shape: a verb-object toggle whose prefix matches its modifiers (activityModel, activityPrompt, ...). `activity: true` alone read ambiguously — it could mean tracking or logging activity. - Document the two endpoint-schema inheritance paths, which behave oppositely and are ~900 lines apart: * `endpoints.all` omits from baseEndpointSchema, so new options are inherited automatically — nothing to maintain. * `azureEndpointSchema` enumerates via .pick(), so a new option is silently unavailable on Azure endpoints until listed there. The activity block now carries a pointer to the Azure caveat. * 🔍 fix: Address Codex Findings on the Config Rework - Pass the matched custom-endpoint config into the label gate. Custom endpoints live in the `endpoints.custom` ARRAY, so without it every custom endpoint resolved as disabled — including the example this PR added to librechat.example.yaml. - Give label usage a unique `runId:seq`. Label usage is billed but never appended to `collectedUsage`, so its length was static: every label event reused the last primary usage's pair and collided with itself, and the client dedupes on exactly that. - Attach `cost` to label usage when `interface.contextCost` is on; aggregateEmittedUsage treats coverage as all-or-nothing, so an event without it suppressed the whole response's cost. - Honor `activityPrompt` on the direct fallback path, not just the SDK bridge — it previously always used the built-in instruction. - Seed the per-response label cap from labels already on the response so a HITL resume cannot mint a fresh quota after every approval. - Reconcile label gaps on resume via a durable per-job `activityLabels` flag instead of probing the snapshot: the FIRST label of a run can be claimed inside the snapshot->subscribe window, which the old signal missed. The flag is read from a job record already fetched there, so runs without the feature still add no content read. - Auto-collapse labeled single-tool groups; one-call batches are common in agent runs and rendering them expanded defeats the grouping. * 🎯 fix: Correct Label Usage Seq, Cross-Endpoint Pricing, Close Scopes - Give label usage a NEGATIVE seq namespace. The previous fix was wrong: seq is a position in `collectedUsage` (push, then emit with the new length), so sink-length + array-length still lands on a real position — primary emits 1, the label computes 2, the next primary also emits 2. Labels have no position at all (billed separately, never appended), so they now occupy a namespace positional sequences cannot reach. The client key is a string used for Set membership, so the sign is inert. - Price cross-endpoint labels with the LABEL endpoint's token config: resolveActivityLabelModel now returns the resolved endpointTokenConfig, and both the streamed cost and recordCollectedUsage use it instead of the agent endpoint's rates. - Make close state per-wiring rather than per-client. A HITL resume rebuilds the wiring, and resetting a shared flag re-opened closures from the pre-pause segment whose provider call ignored the abort; settle now closes every retained scope, past generations included. * 🎯 fix: Make the Activity Header Say Something the Cards Cannot The header read "ran 1 command" next to a card already labeled "Code" — it restated the UI beneath it instead of adding to it. Two causes, both about content rather than timing: - A deterministic tool-type tally was the primary display and also fed the prompt, so the best case was a tally and the worst case was a tally dressed as prose. Removed from the metadata, the prompt, the part type, and the client. - The instruction only ever reached the fallback path. The wiring passed a prompt only when was configured, so the preferred SDK path silently used the published package default. The wiring now always supplies one and the hook forwards it on both paths. The register is rewritten around what the cards cannot show: past-tense git-commit-subject, leading with the distinctive noun, outcome over attempt, tool names and counts and arguments explicitly forbidden. The batch entries are labeled as reference material so the model stops transcribing them. Claiming a slot no longer emits. The slot still reserves its index so streamed parts never collide, but with nothing to say there is nothing to render: until a description exists the block looks exactly as it does without the feature. * 🧹 fix: Drop the Localize Hook Left Unused by the Counts Removal * ✅ test: Add Activity-Label e2e Coverage with a Recording Label Server Activity labels are the one model call a mock run does not already fake: fake-model.js swaps the GRAPH model via overrideTestModel, while run.generateActivityLabel() calls the endpoint resolved client options over HTTP. The custom endpoints already point baseURL at 127.0.0.1:8889, so serving that port exercises the real path with no production seam. fake-label-server.js answers it in both JSON and SSE form, records each prompt, and can inject blank/error responses. Recording is what lets the spec assert the CONTRACT rather than the rendering: that this repo register and the tool OUTPUTS actually reach the model. That is the bug class that produced unusable labels before, and rendered text looks identical whether or not the instruction arrived. Labels get a dedicated endpoint (Mock Provider E). A labeled block auto-collapses even at one tool call, which hides the tool cards other specs assert on -- enabling this on a shared endpoint broke steering.spec.ts. Provider D is the unlabeled control. Request-count assertions are scoped to a per-test token: a 5xx label response is retried by the provider client, and a retry can land after the next test has reset the server. * 🩹 fix: Address Review Findings on Activity-Label Indexing and Pricing Replay index (P1). Reserving the slot only in server memory left no event for it, so a cross-instance replay rebuilt content as [tool, hole, later], compacted the hole away, and the fill for the reserved index landed on the following part and overwrote it. The claim now publishes the empty, pending part so the index is real for every consumer, and fill publishes even when generation returned nothing so the client cannot stay pending. It stays invisible: an empty label still DELIMITS its batch in groupSequentialToolCalls but is not attached as the header, so grouping does not re-shuffle when the text lands and the block renders exactly as it does with the feature off. Edited-response index (P1). Edit-and-resubmit replays the kept prefix and the server indexes only new content, so run steps offset by that prefix. Labels are claimed in the same space and now take the identical shift; without it a label could land inside the prefix and overwrite it. Redis flag. deserializeJob never read activityLabels back, so every Redis reload left it undefined and resume skipped label gap reconciliation. Executing agent. RunActivityLabelOptions.agentId selects the executing agent tracing metadata AND its tool-output redaction policy; omitting it let a handoff be redacted under the default agent configuration. Label pricing. An undefined endpointTokenConfig is meaningful for a built-in label endpoint (priced from the shared table), so the nullish fallback billed those labels at a custom primary rates. Inherit only when the label runs on the agent own endpoint. HITL usage sequence. runId is the response message id and the counter was instance-local, so a resume restarted at -1 and the client runId:seq deduper discarded the post-approval label usage. Seeded past the labels already on the response. Also distinguishes "cannot serve" (undefined) from "no label" (null) in the SDK bridge, so a missing run falls back to the direct call instead of filling the slot empty. Version gating already happens at wiring time via the sdkCapable prototype probe. * 🩹 fix: Keep Unfilled Activity Labels Invisible and Unmask Endpoint Settings Follow-up review round. Publishing the reservation on every batch made two latent rendering paths reachable on every run, and both are fixed here. Empty labels no longer change grouping. The previous pass still formed a tool-group for a textless label, which wrapped even a single tool call and pulled THINK parts inside it — and since a reservation is published the moment each batch ends, that applied during every generation and permanently after a blank or failed fill. An empty label now flushes the legacy way instead: it still delimits its batch, but the block re-splits exactly as it renders with the feature off. Parallel lanes no longer show a blank line. Lanes render raw parts, so an unfilled label had nothing to draw; empty ones are dropped. Making labels act as collapsible headers inside lanes is still a separate gap. Edited responses no longer offset on resume. The sync replaces initialResponse.content with the server's aggregatedContent, which already contains the kept prefix AND everything generated since — so its length is not the prefix length, and indices reconciled from that snapshot are already absolute. Offsetting again pushed the label past its slot onto a later part. The shift now applies only to a fresh edited submission. Activity settings resolve field by field. Selecting one config object whole meant any endpoints.all block — even one carrying nothing but headers — shadowed the named or custom endpoint and silently disabled activity labels everywhere. Global still wins per field. Adds groupToolCalls coverage for the invisible-while-empty contract, which is the part most likely to regress: it is normal state on every run, not an edge case. * 🔒 fix: Scope Detached Label Writes to Their Generation Epoch Epoch scoping (P1). Label generation is detached and can outlive the generation that started it. emitChunk only proves that SOME runtime is current, not that the caller belongs to it, so an aborted generation's fill(null) -- and its usage event -- could be attributed to whichever generation replaced it, landing an index from the abandoned response on top of the new one. Because an empty label renders nothing, that overwrote content silently. emitChunk now takes an optional jobCreatedAt and drops the event when the runtime epoch differs, mirroring the existing setGraph/setContentParts convention, and both label emitters pass it. An abort now CLOSES the label scope instead of only cancelling the call: the rejected generation still runs its catch and calls fill(null), which would otherwise emit into a stream the next generation may already own. Edited-response indexing (P1). The previous pass skipped the prefix offset on resume, which was the wrong half of the problem: a sync replaces initialResponse.content with the server's aggregatedContent, which is completion-local, so after a reconnect its length is not the kept-prefix length and the offset is wrong -- but it is wrong for run steps in exactly the same way. Tool cards and the label that heads them must share one index space; a label shifting differently from its tools lands on another part. The label path now uses the identical expression as useStepHandler, with no resume special-case. Correcting the post-resume prefix length belongs in calculateContentIndex, where it fixes both at once. titleModel masking. The activity settings were made per-field last pass, but the titleModel fallback a few lines below still selected an entire config object, so a partial endpoints.all (for example one carrying only headers) hid a named endpoint's titleModel and quietly fell the label back to the main agent model. Both now read through one shared per-field helper. Resume reconciliation no longer depends solely on markActivityLabels, which is best-effort yet had come to gate correctness: a lost flag write silently dropped a label. The snapshot is consulted as a fallback. The exported host type for generateLabel now admits undefined, which is the documented "cannot serve, fall back to the direct call" signal the hook keys on -- distinct from null, meaning it ran and produced nothing. * 🧷 fix: Keep Group Identity Stable and Memoize Label Endpoint Resolution Group remount. Tool-group identity was keyed on the first part in the block. An activity label absorbs the block's leading THINK part the moment its text lands, so the key flipped from tool:<id> to fallback:<scope>:<idx> mid-run, remounting the group and discarding whatever the user had expanded. The key now scans for the first tool call, which does not move when the block re-forms. Label endpoint resolution is memoized per response. It reads provider config and can hit the database for user keys, yet nothing it depends on changes between batches of one run — and it ran twice per batch, once for generation and once for usage accounting. The promise is cached rather than the value so concurrent batches share a single in-flight resolution, and a rejection is evicted so one transient credential failure cannot disable labels for the rest of the response. * 🎯 fix: Offset Edited Resubmissions by a Prefix Length That Survives Resume The server indexes only NEW content for an edited resubmission, so the client offsets incoming indices by the prefix it retained. That prefix was read as initialResponse.content.length, which is correct only until a resume: the sync replaces that array with the server's completion-local snapshot, whose length is unrelated to the prefix. After a reconnect every offset was therefore wrong -- run steps and activity labels alike -- and could write over content the edit kept. For a label the symptom is worse than a bad position: the fill misses its own reservation, so the pending placeholder is never resolved. The prefix length is now captured when the submission is built, while initialResponse.content still IS the retained prefix, and carried on the submission as editPrefixLength. calculateContentIndex takes that length instead of deriving it from an array that a resume may have replaced, so run steps and labels share one index space by construction rather than by both happening to read the same field. Note the prefix is the FULL original content with the edited part substituted in place (useChatFunctions clones latestMessage.content and mutates one entry) -- it is not a slice, so the length cannot be inferred from editedContent.index. Group identity no longer changes when a label fills. Tool-group keys were derived from the first part in the block; an activity label absorbs the leading THINK part when its text lands, flipping the key mid-run and remounting the group, which discarded the user's expansion state. The key now scans for the first tool call, which does not move. Label endpoint resolution is memoized per response. It reads provider config and can hit the database for user keys, yet ran twice per batch -- once to generate, once for usage accounting -- while nothing it depends on changes within a run. The promise is cached so concurrent batches share one in-flight resolution, and rejections are evicted so a transient credential failure cannot disable labels for the rest of the response. The resume gap passes for steers and activity labels now share a single lazy content read instead of each issuing its own. The label pass stays gated on the run flag with a snapshot fallback: reconciling unconditionally would also close the residual first-label window, but it would bill a read to every resume of every run, including deployments with the feature off -- which the steer pass deliberately avoids. That residual requires a lost flag write, which shares fate with the content writes the labels live in. * 💵 fix: Bill Cross-Endpoint Labels at Their Own Rates recordCollectedUsage never accepted an endpointTokenConfig, so the value the activity-label caller passed was dropped and the balance transaction was written at the primary agent's rates. Only the UI cost honored the label endpoint, so a custom primary pointing activityEndpoint at another endpoint showed one price and charged another. The parameter is now accepted, and an explicit config wins outright over per-agent resolution: that map is keyed by AGENT, so it cannot describe usage that ran on a different endpoint. Group identity is stable for id-less tool calls too. The previous pass anchored the key to the first tool call ID; where a supported tool call carries no id the fallback still used the block's first part index, which shifts when a filled label absorbs the leading THINK part. The fallback now anchors to the first TOOL entry's index, so only a block containing no tool call at all keys off parts[0]. markActivityLabels is retried rather than fire-and-forget. It gates resume gap reconciliation and is a SEPARATE write from the durable label append, so a single lost write silently drops a label the content itself recorded. The earlier "shared fate with content writes" reasoning was wrong. One retry at run setup costs nothing and removes the only realistic way the gate goes stale, without billing a content read to every resume. * 🧮 fix: Stop Offsetting Once SYNC Drops the Edited Prefix The edit offset was applied unconditionally, but whether it is correct depends on which branch SYNC took. SYNC either preserves the content already loaded for the response -- which still contains the retained prefix, so the offset is required -- or replaces it with the server's aggregatedContent, which is completion-local and indexed from zero, after which any offset writes past the end of a now shorter array. That is why the two previous attempts each fixed half of it: skipping the offset on resume was right for the replace branch, applying it unconditionally was right for the preserve branch, and neither holds on its own. The offset now tracks the actual state of the rendered content. For an activity label the replace branch was worse than a bad position: the fill landed past its own reservation, so the pending placeholder was never resolved and the block kept its generic header for the rest of the run. Applied to run steps as well, not just labels. useStepHandler reads the prefix from the same submission and had the same unconditional offset, so after a mid-session resume of an edited response tool cards were misplaced too. Normalizing at the dispatch boundary keeps both in ONE index space by construction: a label that shifted differently from the tools it heads would land on another part. Note the reload path was already coherent -- useResumeOnLoad rebuilds the submission without editedContent or editPrefixLength, giving no offset against server-supplied content -- so only the mid-session SYNC path was inconsistent. * 🧾 fix: Keep Label Accounting Out of the Primary Usage Slot Label usage no longer owns getStreamUsage(). recordCollectedUsage assigned its result to this.usage unconditionally, so when the primary provider reported no usage metadata but the label provider did, BaseClient took the label's output tokens as the assistant response's authoritative count. The later primary call returns early on an empty collectedUsage and never replaced it, so the wrong value stood, the text-based fallback was skipped, and the real generation went unbilled. Secondary usage is still billed but no longer writes that slot. Cross-endpoint pricing keys off an explicit discriminator rather than the presence of a value. A built-in label endpoint prices from the shared table, so an undefined endpointTokenConfig is its MEANINGFUL value -- reading that as "no override" fell back to the primary's custom rates and restored the exact mismatch the previous pass set out to fix. The caller already knows whether the label ran elsewhere and now says so. markActivityLabels rejects on failure instead of swallowing it. The flag gates resume gap reconciliation and the caller retries it, but the internal catch resolved successfully and made that retry unreachable -- so the two changes cancelled out and a transient write failure still left the flag absent. Late label accounting is suppressed with the same gate as the late fill. A straggler that outlived the settle timeout still ran its finally block, so it charged the balance and appended to usageEmitSink after the response had passed its usage flush and metadata snapshot: a cost the user pays but is never shown. The cleared-prefix state is scoped to one generation. It was set on a resume SYNC that replaced the response and then never reset, so a later edited resubmission in the same mounted hook dispatched run steps and labels with no offset against content that still held its retained prefix. Reconnects pass isResume and keep the state; a new generation clears it. * 🔑 fix: Key Prefix State to the Stream and Honor current_model for Labels The cleared-prefix reset keyed on isResume, which skips exactly the case it was added for: a submission whose POST succeeded server-side but lost its response is retried, comes back resumed: true, and subscribes in resume mode even though it is a NEW generation. A previous generation's cleared state then survived into it, and incoming run steps and labels applied no offset against content that still held its retained prefix. The state is now keyed to the stream id, which changes with the generation and stays put across reconnects of one. activityModel now honors current_model. The options are documented as title-shaped and the titleModel fallback already excludes the sentinel, but the higher-precedence activity override passed the literal string through to getOptions and the provider, so an endpoint following that convention failed every label instead of using the agent model. * 🎯 fix: Key Prefix State to the Generation and Resolve the Run Model The cleared-prefix state was keyed to the stream id, which never changes within a conversation: request.js sets streamId = conversationId, so once a reconnect cleared the state every later edited resubmission in that conversation dispatched run steps and labels with no offset and could overwrite the prefix it retained. It is now keyed to the response message id, the only per-generation identity available here -- minted per submission and carried through a resume unchanged. That is the third identity tried for this state. isResume missed the deduplicated-retry path (a lost response returns resumed: true for a new generation); the stream id is conversation-scoped. The response id is the boundary that actually matches a generation. current_model labels now resolve the model the run is really using. initializeAgent merges the request's endpointOption override into model_parameters and the run gives it precedence, so preferring the saved agent.model could send labels to a different, potentially unavailable or more expensive model than the conversation is on. * 🆔 fix: Key Prefix State to the Submission and Keep the Origin Title Model Editing an assistant response reuses that response's messageId as editedMessageId, and useChatFunctions carries it onto initialResponse.messageId -- so re-editing the same response produced two generations with the same key and the cleared-prefix state survived between them, leaving run steps and labels with no offset against content the edit retained. Keyed now to clientRequestId, the per-submission uuid, which is minted fresh per edit attempt and forwarded unchanged on retries. That is the fourth key this state has had, and each earlier one failed at a real boundary: isResume missed the deduplicated-retry path, the stream id is the conversation id, and the response message id is reused across edits of one response. clientRequestId is the identity that actually means "this submission". The titleModel fallback is read from the ORIGINATING endpoint again, matching how titleConvo captures its config before switching credentials. Reading it after an activityEndpoint switch meant an OpenAI endpoint configured with titleModel claude-haiku and activityEndpoint anthropic fell through to the OpenAI run model and sent that name to Anthropic, failing every label. The destination endpoint supplies credentials, not the model choice. * 🧷 fix: Close the Remaining Edit, Epoch, and Scope Gaps for Labels SYNC clears the edit prefix on the new-row branch too. When a resumed edited submission cannot match an existing assistant row, that branch builds the response straight from the server's completion-local aggregatedContent, so it holds no retained prefix -- but the reset lived only in the matched branch, leaving later steps and labels adding an offset to indices that were already absolute. Label usage is keyed per GENERATION. Editing one assistant response reuses its responseMessageId while each fresh generation restarts activityLabelUsageSeq, so a second edit re-emitted the same runId:seq and the client discarded the newer usage while its balance transaction was still written. The key now carries jobCreatedAt, the run's own epoch: stable across reconnects and HITL resumes, distinct between generations. The scope is revalidated at commit time. Checking once before the await let a scope that closed mid-flight still charge the balance after finalization, while the matching fill saw the closed scope and dropped the label -- billed but never surfaced, the exact outcome the guard exists to prevent. The titleModel fallback no longer reaches the destination endpoint. With activityEndpoint set and no titleModel on the originating endpoint, it picked up the destination's, so changing only the credential target silently changed the model and its cost. Precedence is activityModel, then the originating endpoint's titleModel, then the run model; the destination supplies credentials only. * ✂️ refactor: Confine the Edit-Prefix Offset to Activity Labels useStepHandler is now byte-identical to dev again. The resume-aware prefix offset was applied there too, which was more correct in principle -- the post-resume prefix length is genuinely wrong for run steps as well -- but it changed index math that EVERY run step flows through, for every user, including everyone who never enables activityLabel. That shared correction needed five revisions in two days (isResume, the stream id, the response message id, clientRequestId, and the SYNC new-row branch), each passing the full suite and each failing at a boundary only review found. Carrying it inside an opt-in feature put every user behind logic with that track record. It belongs in its own change, with tests that construct the edit-plus-resume states none of the current suites reach. The offset now applies only where the label handler places its part, so this PR cannot alter rendering for anyone with the feature off. The known consequence is recorded in the description: with activity labels ENABLED, an edited response that reconnects mid-generation can place its label and its tool cards in different index spaces. That is a bug for opt-in users rather than a regression for everyone, and it disappears once the shared fix lands. submission.editPrefixLength stays: the label path still needs a prefix length that survives a SYNC replacing initialResponse.content. * 🧾 fix: Commit Labels Before Billing and Keep Blank Slots Invisible Round-nine review (all P2, feature-scoped): - Billing ordering (client.js:409, runtime.ts): usage accounting ran BEFORE the slot commit on both generation paths, so the settlement deadline could expire during the balance write — charged, then the fill dropped as out-of-scope: billed, never shown. `slot.fill` now resolves a commit flag, generators register their accounting via `deferUsage`, and the hook runs it only after a committed fill. - Scope gates (client.js:757): the direct-fallback `collect` omitted `scopeOpen`; both paths now gate on the OWNING wiring's scope, so a pre-pause straggler cannot bill because the resumed generation's scope is still open. - Blank-label grouping (groupToolCalls.ts:81): a blank slot forced a flush, splitting adjacent single-call batches into standalone cards where the feature-off path merges them. Blank labels now only mark the claim boundary — structurally invisible, while a later filled label still cannot claim an earlier batch. - Stale fill indices (wiring.ts:301): the skill-card unshift and the hide-sequential filter reshape contentParts before the finalization settle, so an in-flight fill emitted its claim-time index against a shifted array. Both completion paths now settle label fills before any post-run content reshaping (the finally settle stays as the error-path net; the second call sees an empty pending list). - Bounded serialization (runtime.ts:238): `JSON.stringify` fully materialized unbounded tool results to keep 200/600 chars per entry. A budget-bounded serializer stops at the limit (which also bounds cyclic values) and preserves the exact truncate-with-ellipsis output. Tests: fill/bill ordering + suppression on dropped fills (runtime.spec), blank-slot merging and claim boundaries (groupToolCalls.test), bounded serialization equivalence and giant-output truncation (runtime.spec). * 🧮 fix: Keep Deferred Label Billing Inside the Settle Window Self-review follow-up to the billing reorder: deferring usage until after the commit moved it PAST the fill's resolution, so a settle keyed on fills alone could let finalization flush the usage sink and snapshot metadata while the label's billing was still in flight — the usage row would silently miss the message rollup even on the happy path. The hook now reports its whole detached task (generate → fill → deferred usage) via a `trackTask` option, wired to the same settle tracker as the fills, so finalization waits for billing exactly as it did when accounting preceded the fill. The task never rejects. Pinned in runtime.spec: the tracked task resolves only after usage collection. * 🧰 fix: Harden Label Resolution, Output Bounds, and Cache Billing Round-ten review (all P2, feature-scoped); the sixth finding is the documented edited+reconnect index-space limitation, answered on-thread as deliberately out of scope for this PR. - Rejected-LLM memoization (runtime.ts): the hook cached a rejected `resolveLLM()` promise permanently, failing every later batch and silently defeating the host resolver's own rejected-cache eviction. The memo now evicts on rejection so the next batch retries. - `current_model` precedence (host.ts): an explicit `activityModel: current_model` resolved to `undefined` and then lost to a configured `titleModel`. The sentinel now resolves straight to the run model; the title fallback applies only when `activityModel` is absent. - Output bounds (runtime.ts): label text was persisted verbatim; a model ignoring the 4–9-word instruction (or steered by injection in untrusted tool output) could emit thousands of tokens duplicated through SSE, the chunk log, persistence, and the UI. `normalizeLabelOutput` keeps the first non-empty line, collapses whitespace, and hard-caps at 200 chars on both generation paths. - Cache-token billing (host.ts, client.js): the usage mapper dropped cache fields, vanishing Anthropic cache tokens from billing and charging OpenAI cache reads at the full input rate. The mapper now normalizes Anthropic/OpenAI/LangChain cache shapes into `input_token_details`, and the emit + cost path carries them with the label endpoint's `provider` (additive-provider adjustment). - Usage-type union (runs.ts): `TTokenUsageEvent.usage_type` now includes the emitted `activity-label` literal; the lone consumer keys on `usage_type != null`, so this is type-level completion. Tests: sentinel/title/explicit model precedence and all three cache shapes (host.spec), transient-resolution retry and output normalization with truncation (runtime.spec), the new usage literal (runs.spec). * 🪗 fix: Let Settled Labels Collapse Void Tools and Keep the Tail Cursor Round-eleven review (all P2, client-side). Two fixed; the other two findings restate documented Known limitations (edited+reconnect run-step index space; parallel-lane collapsible headers), answered on-thread. - Void-tool auto-collapse (ToolCallGroup.tsx): `allCompleted` keyed solely on output truthiness, so a tool that legitimately returns an empty string kept its labeled group expanded forever. A settled, filled label is itself a completion proof — the PostToolBatch claim only happens after every output in the batch returned — so it now satisfies `allCompleted`; pending labels keep the group live. - Trailing-reservation cursor (ContentParts.tsx): a blank label reservation at the content tail renders nothing but still counted as the last part, stripping the streaming cursor and last-item affordances from the last VISIBLE part until the next delta. `lastContentIdx` now walks back past empty label slots. Tests: labeled void-tool group auto-collapses, pending-label group stays expanded (ToolCallGroup.test). * 💳 fix: Price Label Cache Correctly, Honor endpoints.agents, Cancel Every Retry Round-twelve review: four fixed here; the remaining P1 (move the client.js bridge into packages/api) is an architecture call answered on-thread for the maintainer. - Provider on billed entries (client.js, P1): round ten added cache details to label usage entries but not `provider`, and `splitUsage` treats an unknown provider as additive — re-adding cache_read and cache_creation on top of an input count that already contains them, double-charging Anthropic/OpenAI cached label calls while the streamed cost (which carried the provider) disagreed. Every mapped entry now carries the label endpoint's provider. - endpoints.agents honored (host.ts, client.js): `initializeAgent` rewrites `agent.endpoint` to the backing provider, so activity settings under the PUBLIC `agents` endpoint — valid config, inherited by `agentsEndpointSchema` — were silently ignored. Field resolution is now `all` > public endpoint > backing provider/custom, applied to both the enable gate and the model/titleModel resolution. - E2E_LABEL_PORT reaches the YAML (playwright.config.mock.ts): an overridden port moved the fake label server and its health check but not the generated config's hard-coded 8889 baseURLs, so readiness passed while every label request targeted the wrong port. The override is now substituted into the generated copy. - Every retry frame cancelled (useResumableSSE.ts): concurrent label retry chains (reservation + fill per slot) overwrote one rAF handle, so cleanup cancelled only the newest chain; the rest ran up to 120 frames past unmount and could apply a stale label to a replacement generation reusing the same response id. Outstanding frame ids now live in a Set that cleanup drains. Tests: public-endpoint gate/precedence/all-above-public (host.spec). * 🖱️ fix: Keep the Last-Part Cursor in Parallel Lanes Too Round-thirteen review (single P2): `ParallelContentRenderer` computed `lastContentIdx` from the unfiltered array, so a trailing blank label reservation — filtered out of every lane — left NO rendered part carrying the last-part cursor and running-subagent affordances until the label filled. The sequential renderer's walk-back is extracted into a shared `lastVisibleContentIdx` helper (utils/activityLabels) used by both `ContentParts` and `ParallelContentRenderer`, so the two index spaces cannot drift again. Behavior pinned in activityLabels.spec: trailing blank skipped, consecutive blanks skipped, filled label counts, label-free content unchanged. * 🧹 chore: Alias the Retry-Frame Set for the Effect Cleanup Lint Rule * 📏 fix: Let activityCharLimit Reach Tool Inputs Round-fifteen review: `activityCharLimit` is documented as the per-entry truncation for tool input AND output, but `buildPrompt` hard-coded inputs at 200 characters — so raising the setting could never surface a distinguishing path, query, or operation that appears past the first 200 characters of a long argument. Inputs now truncate at the configured limit alongside outputs; the 200-char constant remains only for the intent line (renamed INTENT_CHAR_LIMIT to match). Config fidelity pinned in runtime.spec: a 400-char argument survives a 450 limit and truncates under a 50 limit. The round's other finding is the fifth restatement of the documented edited+reconnect index-space limitation, answered on-thread with the prior four cross-references. * 🤝 fix: No Labels for Pure Handoff Batches Round-sixteen review: a PostToolBatch containing only `transfer_to_*` calls claimed a label slot, but transfer parts are never groupable — the client flushed the handoff card standalone and the label orphaned into a stray line after it, restating what the card already says. Two-sided fix: - Hook (runtime.ts): a batch whose every entry is a transfer call claims nothing — no slot, no model call, no `maxPerRun` consumption. Mixed batches still label (the header describes the real work). - Renderer (groupToolCalls.ts): an orphan label whose `tool_call_ids` are all transfer calls is dropped instead of rendered standalone, covering content persisted before the hook-side skip. The round's two P1s are repeats answered on-thread: the packages/api extraction (maintainer-decided follow-up, recorded in the description) and the sixth restatement of the edited+reconnect index limitation. Tests: transfer-only batch claims nothing, mixed batch still claims (runtime.spec); transfer-only orphan label dropped, real-batch orphan label still renders (groupToolCalls.test). * 🎛️ fix: Sanitize Label Client Options and Bound the Batch Prompt Round-seventeen review: two fixed; the other two findings repeat the maintainer-decided packages/api extraction (follow-up) and the edited+reconnect index limitation (seventh instance), answered on-thread. - Primary-option strip (host.ts): the label client copied the resolved `llmConfig` wholesale, so an endpoint whose defaults enable extended thinking or carry model-specific output caps forwarded them to the (often cheaper) label model — unsupported options failed every label, and supported thinking spent real tokens and the settlement window on a 4–9 word header. The copy now strips `omitTitleOptions` keys and the `modelKwargs` output caps exactly like the title path, restoring the Anthropic `clientOptions` carrier by reference so proxy `defaultHeaders` still reach label requests. - Batch prompt budget (runtime.ts): per-entry truncation left the batch dimension unbounded — hundreds of parallel calls could build a prompt past the fast model's window. The entries section now has a total budget (8k chars, scaling with `activityCharLimit` so a raised limit still fits several entries); entries past it are skipped without paying their serialization cost, and the list notes how many were omitted. The first entry always renders in full. Tests: option strip with header-carrier survival (host.spec); giant batch bounded with omission marker, small batch untouched (runtime.spec). * 🛡️ fix: Keep SSRF Guards on Label Calls, Skip Mixed Handoff Batches Round-eighteen review: four fixed; the fifth repeats the maintainer-decided packages/api extraction (eighth instance), answered on-thread. - SSRF-safe carrier (host.ts, P1): the sanitize step restored the Anthropic `clientOptions` carrier only when `defaultHeaders` existed, but for user-provided base URLs `getLLMConfig` stores the guarded Undici dispatcher and `redirect: 'error'` there — dropping it reopened DNS-rebinding/redirect paths on label calls to user-controlled URLs. The carrier (client CONSTRUCTION options, not generation params) is now restored whenever present, same reference. - Primary maxTokens (host.ts): top-level `maxTokens` is not in `omitTitleOptions` and survived the strip; the title path deletes it explicitly, and a cap sized for the primary model can be rejected by the substitute. Deleted on the copy. - Bounded keys (runtime.ts): the object branch materialized every key via `Object.keys` and quoted oversized keys in full before the budget check. Enumeration is now lazy (`for..in` + own-property guard) and keys slice to the budget before quoting, like string values. - Mixed handoff batches (runtime.ts, groupToolCalls.ts): the client flushes the block at the transfer card, so a mixed batch's label orphaned exactly like a pure one. The hook now skips ANY batch containing a transfer call, and the renderer drops orphan labels covering one (legacy content). Tests: carrier survival without headers by same reference, maxTokens strip (host.spec); mixed batch claims nothing (runtime.spec); mixed orphan dropped, real-batch orphan kept (groupToolCalls.test). * 🧢 fix: Cap Label Generation, Order the Flag Persist, Detach Settled Listeners Round-nineteen review: three fixed; the fourth is the ninth instance of the edited+reconnect index limitation, answered on-thread. - Generation cap (host.ts): stripping the primary output caps left label calls with NO cap at all — `normalizeLabelOutput` bounds what persists, not what the provider generates and bills, so a model ignoring the 4–9-word instruction (or steered by injected tool output) could emit its provider-default output per batch. The sanitize step now installs a 256-token label cap (per provider family: `maxOutputTokens` for Google-style wrappers, `maxTokens` otherwise), after the filter so the omit set cannot remove it. - Flag-persist ordering (client.js): the `markActivityLabels` write was fire-and-forget, so an immediate cross-replica reconnect could read the job between the write and the first claim, see neither flag nor snapshot label, and skip gap reconciliation. Label emission now awaits the (settled-on-failure) persist chain, making "a label event exists" imply "the flag is durable" — the race window is gone; only the documented double-write-failure residual remains. - Listener detach (client.js): each HITL approval cycle's wiring adds a `once` abort listener to the shared job signal that only an actual abort removes; settled segments now detach theirs in `settleActivityLabels`, so long multi-approval runs cannot accumulate dead closures toward the listener-limit warning. Tests: the primary cap is REPLACED by the 256-token label cap (host.spec). * 🎯 fix: Route the Label Cap Per Model Family Round-twenty review: the 256-token label cap set maxTokens unconditionally, but GPT-5+ rejects max_tokens (the OpenAI builder routes its cap into modelKwargs.max_completion_tokens / max_output_tokens) and o-series models reject it with no stable kwargs alternative — every label on those models would have failed. The cap now mirrors the builder: modelKwargs for GPT-5+ (responses-API aware), no cap for o-series (title parity; the 200-char persistence bound still applies), maxOutputTokens for Google, maxTokens otherwise. Pinned in host.spec for both reasoning families. The round's other finding is the tenth instance of the documented edited+reconnect index limitation, answered on-thread. * ⏱️ fix: Persist the Label Flag at Run Start, Not on the Emit Path Round-twenty-one review: two fixed; the other three repeat the maintainer-decided packages/api extraction, the edited+reconnect index limitation, and the parallel-lane header limitation — all answered on-thread with their standing decisions. - Flag ordering, corrected (client.js): sequencing label emission behind the flag persist (previous round) delayed the claim-time reservation while the shared index offset had ALREADY shifted subsequent SDK chunks — reopening the cross-instance hole-compaction overwrite the reservation emit exists to prevent. The reservation emits immediately again; instead, run start (processStream and resume alike) awaits the settled-on-failure persist chain, so the flag is durable before any batch can claim a label. Same guarantee, zero latency on the emit path. - Tail-label cursor (ContentParts.tsx): a filled label at the content tail is consumed into the group header rather than listed in `group.parts`, so the `isLast` check missed it and nothing held the streaming cursor until the next delta. The check now includes `labelPart.idx`. * 🔌 fix: Detach Label Abort Listeners Even Without Claims A segment with labels enabled can end without a single claim (text-only, or handoff batches, which skip labels); the early return in settleActivityLabels skipped the detach added for HITL listener accumulation. The detach now runs on both paths. * ⚖️ fix: Make the Commit Flag the Sole Billing Authority Round-twenty-three review: a committed fill racing a late scope close (user abort or settle timeout during the durable emit) stayed visible — the part is mutated and persisted before the close — yet the deferred accounting's scope gates then skipped the charge: a completed provider call escaping both the label charge and the primary abort accounting. The scope gates on the deferred-usage path are removed; the hook's commit flag is now the single billing authority in BOTH directions. A dropped fill never reaches the accounting callback (billed-never-shown stays impossible), and a committed fill bills regardless of when its scope closed (shown-never-billed now impossible too). The dead `scopeOpen` payload threading is removed with it; the `recordActivityLabelUsage` parameter survives, defaulting open, for callers that own no commit signal. The round's other finding is the twelfth instance of the documented edited+reconnect index limitation, answered on-thread. * 🧮 feat: Bill Labels by Estimate When Providers Omit Usage Maintainer decision: follow the title convention rather than leaving label calls unbilled when a provider returns no usage metadata. The hook now passes a LAZY estimate thunk with the deferred accounting on the success path — the EXACT prompt the direct path sent (or the locally built equivalent for the SDK path: same entries, context, instruction, truncation contract, and continuity headers) plus the final normalized label. `recordActivityLabelUsage` invokes it only when no collected entry carries a real token count, counts both texts with the shared o200k_base tokenizer, and feeds the synthesized entry through the SAME pipeline (provider-tagged, streamed event, cost, balance transaction). Real provider usage always wins when present. The failure path passes NO estimate: a throw before a response bills only real collected metadata, never a full phantom prompt. Tests: the estimate thunk carries the exact invoked prompt and final label; the failure path defers with no estimate (runtime.spec). * 💵 fix: Estimate From the Raw Completion, Not the Normalized Label The fallback estimate counted the normalized label (first line, 200-char cap) while the provider generated and would bill the raw output up to the 256-token generation cap — under-recording verbose replies. The estimate thunk now carries the raw pre-normalization text; the persisted label is unchanged. Pinned with a multi-line reply test. * 🧾 fix: Commit Label Text Only After the Durable Emit, Estimate the Real SDK Prompt Round review on the billing work: two fixed; the third is the fourteenth instance of the edited+reconnect index limitation, answered on-thread. - Copy-first fill (wiring.ts): the fill mutated the shared content part BEFORE its durable emit, so a failed emit left the label text on `contentParts` anyway — persistence could save and display a label no client ever received and billing (keyed on the commit flag) never charged. The new state is staged on a copy; the shared part mutates only after the emit succeeds, so content, delivery, and billing move together. - Real SDK prompt for estimates (client.js): the estimate thunk carried this module's locally built prompt, but the SDK path frames entries differently — the estimated input count was for a prompt never sent. Chain-start callbacks (handleLLMStart/handleChatModelStart) now capture the prompt the SDK actually rendered, and the deferred accounting substitutes it into the estimate when capture succeeded, falling back to the local approximation otherwise.	2026-07-29 14:05:47 -04:00
Dustin Healy	f4e0888f14	🕶 feat: Generalize Admin Config Secret Redaction (#14509) ... * feat: generalize admin config secret redaction to a field registry Replace the single hardcoded langfuse.secretKey handling in the admin config secrets module with a registry (CONFIG_SECRET_FIELDS) so every credential-shaped config field is encrypted at rest, redacted on read, and preserved when omitted on a subsequent write. Registry covers langfuse.secretKey plus speech tts/stt provider apiKeys, ocr.apiKey, the webSearch provider apiKeys, and the assistants / azureAssistants endpoint apiKeys. Fields that conventionally hold ${ENV_VAR} references keep those references as plain, visible values; literal secrets are always encrypted. langfuse.secretKey behavior (display companion, always-encrypt, array-section handling) is unchanged. Wire runtime decryption for the consumers that read these values from the merged app config: resolveConfigSecret in the speech STT/TTS services and decryptConfigSecret in the Mistral OCR auth loader. Legacy plaintext literals and ${ENV_VAR} references continue to resolve. * feat: add masked display companions for every registered config secret Every non-langfuse field added to the secret registry was missing the displayPath that langfuse.secretKey already had, so redacted admin reads returned nothing for those fields instead of a masked value like sk-mis...Z789. The registry-driven encrypt/redact/preserve/mutation-path logic in secrets.ts was already field-agnostic; the only backend fix is setting displayPath on the other 15 registry entries. Add the matching optional display<Field> companion to each zod schema in librechat-data-provider (ocr, speech tts/stt providers, webSearch providers, the shared assistants/azureAssistants endpoint schema) so the field is typed for consumers, mirroring langfuse.displaySecretKey. DB overrides are Mixed-typed, so nothing breaks without this at the storage layer, but the type is needed for any typed consumer of TCustomConfig. A display path can never be written as a secret: direct writes to it are rejected, and an ancestor-object write that includes a spoofed display value alongside or instead of the real secret is overwritten or dropped, never encrypted or persisted. * fix: harden the config secret write path against masking and mixing bugs getDisplaySecretKey disclosed the entire value for any secret of 10 characters or fewer, since the first-6/last-4 mask overlaps or covers the whole string at that length (e.g. self-hosted LocalAI tokens). Short secrets are now fully masked instead. writeSecretIntoSection/writeDottedSecret encrypted a literal secret's raw string verbatim, including leading/trailing whitespace, so a padded paste round-tripped with the whitespace intact and a whitespace-only value was not treated as empty. Literals are now trimmed before encrypting and masking. Both functions also returned early on an env-placeholder value without clearing the display companion, so replacing a literal secret with ${ENV_VAR} left the previous masked value stale in the stored config, and a client-supplied display value submitted alongside a placeholder secret was never overwritten. The placeholder branch now clears the display companion in both the dotted-patch and object-valued write paths. * fix: fail closed in Mistral OCR auth when a stored ciphertext can't decrypt loadAuthConfig fell back to the raw v3: ciphertext string whenever decryptConfigSecret returned undefined, so a corrupted or otherwise undecryptable stored secret was sent to the Mistral API verbatim as the apiKey instead of triggering the existing env-var fallback. isEncryptedConfigSecret is now exported so the OCR auth loader can distinguish "this looks like ciphertext and failed to decrypt" from "this was never encrypted" and treat only the former as empty, preserving literal and ${ENV_VAR} values exactly as before. * fix: omit undecryptable TTS provider headers instead of sending "undefined" openAIProvider, elevenLabsProvider, and localAIProvider built their Authorization/xi-api-key headers directly from resolveConfigSecret's return value, which is undefined on a decrypt failure. That produced a literal "Bearer undefined" header (or an undefined-valued xi-api-key header) sent to the provider instead of failing gracefully. Each provider now resolves the key once and only includes the header when it's non-empty, matching the pattern already used by azureOpenAIProvider and STTService's providers. * fix: strip secret-ancestor arrays at any depth, not just the top level encryptConfigSecrets/redactConfigSecrets only stripped an array-valued registered-secret ancestor when it appeared as a literal top-level key (e.g. a dotted "speech.tts.openai" key). A true nested array at any depth, e.g. { speech: { tts: { openai: [{ apiKey: "sk-secret" }] } } }, made walkToParent return null and silently skip that field entirely, so the literal secret was stored unencrypted and returned verbatim to any reader with section-level read access. pruneSecretAncestorArrays now walks every registered field's ancestor chain and deletes any array found at any depth before encryption or redaction runs, closing the gap for both write and read paths. * fix: migrate legacy plaintext secrets instead of dropping them on preserve preserveConfigSecrets only restored an omitted secret when the existing stored value was already v3-encrypted. Every field this PR newly registers was previously stored as plaintext with no protection at all, so any deployment upgrading into this registry has real credentials sitting in Mongo as plaintext today. The first time an admin edited an unrelated field in the same section (e.g. mistralModel next to ocr.apiKey), the omitted plaintext secret failed the "already encrypted" check and was silently dropped instead of preserved, breaking the integration. The existing value is now encrypted in place when it isn't already ciphertext or an allowed env placeholder, so the credential survives the edit and gets a computed display companion instead of being lost. * refactor: derive masked-preview companions as <field>Preview Replace the display*-prefixed companion names (displaySecretKey, displayApiKey, displaySerperApiKey, ...) with a uniform <field>Preview suffix (secretKeyPreview, apiKeyPreview, serperApiKeyPreview, ...) derived automatically from the registered secret path — registry entries no longer declare a displayPath, and the name never collides with display-label config fields like modelDisplayLabel. Legacy langfuse.displaySecretKey companions (the only shipped instance, with no released reader) are stripped from writes and reads and migrated to secretKeyPreview on preserve, so stored documents self-clean. Also reject MongoDB operator segments ($, $[], $[id]) in admin config field paths: isValidFieldPath previously accepted them, letting a patch like webSearch.$[].serperApiKey reach patchConfigFields as a positional update that bypassed secret-path validation. * fix: translate legacy displaySecretKey to secretKeyPreview on reads Redaction previously deleted the legacy companion outright, so the first admin read of a not-yet-migrated document showed no configured-secret indication until a later write migrated it. Reads now surface the legacy value under secretKeyPreview (when no new-name preview exists) while still stripping the legacy key from the response; stored documents migrate for real on their next write. * fix: detect runtime ciphertext by full encryptV3 payload shape Runtime resolution (resolveConfigSecret, mistral OCR auth) now identifies decryptable values by the exact v3:<32-hex-iv>:<hex> shape encryptV3 produces instead of the bare v3: prefix, so a legitimate literal credential that merely starts with v3: (e.g. from YAML, which the admin write path never encrypts) resolves as a literal instead of failing decryption. Write-side prefix rejection stays broad as spoof/echo defense. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-07-29 13:55:45 -04:00
Danny Avila	3edb497502	📦 chore: bump @librechat/agents to ^3.3.5 (#14506)	2026-07-28 23:23:51 -04:00
Danny Avila	6dae785e31	🌯 chore: Retire Rollup-Era devDependencies After tsdown Migration (#14496) ... Removes 26 of the 32 Rollup-era devDependency declarations left behind when these packages moved to tsdown, plus two stale config references and an override that went inert in #14483. - Drop all 8 from `packages/api`, all 8 from `packages/client` (including `concat-with-sourcemaps`), and all 10 from `packages/data-schemas`. None of their tsdown configs import anything from rollup, and none has a rollup script or config file. - Keep all 6 in `packages/data-provider`. Five of them back the `rollup:api` script, which the "Circular dependency checks" CI job runs to surface rollup's circular-dependency warnings, and `@rollup/plugin-replace` is imported directly by that package's tsdown config. - Drop `rollup.config.js` (exists nowhere in the repo) and `server-rollup.config.js` (real, but never read by the `build` task, so listing it only caused spurious cache invalidation) from `turbo.json`. - Drop the `**/rollup.config.js` glob from `eslint.config.mjs`. It matches nothing, and never matched `server-rollup.config.js`. - Drop the root `svgo` override, dead since #14483 removed `rollup-plugin-postcss`, the only thing that pulled svgo into the tree.	2026-07-28 22:08:52 -04:00
Danny Avila	1fce7e1f3c	💬 refactor: Raise ask_user_question Option Label Cap to 280 Chars (#14491) ... * 💬 fix: Raise ask_user_question Option Label Cap to 280 Chars Raise OPTION_LABEL_MAX from 120 to 280 and make every ask_user_question surface wrap long, model-generated strings instead of overflowing. * 🪟 fix: Bound ask_user_question Popover to the Viewport The popover is absolutely positioned, so content taller than the viewport is unreachable by page scroll. Cap the panel at 60vh with the option list as the only flexible scroll region, and scroll the keyboard-selected row into view since selection paints a highlight without moving focus.	2026-07-28 14:23:23 -04:00
Danny Avila	9c95bf445f	🍂 chore: Prune Deprecated Packages From the Dependency Tree (#14483) ... Removes three of the eight deprecation warnings emitted on `npm install`. - Drop `@types/winston` from `packages/api` and `packages/data-provider`. The published tarball ships no type declarations at all, so `winston`'s own types were already being used. Declare `winston` as a devDependency instead, since both packages `import type { Logger } from 'winston'` and were relying on hoisting to resolve it. - Drop `rollup-plugin-postcss` from `packages/client`. It is unreferenced since the package moved to tsdown, and pulled in `cssnano -> postcss-svgo -> svgo@2`, which is the only consumer of the deprecated `stable`. - Override `test-exclude` to ^8 so `babel-plugin-istanbul` stops resolving `test-exclude@6`, which pins the deprecated `glob@7`. The remaining five warnings (`ldapjs`, `whatwg-encoding`, `node-domexception`, and workbox-build's `glob`/`source-map`) are transitive with no non-deprecated version available upstream.	2026-07-28 09:28:06 -04:00
Danny Avila	324584552c	⏱️ feat: Configurable HTTP Server Timeouts (#14481) ... * http server config added * Fix TypeScript compatibility by accepting NodeJS.ProcessEnv directly when applying optional HTTP server timeout configuration. * fix(api): configure HTTP server timeouts for clustered workers * 🕰️ fix: Warn When HTTP Timeouts Are Not Enforced Codex review of the rebased contributor work surfaced two ways these settings silently do nothing. Both reproduce, and neither was reported to the operator. Bun accepts the four property assignments and reflects them back, but does not enforce them: with keepAliveTimeout=100 and buffer=1000, Bun 1.3.13 held a keep-alive connection past 3s where Node 24 closed it at 1101ms. Since `b:api` runs the server under Bun, the existing info log confirmed a configuration that was not in effect. Warn instead. Node sweeps header/request timeouts on `connectionsCheckingInterval`, a createServer option that `app.listen()` leaves at 30s, so sub-30s values round up to it: headersTimeout=2000 returned 408 at 30004ms by default versus 2010ms with a 250ms interval. Warn on values below the sweep interval rather than restructure server construction, since every documented value and both Node defaults already sit well above it. keepAliveTimeout is socket-driven and stays exact, so it is excluded. Both caveats documented in .env.example. * 🩹 fix: Inject Runtime Versions Instead of Mutating `process.versions` The spec deleted `process.versions.bun` to reset between cases, which failed typecheck with TS2790: `@types/bun` is a packages/api dependency and augments NodeJS.ProcessVersions with a required `bun: string`, so the property is not optional and cannot be deleted. Assigning undefined would fail for the same reason. That augmentation also made the production check dishonest: TypeScript saw `process.versions.bun` as always a string, so `!= null` read as a no-op branch even though it is correct at runtime under Node. Both resolved by taking runtime versions as a third injectable parameter, matching the existing `environment` parameter. Callers in api/server are unchanged, the narrow `{ bun?: string }` type restores honest narrowing, and the tests no longer mutate global state, so they assert the same behavior whether the suite runs under Node or `bun jest`. * 📏 fix: Stop Claiming a Ceiling on Sweep-Delayed Timeouts The warning added in 9adc3eb1c said the effective timeout is "up to 30000ms", which promises a bound that does not hold. Node detects header/request expiry only on the next connection sweep, so the delay is relative to the deadline rather than capped by the interval: measured against the default 30s sweep, a 2000ms headersTimeout closed at 30004ms, 15x the configured value, and cases where the timeout is near the interval did not fire within a 9s window at all. Reworded to state the mechanism without asserting a ceiling, and to point operators at values of 30000ms or above for predictable enforcement. Same correction applied to the .env.example note, which claimed short values "round up" to the interval. * ⏳ fix: Clamp Headers Timeout to the Request Timeout Setting only HTTP_REQUEST_TIMEOUT_MS below the 60s headers default left headersTimeout > requestTimeout, a pairing createServer rejects outright with ERR_OUT_OF_RANGE. Assigning the properties after construction skips that validation, and the mismatch silently defeats the request timeout for a stalled body: with requestTimeout=4000 and headersTimeout at its 60000 default, a client that completed its headers and then stopped mid-body was still connected after 12s. Clamping headersTimeout to 4000 closes the same connection at 4017ms. An earlier round dismissed this after testing partial *headers*, where requestTimeout does evict on time. The gap only appears once headers are complete and the body stalls, which is the case these timeouts exist to bound. Mirrors Node's own rule rather than its constructor default: zero on either side means disabled and is left alone, and an explicitly configured HTTP_HEADERS_TIMEOUT_MS that conflicts is warned about before being clamped instead of failing startup over a config typo. --------- Co-authored-by: Peter Rothlaender <peter.rothlaender@ginkgo.com>	2026-07-28 09:10:17 -04:00
Danny Avila	f7bc50ae5b	📦 chore: bump @librechat/agents to v3.3.4 (#14482)	2026-07-28 08:36:18 -04:00
Dustin Healy	044c134ecf	🤏 fix: Filter Admin Config Reads by Section-Scoped Read Capability (#14472) ... listConfigs, getBaseConfig, and getConfig only checked the broad read:configs capability, so a caller holding nothing but read:configs:<section> grants got a blanket 403 on all three instead of a response filtered to the sections they hold. Any deployment using section-scoped config grants hits this. Adds hasAnyConfigReadAccess as a cheap pre-flight check covering broad and section-scoped read and manage grants (manage implies read), so a zero-access caller still 403s before a DB fetch while a section-scoped caller gets the response filtered to exactly what they hold. The same manage-implies-read rule is fixed at its root in getParentCapabilities so a manage-only caller sees the section they manage instead of having it stripped after passing the pre-flight. Resolves every section for a request in one batched getHeldCapabilities query via getReadableConfigSections instead of one round trip per section. Includes AppConfig field-renaming normalization (interfaceConfig, turnstileConfig, mcpConfig) so the filter checks the canonical section name rather than the renamed response field, and stops availableTools from bypassing the filter by gating it on its filteredTools/includedTools source sections.	2026-07-28 07:38:37 -04:00
Danny Avila	728fc1276e	🔒 fix: Bound /files/usage TTL Hold Instead of Clearing It (#14470) ... * 🔒 fix: Bound `/files/usage` TTL Hold Instead of Clearing It `POST /files/usage` marks queued attachments so the 1-hour upload-window TTL cannot reap them before the client queue drains. It did this by calling `updateFilesUsage`, which unsets `expiresAt` outright, turning every touched upload into a permanently retained file. The client queue is ephemeral browser state, so this also leaks in normal use: a closed tab or cleared queue leaves nothing referencing the files, but their TTL is already gone. The same mechanism let an authenticated user pin arbitrary owned uploads indefinitely, and the route was excluded from the file limiters, so the touch was entirely unmetered. Make the operation match its intent, a renewable hold rather than a release: - Add `extendFilesTTL`, which pushes `expiresAt` forward by a bounded window in a single owner-scoped `updateMany`. Two filter guards keep it safe under client-supplied ids: `$exists: true` so an already-released file never has a TTL re-added (that would schedule a live file for deletion), and `$lt` so a hold only ever moves the deadline later. The owner scope is a required argument, so an unscoped call is a no-op rather than a cross-user update. - `handleFilesUsageRequest` now holds for 24h instead of clearing, and no longer increments `usage`, since a queue touch is not a send. The real release still happens at drain, where `updateFilesUsage` marks the files used against an actual message. - Give `/usage` its own per-user limiter. Keeping it off the upload quota was intentional, leaving it unmetered was not. Abandoned queues are now reaped on schedule, and a replayed touch can only ever re-assert the same bounded window. * 🔒 fix: Anchor the `/files/usage` hold to upload time Codex review on b687922. The hold derived each new deadline from `Date.now()`, so a caller touching once a day advanced it by another 24h every time, far below the rate limit. That left indefinite preservation reachable and made the PR's replay claim wrong: the window was bounded per call but not in aggregate. Anchor the deadline to the file's immutable `createdAt` instead of the request clock. `extendFilesTTL` now takes a lifetime and sets `expiresAt = max(expiresAt, createdAt + holdMs)` in an aggregation pipeline, so the target is a fixed point per file and replay is inert rather than merely bounded. `$max` keeps the widen-only property and the `expiresAt: {$exists: true}` filter still refuses to resurrect a released TTL; `createdAt: {$exists: true}` fail-closes when the anchor is absent. The update runs with `timestamps: false`: a hold is TTL bookkeeping, not a content write, and bumping `updatedAt` also made every re-touch count as a modification, hiding whether the deadline actually moved. Also drop four `.node_modules-*` symlinks that `git add -A` swept in from an npm install. They pointed at absolute paths on one machine, so every other checkout got dangling entries. Added the pattern to .gitignore so a workspace install cannot reintroduce them. * 🔒 fix: Track the configured approval window in the `/files/usage` hold Codex review on 9277620. `endpoints.agents.checkpointer.ttl` is a positive int with no upper bound, and its docs invite raising it for longer review windows. It drives the pending-action expiry, so a run can legitimately stay paused past 24h. The fixed 24h lifetime would then let Mongo reap an attachment while its approval was still live, and the later queue drain would send a file that no longer exists. Replace the fixed constant with `resolveFilesUsageHoldMs`, which adds the configured approval window to a 24h baseline covering upload, enqueue, and the run reaching its pause. The route reads the window from the same `getApprovalTtlMs(checkpointerCfg)` the pending action uses, so the two stay in lockstep. The replay bound is unaffected: the window is a per-deployment constant and the deadline is still `createdAt + holdMs`, so a replayed touch re-asserts the same instant and `$max` skips the write. Only an operator config change moves it, never a client. * 🔒 fix: Renew the `/files/usage` hold across queued runs, under a ceiling Codex review on 2bd3c52. The drain sends one queued item per run completion, and each item starts a run that may itself pause for the full approval window. Since the hold was taken once at enqueue and pinned to the upload time, an item several places back could sit through multiple approval windows and lose its attachment while its chip and the live approval were still there. Another regression from this PR: the old `$unset` made retention permanent, so deep queues happened to work. The queue is unbounded, so no fixed lifetime covers it. Split the hold into a renewable window and a ceiling: expiresAt = max(expiresAt, min(now + renewMs, createdAt + maxLifetimeMs)) `renewMs` covers one run's wait and is granted from now, so a queue that is still draining re-asserts it at each transition; `useQueueDrain` now marks the remaining items' files whenever it pops one. `maxLifetimeMs` is measured from the immutable upload time and clamps every renewal, so repeated touches converge on a ceiling instead of advancing per call, which keeps the replay bound from the previous round intact. This also tightens abandonment: a queue nobody drains now lapses one `renewMs` after its last touch instead of surviving to the ceiling. `useQueueDrain`'s spec gained a QueryClientProvider, since the renewal goes through react-query. * 🔒 fix: Renew queued holds on a heartbeat, and stop dropping batches Codex review on f616bed. Three gaps in the renewal added last commit: - `collectQueuedFileIds` returned early at the server's 10-id cap, so a remainder holding more than one batch renewed only its first message and left the rest on their enqueue-time hold. Collect everything and split into capped requests instead of truncating. - A refused `ask()` restores the popped item, but renewal ran before the send and covered only the pre-existing remainder. Since the run-end signal is already consumed, nothing would touch that item again. Renewal now runs after `ask` and includes the restored item. - A single run can interrupt for approval more than once, each pause running to the configured window, so renewing only at drain transitions leaves a gap longer than `renewMs` with no renewal in it. The ceiling cannot help when nothing renews. The third is the same structural gap as the previous round along a new axis: renewal tied to discrete events loses the file whenever two events are further apart than the hold. Rather than hook each transition, renew on a 30 minute heartbeat while anything is queued, which is far below the smallest hold (24h) and so covers any single gap regardless of cause. Still bounded: every renewal is clamped against the file's upload time, so the ceiling is unchanged. A queue nobody has open emits no heartbeat and lapses one `renewMs` after its last touch, preserving the abandonment behaviour. * 🔒 fix: Cover the pre-migration queue, first tick, and `/usage/` Codex review on 892a27d. - The heartbeat watched only the active conversation id, but `drainNext` merges in the `NEW_CONVO` queue, which outlives the URL update: items queued during the first turn stay keyed there until that run ends. It now renews the union of both, deduped since they are the same atom before migration. - The interval installed without firing, so returning to a conversation whose hold was nearly up waited out a full period before the first renewal. It now renews immediately, then on each tick. - Express's non-strict routing sends `POST /files/usage/` to the same handler with `req.path === '/usage/'`, so the exact comparison pushed it onto both upload limiters. A trailing-slash client would have spent its upload quota, and collected file-upload violations, on metadata heartbeats. Matching now tolerates the trailing slash. Firing on effect start also made the drain-time renewal redundant: popping an item changes the held set, so the renewal effect re-runs on its own. The one case it cannot see is a refused send, where restoring the item leaves the set identical, so that branch keeps an explicit renewal and the rest is removed. Net one request per transition instead of two.	2026-07-28 07:37:26 -04:00
Danny Avila	e0892bb291	🍃 fix: Strip $-Prefixed Schema Keywords Before Persisting MCP Tool Params (#14464) ... * 🐛 fix: strip $-prefixed schema keywords from MCP tool params before storage MCP tools whose inputSchema carries a spec-compliant $schema keyword (or any other $-prefixed JSON Schema keyword) failed to register: MongoDB rejects field names beginning with $, so persisting the tool's parameters blob threw "The dollar ($) prefixed field '...$schema' is not valid for storage". Normalize the schema when building stored toolFunctions (resolve $refs and drop $-prefixed keywords via the existing resolveJsonSchemaRefs + normalizeJsonSchema pipeline). normalizeJsonSchema now strips every $-prefixed keyword, not just $defs, while preserving property names that happen to start with $. * fix: recurse through every schema-valued keyword when stripping $ keys MongoDB rejects $-prefixed field names at any depth, but the normalizer only recursed through properties, items, additionalProperties and unions, so a $schema or $comment nested under not, if/then/else, contains, propertyNames, patternProperties, dependentSchemas or prefixItems survived into the persisted tool parameters and still failed registration. The keyword sets are now explicit, covering the single-subschema, map-of-schema and list-of-schema forms. A $-prefixed property name is deliberately left alone: it is an argument the tool actually accepts, so dropping it would silently remove the parameter from the schema the model sees. * fix: recurse into draft-07 dependencies and 2020-12 contentSchema Both are schema-bearing and were absent from the traversal sets, so a nested annotation survived into the persisted tool parameters and still hit the MongoDB dollar-prefixed-field failure. dependencies is polymorphic - a value may be an array of property names rather than a subschema - and that form round-trips unchanged. * fix: keep __proto__ entries when normalizing schema maps Schema-map keys name instance properties, so __proto__ is a legal entry and arrives as a real own property via JSON.parse. Plain assignment invoked the prototype setter instead, silently dropping the constraint; entries are now defined rather than assigned. * fix: bound MCP schema reference expansion and keep __proto__ arguments A remote MCP server controls the schema fetched at registration, and sibling references to the same definition each re-expand because visited is cleared after resolving - so an acyclic graph where each Dn holds two refs to Dn-1 expands 2^n. At depth 24 that is over 16 million nodes, enough to block the event loop or exhaust memory before registration finishes. Resolution now carries a node budget and leaves a reference unexpanded once it is spent, and assignments use defineProperty so an argument legitimately named __proto__ is not swallowed by the inherited setter during resolution. --------- Co-authored-by: Arham Wani <arhamwani765@gmail.com>	2026-07-27 18:50:34 -04:00
Danny Avila	250aca375a	🔗 fix: Resolve MCP Tool-Key Boundary Against Configured Server Names (#14448) ... * fix: resolve MCP tool-name delimiter collision at invocation time MCP tool keys are identified internally as `${rawToolName}${mcp_delimiter}${serverName}` (delimiter `_mcp_`). Several call sites parsed this back apart with a naive `toolKey.split(Constants.mcp_delimiter)`, assuming the delimiter occurs exactly once. When the raw upstream tool name itself contains the delimiter substring - which happens whenever it's exposed through a gateway that prefixes aggregated tool names by server (e.g. a gateway's own "gitlab-get_mcp_server_version" for GitLab's "get_mcp_server_version" tool) - the combined key has the delimiter more than once. `.split()` then produces more than two segments, and destructuring `[toolName, serverName]` silently keeps only the first two, yielding a bogus server name that matches no configured server. Tool listing still worked (a different code path builds keys directly without re-splitting), but invocation failed with `Tool {name} not found`, and `filterAuthorizedTools` rejected such keys outright as malformed. Add `splitMCPToolKey`, which splits on the *last* occurrence of the delimiter instead: the server-name half is always LibreChat's own normalized suffix (guaranteed not to contain the delimiter), while the raw tool-name half is untrusted and may legitimately contain it. This matches `.split()`'s result whenever the delimiter occurs once, and correctly resolves the collision case. Update the four call sites that parsed this manually (`handleTools.js`, `MCP.js`, `mcp.js` controller, `filterAuthorizedTools` in `v1.js`) plus one in the client (`useVisibleTools.ts`) to use it. Fixes #14440 * fix: resolve MCP tool-key boundary against configured server names splitMCPToolKey moves to librechat-data-provider so the client and backend share one parser, and takes the configured server names when the caller has them: the longest name the key actually ends with wins, which is exact. Position alone cannot identify the boundary because both halves may contain the delimiter. lastIndexOf alone fixes gateway-prefixed tool names but regresses servers whose own name contains it, which ToolService.spec.js already covered; the last-delimiter path now only serves as the fallback for callers with no configured set. Also converts the remaining first-occurrence parsers that the delimiter fix missed - mcp/auth.ts (custom user vars silently unresolved), mcp/oauth/events.ts, agents/initialize.ts, and the three client parsers that labelled tool calls with the wrong server. * fix: keep client tool-call labels on first-delimiter parsing The three client parsers had deliberate, tested first-delimiter semantics (ToolCall.test.tsx asserts the full server name for 'foo_mcp_bar' and the synthetic 'oauth_mcp_server' call), and the client has no configured server list in scope to resolve the boundary exactly, so they are left as they were. Threads the configured names into the event-driven definition loader so it resolves the same boundary as the authorization filter that admits the key, and documents the one case that stays undecidable without provenance. * fix: resolve tool-key boundary against all configured servers resolveConfigServers only returns lazily-initialized config overrides - ensureConfigServers skips unmodified YAML servers - so on a stock deployment the known-name list was empty and suffix resolution never engaged. Adds resolveMcpServerNames, which keeps every configured server in the normalized form tool keys carry, and uses it at the loading, auth-map and definition sites. Background-tool eligibility now resolves against all configured names before testing ephemeral membership, so a non-ephemeral server whose name ends in an ephemeral one is no longer misclassified, and useVisibleTools resolves against the server map it already receives. * fix: use resolved server provenance and one app-config read createMCPTool now uses the serverName loadTools already resolved for the key and only parses as a fallback, so an unmodified YAML server whose name contains the delimiter no longer resolves to the wrong server for auth, reconnection and callTool. resolveMcpServerContext derives config servers and all configured names from a single getAppConfigForRequest, replacing two independent lookups on the chat startup path, and degrades to empty like resolveConfigServers instead of aborting tool loading when the config lookup fails. * chore: drop unused resolveConfigServers import * fix: forward server provenance on the all-tools path and read config once createMCPTools builds each toolKey from the server name it already has but did not forward it, so the sys__all__sys path re-derived it by parsing and bound an unmodified YAML server whose name contains the delimiter to the wrong auth and invocation context. loadAgentTools now resolves the MCP server context once and threads it into loadTools, replacing the second app-config read it had introduced on the non-event-driven chat startup path. * fix: carry resolved MCP server name through tool classification definitions.ts resolves the server for each key and then dropped it when building loadedTools, so buildToolClassification re-derived it with a last-segment split and recorded 'Workspace' for a server configured as 'Google_mcp_Workspace'. The resolved name now rides along on the tool instance and classification prefers it over re-parsing. * fix: consume carried server name when extracting MCP servers extractMCPServers re-derived the name with a last-segment split, so a server configured as Google_mcp_Workspace resolved to Workspace and its instructions were silently omitted. Prefers the name carried on the tool definition instance, falling back to the split. * fix: fail closed on ambiguous MCP keys when persisting server names Persisted mcpServerNames grant agent-scoped access to a DB server by name (ServerConfigsDB.getAccessibleServers), so a wrong guess exposes an unrelated server to everyone who can view the agent. The last-segment split turned search_mcp_Google_mcp_workspace into 'workspace'; such keys were previously rejected outright at agent save, so admitting them opened this path. Derives a name only from unambiguous single-delimiter keys. This is #12250's guard moved to the boundary it was actually protecting, instead of blocking tool admission. * fix: keep DB server access for multi-delimiter tool keys The fail-closed guard was wrong for the case this PR exists to fix. This index only grants DB-backed servers, and DB names are slugs that cannot contain the delimiter (generateServerNameFromTitle strips underscores), so the trailing segment is always the real server for them - dropping it cost every consumer of a gateway-prefixed tool their shared-agent access. Also gates the MCP server-context lookup on the filtered MCP set, so an agent with no MCP tools no longer pays an app-config read on startup. * fix: resolve tool-call display names without breaking OAuth calls The display parsers could not use the shared boundary parser because their tested behavior depends on first-delimiter semantics. That constraint only applies to synthetic MCP OAuth calls, whose tool half is always exactly 'oauth', so everything after the first delimiter is the server even when the server name carries one. splitToolCallName special-cases that form and defers to splitMCPToolKey for real tool keys, so a gateway-prefixed tool now renders its own name and server while oauth_mcp_foo_mcp_bar still resolves to foo_mcp_bar. * fix: persist resolved MCP server provenance on agents Deriving mcpServerNames from the tool key cannot tell a config server's trailing segment from a real DB server name, so a config server named a_mcp_b indexed an unrelated DB server b and shared the agent's viewers into it. Neither string rule works: the suffix guess exposes, and failing closed drops legitimate DB access for gateway-prefixed tools. filterAuthorizedTools already resolves each tool's server against the merged registry config, so it now collects those names and create, update and duplicate persist them. No extra registry queries: the update path unions the newly resolved names with what the agent already had, and duplicate replaces the copied list rather than inheriting the source's servers. Display parsing also takes the configured names, so a real tool call on a delimiter-bearing server renders the right server and icon. * test: teach MCP hook mocks about useMCPServerNames Three specs mock ~/hooks/MCP with a hand-listed factory, so adding the hook to ToolCall made useMCPServerNames undefined under test and every render threw. Returns a stable array so the mock cannot perturb render counts. * fix: rebuild agent MCP server index from surviving tools Unioning the prior names kept a server indexed after its last tool was detached, so viewers of a shared agent retained agent-scoped access to it. The index is now rebuilt from the tools that survive the edit: a prior name carries forward only while some retained tool still resolves to it, using the agent's own persisted names as the candidate set, and the rebuild runs on any tool change rather than only when a new MCP tool is added. * fix: keep duplicate indexes on registry fallback and harden the oauth split Duplication blanked mcpServerNames when the registry was unavailable, because filterAuthorizedTools grandfathers the source's tools without resolving them - the copy kept tools it could no longer resolve. Source names now carry forward for the tools that still point at them. splitToolCallName also treated any oauth_mcp_ prefix as a synthetic OAuth call, so a genuine upstream tool by that name resolved to the wrong server. A configured server name now decides when one matches, since a real key always ends in its server, and the prefix only breaks ties for unconfigured servers. * fix: thread configured server names through display parsing parseToolName and getMCPServerName resolved context-free, so a configured server whose name contains the delimiter showed the wrong server in grouped tool summaries and subagent tool labels, and stacked icons missed its entry in the icon map. Both take the configured names now, supplied by the components that render them. Adds the hook to SubagentCall's mock factory: the spec renders the real component, so an unmocked useMCPServerNames would reach the query with no provider. * test: cover the auth-map boundary, server provenance and context fallback Adds regression coverage for three behaviors this PR changed that no test exercised: customUserVars resolving under the right plugin key for a gateway-prefixed tool name (the failure that made these tools loadable but unusable), the resolved server name reaching createMCPTool instead of being re-parsed, and resolveMcpServerContext degrading to empty rather than aborting tool loading when the config lookup fails. Each was checked against a mutated source to confirm it fails when the behavior is broken. * fix: normalize server-name candidates and cover the boundary guard Tool keys embed normalizeServerName's output while the config is keyed by the raw name, so callers passing raw keys never matched a server whose name needs normalizing and silently fell back to the last delimiter. filterAuthorizedTools now maps normalized names back to their config key, and createMCPTool normalizes its candidates. Adds the cases an audit found surviving mutation: a configured name that is a bare but not delimiter-aligned suffix must not match, an empty candidate list behaves as no list, and splitToolCallName still falls back to the oauth prefix when a list is supplied but nothing in it matches. * fix: keep resolved server names when a non-owner retains MCP tools The shared-agent path keeps an agent's existing MCP tools verbatim but supplied no mcpServerNames, so persistence re-derived them and reduced a configured server like Google_mcp_Workspace to Workspace - which ServerConfigsDB then treats as a DB server, granting the agent's viewers access to an unrelated one. Carries the existing resolved names across instead, and clears the index on the owner path where every MCP tool is removed. * fix: preserve resolved MCP names for every tools update extractMCPServerNames was reachable from any caller that writes tools without mcpServerNames - the Action edit path does exactly that - so a configured Google_mcp_Workspace was reindexed as Workspace and ServerConfigsDB granted shared-agent viewers an unrelated DB server by that name. updateAgent now rebuilds the index from the agent's own resolved names: one carries forward while a retained tool still resolves to it, and only keys matching none of them fall back to derivation. Callers are safe by default rather than by remembering to pass the set. normalizeServerName moves to librechat-data-provider so the client can match its candidates against tool keys, which embed the normalized form; the icon map is keyed the same way since it is looked up with a parsed server name. * refactor: move MCP context resolution into packages/api New backend logic belongs in the TypeScript workspace per CLAUDE.md, with /api kept to a thin wrapper. resolveMCPServerContext now lives in packages/api/src/mcp/context.ts and takes ensureConfigServers by injection, since the registry accessor is still legacy-only; the /api function is reduced to loading the request app config and translating failures into the empty degrade it already promised. * test: teach the MCP service mock about resolveMCPServerContext The spec mocks @librechat/api with a hand-listed factory, so moving the resolver into that package left it undefined and the wrapper degraded into its own catch, returning empty config servers. The stub mirrors the real resolver so these tests still cover what the wrapper owns - loading the request config and degrading on failure - while the resolution logic is unit-tested in packages/api. * fix: only persist an authoritative MCP server index on update Assigning the resolved set unconditionally pinned the index to [] whenever nothing authoritative was available - a legacy agent holding MCP tools with no stored mcpServerNames - which suppressed updateAgent's derivation and stripped agent-scoped access to its DB-backed server. The field is now supplied only when the result is authoritative: names were resolved, or no MCP tool survives so the index genuinely is empty. The retained-tools branch likewise leaves it unset when the agent has none stored. --------- Co-authored-by: Jens Schumann <schumajs@gmail.com>	2026-07-27 14:45:38 -04:00
Danny Avila	531fecc82b	🍃 chore: Bump Mongoose to 8.24.1 to Patch Prototype Pollution (#14461) ... Closes GHSA-664h-wqgq-64gw (CVSS 6.5, CWE-1321), a prototype pollution in update casting via a __proto__-prefixed dotted path. Affected range is >=8.0.0 <8.24.1, so 8.23.1 was flagged by npm audit.	2026-07-27 12:40:14 -04:00
Danny Avila	a53936d273	🧭 test: Cover Agent Handoffs End to End (#14428) ... * test: cover agent handoffs end to end * style: sort handoff imports * fix: normalize missing agent handoff edges * chore: update package dependencies and versions in package-lock.json and package.json * chore: bump agents SDK	2026-07-27 08:47:15 -04:00
Danny Avila	d8427ffc5e	🛂 test: Cover Tool Approval Workflows End to End (#14427) ... * test: cover tool approval workflows end to end * fix: preserve tool approval state across resume * fix: preserve agent context in mock stream responses * fix: preserve nested approvals in collapsed groups	2026-07-26 21:58:25 -04:00
Danny Avila	73699b5c25	⚡ perf: Reduce Agent Chat Startup Latency (#14423) ... * perf: reduce agent chat startup latency * test: align Redis stream readiness assertions * perf: overlap remaining agent startup work * perf: persist initial agent job metadata atomically * test: add agent startup latency benchmark * fix: harden resumable agent stream lifecycle * fix: isolate replacement stream lifecycles * fix: preserve terminal stream epochs	2026-07-25 07:58:20 -04:00
Danny Avila	cd215150cc	✳️ feat: Claude Opus 5 Support (#14422) ... * ✳️ feat: Claude Opus 5 Support - Add claude-opus-5 to Anthropic/Bedrock model lists, token maps, and pricing - Extend requiresExplicitThinkingDisabled to Opus 5 so thinking-off sticks - Clamp xhigh/max effort to high when thinking is disabled (Opus 5 400) * 🪣 fix: Use Bedrock Inference Profiles and Add Vertex Opus Models Bare `anthropic.` Claude 4+ IDs are not invocable on-demand via Converse: Bedrock rejects them with "Retry your request with the ID or ARN of an inference profile that contains this model." Verified live against us-west-2 for Fable 5, Opus 5, Opus 4.8, Sonnet 5, Sonnet 4.6, Opus 4.6, Sonnet 4.5, Haiku 4.5, and Opus 4.1. Switch those defaults to the `global.` profile (no regional pricing premium); Opus 4.1 has no global profile, so it uses `us.`. Also add the modern Opus family to the Vertex defaults. `loadEndpoints` swaps the shared Anthropic list for the Vertex model names, so Opus was invisible to every Vertex deployment that did not enumerate models by hand. * 📋 chore: Cover Opus 5 Gaps From PR #14420 Picks up items from the parallel community PR by @jona7o: - Add claude-opus-5 to the librechat.example.yaml Vertex example (both the legacy array and the deploymentName map), which already lists Fable 5 and Opus 4.8 - Mention Opus 5 in the configureReasoning doc comment, and note that its early return is why the effort cap is enforced by the caller - Assert Opus 5 carries no long-context premium pricing - Cover the Sonnet 5 negative case for the effort cap, and the persisted disabled-object round-trip carrying an effort * 🌍 docs: Warn That Vertex Regional Endpoints Reject Modern Models Anthropic serves Sonnet 4.6 and earlier on specific Vertex regional endpoints; newer models (Opus 4.7+, Opus 5, Sonnet 5, Fable 5) require `global` or a multi-region location and 404 on a specific region. The `us-east5` default therefore cannot serve the Opus models added here, nor the Sonnet 5 entry that predates this branch. Documents the constraint at all three places an operator sets the region, and at the fallback itself. Leaves the default unchanged: switching it to `global` would silently alter data routing and residency for existing deployments, which is a separate call. * 🩹 fix: Restore PDF Exemption for Undated IDs and Gate Vertex Defaults Two issues raised in review: - BEDROCK_CLAUDE_4_PLUS_RE required a `-` after the major version, so it matched `claude-opus-4-8` but not undated IDs like `claude-opus-5`. Those models silently lost the Claude 4+ PDF exemption and fell back to the 4.5 MB limit. Sonnet 5 and Fable 5 were already affected before this branch; Fable/Mythos were also missing from the family alternation. - The Vertex defaults advertised models that only `global` and the multi-region locations serve, so a default `us-east5` deployment listed Opus choices that 404 on first request. Filter the built-in defaults by configured region instead of changing the region default, which would alter data routing for existing deployments. An explicit `vertex.models` list is the operator's choice and is never pruned. * 🧩 fix: Match Bare Claude IDs in the Bedrock PDF Exemption An application inference profile maps a LibreChat model ID with no `anthropic.` segment, so `claude-opus-5` failed the Claude 4+ check and fell back to the 4.5 MB PDF limit. Make the prefix optional and accept both segment orders, mirroring BEDROCK_CLAUDE_4PLUS_THINKING in librechat-data-provider, which matches on the family token for exactly this reason. Only reached for the Bedrock provider, so the looser prefix cannot leak into other endpoints. Verified Claude 3.x, Nova, Llama, Cohere, and Mistral IDs still fall through to the default limit. * 🧹 fix: Drop Retired Claude 3.5 Models From Bedrock Defaults The three Claude 3.5 entries reached end of life at AWS and return ResourceNotFoundException in every prefix form (bare, `us.`, `global.` — verified live against us-west-2), so selecting one was a hard error. Their modern equivalents are already in the list: Sonnet 5 / Sonnet 4.6 supersede the 3.5 Sonnets, and Haiku 4.5 supersedes 3.5 Haiku. Every remaining Anthropic default is now live-verified invocable. `.env.example` swaps its retired example ID for Haiku 4.5. * 🔒 refactor: Narrow Effort-Clamp Types Instead of Asserting Both clamp sites reached into loosely-typed containers with assertions: llm.ts used an `as unknown as { type?: string }` double assertion to read the thinking type, and the Bedrock parser cast `output_config` to `{ effort?: unknown }` before confirming it was an object. CLAUDE.md's type-safety rules call for narrowing over both. Adds `isThinkingDisabled` and `clampOutputConfigEffort` to librechat-data-provider, using `in`-operator narrowing and a type predicate so no assertion is needed at all. Both call sites now share one implementation rather than duplicating the clamp. Behavior is unchanged; existing clamp tests cover it.	2026-07-24 21:45:32 -04:00
Danny Avila	6c97a7f467	♾️ fix: Preserve Resumable Stream Ordering Across Turns (#14411) ... * fix: preserve resumable stream ordering across turns * chore: sort stream regression imports * test: mirror sliding sequence ttl in publisher mock * fix: prevent duplicate early stream replay * fix: preserve replay frontier when sync fails	2026-07-23 13:25:38 -04:00
Danny Avila	00c5a747e9	🧵 feat: Native Background Execution for Code Interpreter Tools (#14386) ... * 🧵 feat: Native Background Execution for Code Interpreter Tools * 🩹 fix: Address Codex Round 1 (fallback dedupe, harvest failure, handle parsing) * 🩹 fix: Live Completion Marker + Unkeyed Attachment Dedupe (Codex Round 2) * 🎨 chore: Sort Imports + Widen Marker Type Comparison (CI) * 🩹 fix: Stale-Harvest Guard, Error Marker Status, Faster Anchor Retry (Codex Round 3) * 🧹 refactor: TS Harvest Module, Claim-Neutral Timestamps, Error Parity (Codex Round 4) * 🩹 fix: Dispatch-Ordered Stale Guard, Foreground Downgrade, Error Wrapper Parity (Codex Round 5) * 🩹 fix: Retry Past Unfinished Rows + Per-Call Attachment Dedupe (Codex Round 6) * 🩹 fix: Writer-Dispatch Ordering, Scoped Live Upserts, Reaped-Task Wrapper (Codex Round 7) * 🩹 fix: Wildcard toolCallId Matching for Bare Attachment Updates (CI) * 🩹 fix: Claim-Insert Dispatch Stamp (Schema-Backed) + Scoped Status Markers (Codex Round 8) * 🩹 fix: Pre-Write Ownership CAS + Agent-Scoped Part Patching (Codex Round 9) * 🩹 fix: Insert-Path Ownership CAS + Agent-Routed Attachments (Codex Round 10) * 🩹 fix: Agent-Scoped Marker Ids and Attachment Dedupe (Codex Round 11) * 🩹 fix: Atomic File Commit and Sibling Preview Fan-Out (Codex Round 12) - Replace the two-step claim-confirm CAS with an atomic conditional updateFile: the ownership predicate (no sourceDispatchedAt, or <= this write's dispatch order) moves into the update filter, removing confirmCodeFileOwnership and the lost-update window between check and write - Thread agentId through createDownloadFallback so fallback download rows scope to the emitting agent like primary rows - Fan terminal preview overlays out to every live attachment sharing the file_id in useAttachmentPreviewSync (sibling tool calls no longer stick on pending) - Restore background artifacts through toStoredArtifact so the size bound applies on re-anchor - Apply filterAttachmentsForPart to grouped tool-call attachments in ContentParts so handoff agents with colliding provider call ids do not cross-contaminate groups * 🩹 fix: Agent-Scoped Live Upserts and Monotonic Dispatch Stamps (Codex Round 13) - Scope the SSE attachment upsert and the useAttachments DB/live merge by agentId with the same wildcard semantics as toolCallId: distinct non-null agentIds stay separate entries, so handoff agents sharing a claimed file_id and a repeated provider tool id (call_0) no longer merge over each other's cards - Extend the attachment identity key to fileKey::toolCallId::agentId and register less-specific key variants so bare and agent-less live records still dedupe after overlay - Stamp background task createdAt from a strictly-increasing per-process dispatch counter: raw Date.now() can tie for same-millisecond dispatches and the stale-output guard accepts equal stamps (needed for idempotent re-commits), which would let an older task overwrite a newer task's committed file	2026-07-22 22:13:15 -04:00