599 commits

Author	SHA1	Message	Date
Danny Avila	29f738dea8	⏭️ feat: Honor an Interrupt Before the Model Has Answered (#15491) ... * ⏭️ feat: Honor an Interrupt Before the Model Has Answered An interrupt armed while the model is still thinking sat as "Interrupting" until the entire turn finished, then landed as a terminal continuation. The cause is upstream: the SDK reads `shouldPreempt` once per streamed chunk, and its seal gate requires non-empty text — so a silent provider is never polled at all, and a reasoning-only turn is never sealable. `@librechat/agents` gains a discard-and-restart path for exactly that window (`HOOK_PREEMPT_RESTART_CAPABLE`). This wires the host half: - `GenerationJobManager.subscribePreempt` registers run-scoped wake listeners, notified after an ACCEPTED arm — local or cross-replica, since both land in `armPreemptIds`. Fenced on `createdAt` like every other preempt entry point, and a throwing listener never fails the arm a queued steer depends on. - `createSteerPreemptPoll` hands that channel to the SDK as `StreamPreemption.subscribe`, gated on a new `isSteerPreemptRestartSupported` probe so an SDK that cannot act on a wake is never handed one. The wake is a hint only: `isPreemptRequested` stays the single authority, so the level-triggered contract the seal path depends on is unchanged. Inert until `@librechat/agents` publishes the restart contract — the probe returns false against the pinned 3.7.11 and the run keeps today's behavior. * 🩹 fix: Replay an Already-Armed Preempt on Subscribe Preempt requests are level-triggered, and the window between a job becoming steerable and the SDK installing its model-attempt listener is real. An interrupt landing there was recorded with no callbacks to notify, and on a silent or reasoning-only turn no later chunk poll may ever run — leaving the request to wait out the whole turn, the exact stall this channel removes. `subscribePreempt` now replays an existing arm to the newly registered listener only, after registration: waking the whole set would re-notify runs that already looked, and waking before registration would leave a concurrent arm with nowhere to land. The `createdAt` fence is unchanged and still gates the replay. The SDK also reads the level flag once at each attempt's start, so this is belt-and-braces rather than the sole cover — but it makes the host correct on its own terms instead of resting on when the SDK happens to look. * 🔧 chore: Update @librechat/agents dependency to version 3.7.14 in package-lock.json and related package.json files * 🔧 chore: Update dependencies in package-lock.json and package.json - Bump versions for @humanfs/core (0.19.1 to 0.19.2), @humanfs/node (0.16.6 to 0.16.8), @xmldom/xmldom (0.8.13 to 0.8.15), and qs (6.15.2 to 6.16.0). - Add @humanfs/types (0.15.0) as a new dependency for @humanfs/node. - Update dependencies for side-channel and side-channel-list to their latest versions.	2026-09-02 13:35:52 -04:00
Dustin Healy	0b96fb9f70	🪪 feat: Propagate Tenant to Agent Model Headers (#15521) ... * fix: propagate tenant to agent model headers * fix: propagate tenant across agent model calls	2026-09-02 13:25:38 -04:00
Danny Avila	1c7a172321	🕹️ feat: Configure BYOM Tool Permissions (#15504) ... * feat: add configurable BYOM permissions * test: cover BYOM permission settings live * fix: harden BYOM permission settings * perf: reuse principals for environment discovery * chore: normalize BYOM settings style * fix: Align BYOM Discovery and Admission * fix: Harden BYOM Permission Boundaries * fix: Align BYOM Skill Approval Routing * fix: Preserve Lazy BYOM Approval Capabilities * perf: Parallelize Cached Environment Reads	2026-09-02 10:38:00 -04:00
Danny Avila	b1c56239f9	🧤 feat: Enforce Safe BYOM Tool Approvals (#15503) ... * feat: add safe default approvals for BYOM code * fix: preserve safe BYOM approval baseline	2026-09-02 08:53:16 -04:00
Marco Beretta	2cf1007f3c	🧺 feat: Preserve Partial Work at Tool Call Limits (#15486) ... * feat: Handle reaching the agent tool-call limit gracefully When the graph exhausts its per-turn step budget, persist the turn as unfinished with finish_reason tool_call_limit instead of a red error. Warn the model as the budget runs low, and offer Keep going or Answer now. * docs: Add screenshots of the tool-call-limit notice Light and dark captures of the unfinished-turn card for the PR description. * style: Tighten the tool-call-limit notice card Pin dismiss to the top-right with uniform rounding, use a slightly smaller hit target, and round the card to xl. Refresh the PR screenshots. * fix: Address Copilot review findings on the step-budget path Skip empty and subagent-scoped PostToolBatch events when counting rounds. Drop the double-cast in the notice test helper and coerce the tool-call-limit finish reason in the Continue-gate spec. * fix: Address Codex review findings on budget and dismiss Pass the resolved recursion limit into the OpenAI-compatible processStream path so the step-budget hook matches the graph. Reset the notice dismiss state when the displayed message id changes. * fix: Unblock CI typecheck and import sorting Declare recursion_limit on the OpenAI service agent type, sort that file's local imports, and route the notice test context through unknown so tsc accepts the partial ChatContext fixture. * fix: Sort remaining PR imports for static checks * chore: Remove tool-call-limit notice screenshots * fix: Keep recovery actions from consuming composer drafts Pass empty file, skill, and quote overrides on Keep going and Answer now so a staged compose is not attached or drained.	2026-09-01 21:38:02 -04:00
Danny Avila	164f146bdf	🛞 feat: Manage Personal Code Environment Lifecycles (#15369) ... * feat: add self-service code worker pairing * fix: enforce code worker principal boundaries * fix: separate pairing control planes from execution * fix: preserve principal worker routing * fix: pin artifact traffic to code workers * fix: authenticate code worker selection * test: mock worker-aware artifact headers * fix: harden self-service code worker routing * fix: fail closed for principal code pairing * feat: manage personal code environments * fix: harden personal code environment lifecycle * fix: satisfy backend lifecycle types * fix: harden principal environment lifecycle and quotas * fix: fence environment lifecycle races * fix: reconcile code environment stack * style: format code execution headers * fix: reconcile code environment lifecycle * fix: close self-service pairing gaps * fix: close code environment reconciliation gaps * fix: close code environment UI review findings * fix: satisfy code environment type checks * fix: route code priming to selected worker * fix: harden code environment lifecycle fences * fix: type lease renewal interval * fix: harden environment lifecycle recovery * fix: close lifecycle recovery races * fix: type compensated agent timestamps * fix: fence lifecycle cleanup by tenant and revision * fix: scope environment revision fencing	2026-09-01 21:10:05 -04:00
Danny Avila	a3c2277c38	🚧 fix: Narrow Chat MCP Selections to What the Picker May Offer (#15478) ... * 🚧 fix: Narrow Chat MCP Selections to What the Picker May Offer The chat MCP picker offers a server only when the operator has not hidden it (`chatMenu: false`) and the user reaches it directly rather than through an agent (`consumeOnly`). Nothing enforced either condition on the request, so a stale client, a replayed body, or a hand-written one could name any server and get its tools — `consumeOnly` in particular is a permission the registry derives per user, so ignoring it let a user reach in plain chat a server they had only been granted through an agent. Narrow the picker's selection in the two ephemeral loaders, against the same predicate the client's `selectableServers` uses. The resolver is injected, so `packages/api` stays free of the registry and the loaders keep their existing `deps` shape. Provenance is why this lives in the loaders rather than at the tool-resolution chokepoint: only here is a server name known to have come from the picker. Servers a model spec pins are the operator's own choice and are added after the narrowing, so pinning a chat-hidden server to a spec keeps working. `chatMenu` only ever comes from the config tier, so the overlay settles it with no lookup at all. Lookups for the rest run concurrently and are cached per server and user. A name the registry cannot resolve — request-tier, declared on the body and never registered — is kept, as is one whose lookup fails: this narrows an already-authenticated selection and is not the authorization boundary. * 📝 docs: Say What chatMenu Now Bounds The schema still described `chatMenu` as dropdown visibility alone, which is misleading now that it also bars a server from the chat selection a request carries. Spell out both halves, and what it deliberately does not restrict: agents, and servers a model spec pins. * 🛡️ fix: Close Four Gaps in the Chat MCP Narrowing Dedupe before resolving. The selection is request-supplied and the loaders used to build a `Set` immediately; mapping the raw array first let one body repeat a name enough times to launch a resolver — registry, cache, and ACL reads, with no single-flight — per occurrence. Resolve names through the tool-loading aliases. A configured `private server` is addressable as `private_server` downstream, so an exact-key lookup let the normalized spelling walk past a `chatMenu: false` check and still load tools. Names are returned as sent; only the decision is canonicalized. Let the request overlay settle `chatMenu`. The config tier is where the flag lives and is what the picker itself was built from, so an overlay that re-enables a server now outranks the durable config the registry returns, which was silently dropping a valid selection. `consumeOnly` still comes from the registry. Narrow the request body, not just the loader's copy. `applyContextToAgent` reads `req.body.ephemeralAgent.mcp` directly and prefers it over the agent's tools, so a hidden server's `serverInstructions` reached the model context even after its tools were dropped. * 🧭 fix: Ask the Registry Which MCP Servers the Picker May Offer Three rounds of review found the same mistake in three disguises: this narrowing re-derived the registry's tier precedence instead of asking for it. A config entry was treated as authoritative over a user-DB or process-backed server of the same name; the alias map was built from config names only, so a DB server named `private_server` was misrouted to a config `private server`; and the overlay short-circuit ran before any of that could be consulted. Resolve the accessible set through `resolveAllMcpConfigs` instead — the same call behind `GET /api/mcp/servers`, so what the request may use and what the picker was offered come from one place and cannot drift. Precedence, cross-tier name collisions, and the config overlay are the registry's job again, and the selectability test is a single flag check over what it returns. The result is smaller: no overlay branch, no per-server lookups, one catalog read regardless of how long the selection is, and aliases built over the whole accessible set so an exact name always outranks another server's normalized form. Publish the servers actually in play back onto the request body, spec pins included. `applyContextToAgent` prefers that list over the agent's tools, so narrowing to the picker selection alone had started dropping a spec-pinned server's `serverInstructions` while suppressing a hidden one's.	2026-09-01 18:47:36 -04:00
Danny Avila	51edb2a660	🛫 feat: Enable Self-Service Code Worker Pairing (#15367) ... * feat: add self-service code worker pairing * fix: enforce code worker principal boundaries * fix: separate pairing control planes from execution * fix: preserve principal worker routing * fix: pin artifact traffic to code workers * fix: authenticate code worker selection * test: mock worker-aware artifact headers * fix: harden self-service code worker routing * fix: fail closed for principal code pairing * style: format code execution headers * fix: close self-service pairing gaps * fix: satisfy code environment type checks	2026-09-01 18:46:33 -04:00
Danny Avila	aa7539909b	🪶 feat: Enable Standalone Skill Authoring (#15480) ... * feat: enable standalone skill authoring * test: align skill e2e flow with enable toggle * fix: preserve legacy skill state on version restore	2026-09-01 18:39:05 -04:00
Danny Avila	dccd7ae496	🧶 fix: Preserve Multi-Agent Tool Call History (#15468) ... * 🧭 fix: Preserve Multi-Agent Tool History * 🧰 fix: Preserve Lazy Subagent Tool History * 🧪 test: Enable Lazy Code History Fixture * 🧭 fix: Derive Effective Lazy Tool History * 🧪 fix: Tolerate Empty MCP Name Audits * 🧪 test: Mock Lazy MCP Name Audit * 🧭 fix: Expand Lazy Historical Tool Names * 🧱 fix: Derive Durable Wildcard History * 🧬 fix: Traverse Nested Wildcard History * 🔭 fix: Resolve Wildcard Server Boundaries * 🔐 fix: Persist MCP Call Identity * 🪪 fix: Stamp Resolved MCP Execution Identity * 🕸️ fix: Resolve Graph Member MCP Identity * 🧭 fix: Fall Back From Empty Graph Member ID * 🧭 fix: Preserve Aggregated MCP Identity * 🧭 fix: Harden MCP Identity Resolution * 🧭 test: Assert Concurrent Graph Limit Invariant * 🧭 fix: Centralize Lazy History Policy * 📦 chore: Bump `@librechat/agents` to v3.7.13 * 🧪 test: Align Content Filter Fixture With SDK History	2026-09-01 18:37:45 -04:00
Danny Avila	97e1e950ad	🧲 fix: Draw OpenID Refresh Lookups Into System Context (#15451) ... * fix: scope refresh user lookups to system context * fix: preserve refresh system context * fix: bind refresh recovery to user tenant	2026-09-01 02:02:21 -04:00
Danny Avila	49e1e32f3a	🧺 fix: Funnel MCP Catalog Work Through a Shared Gate (#15429) ... * 🧯 fix: Bound MCP Catalog Work Across Requests * 🛑 fix: Reject Saturated MCP Discovery * fix: fairly bound MCP catalog work * test: type catalog snapshot mock arguments * test: assert snapshot deadline forwarding * fix: harden MCP catalog scheduling	2026-08-31 14:13:31 -04:00
Danny Avila	982a9a5141	🧴 fix: Strip LangChain URLs From Provider Errors (#15421) ... * 🧹 fix: Strip LangChain Attribution From User-Facing Provider Errors LangChain classifies provider failures by mutating `error.message` with a docs URL, and that message is forwarded verbatim into the error content part we persist, so users saw `docs.langchain.com/...` credited for a provider's own 429. The client only intercepted `MODEL_NOT_FOUND`, leaving every other code to render raw. Resolve the classification server-side instead: codes with localized copy become the typed payload the client already knows how to localize, and every other message keeps the provider's wording with the docs URL removed before it reaches the database. The untouched error still reaches the logs through `getSafeErrorMetadata`. The client keeps a code-driven lookup for messages persisted before this, and gains `com_error_model_rate_limit` alongside the existing not-found copy. The OpenAI-compatible and Responses controllers now share one provider-error helper rather than three copies of it, so the strip applies on those surfaces too. * 🛡️ fix: Harden Error Sanitization For Non-String Messages And Legacy Text Codex review follow-up. `stripLangChainTroubleshootingUrl` assumed a string, so an error-like object with a truthy non-string `message` threw `.replace is not a function` inside the error handler that was meant to report it — the template interpolation it replaced had coerced such values safely. It now coerces at the boundary, as does the code lookup. Legacy messages whose LangChain code has no localized copy still rendered the docs URL, which is exactly the historical case the client-side path exists to clean up. The rendered fallback is now stripped too. Both helpers move to `librechat-data-provider`, which is shared by the client and the backend, so one implementation serves both instead of a second regex drifting in the component.	2026-08-31 12:51:34 -04:00
Danny Avila	b7ae67307e	🕰️ fix: Anchor Temporal Variables to Turn Start (#15419)	2026-08-31 10:38:10 -04:00
Danny Avila	413906abc3	🖍️ fix: Honor Agent Image Detail Settings (#15407) ... * 🖼️ fix: Apply an Agent's Configured Image Detail The Agent Builder exposes an Image Detail slider and saves it to `model_parameters.imageDetail`, but nothing ever read it back: `encodeAndFormat` resolves the level from `req.body.imageDetail`, which the agents route never populates, so every image was encoded as `detail: "auto"`. Providers that reject `auto` — MiniMax's OpenAI-compatible API among them — fail every image upload through an agent, and the setting that would fix it has no effect. The root cause is a classification error rather than a missing argument. `extractLibreChatParams` splits LibreChat's own settings out of an agent's model parameters, and its key set is derived from the `librechat` group in `parameterSettings`. `imageDetail` sat in `baseDefinitions` instead, so it was neither extracted for LibreChat's use nor stripped from the options handed to the provider. It is now grouped with `resendFiles` and the other LibreChat-level parameters, which gives it the same path those already take: extracted during agent initialization, carried on the initialized agent, passed to the client, and read by `encodeAndFormat`. `AgentClient.getSaveOptions` already expected `options.imageDetail` — nothing had ever populated it. The request body still wins where it is set, so the classic endpoints keep resolving detail from conversation settings exactly as before. Fixes #15359 * 🔤 fix: Type Image Detail as Its Enum, Not a Bare String `TConversation.imageDetail` is the `ImageDetail` enum, so the extractor's return type and the initialized agent's field are now declared as that enum rather than widened to `string`, and the tests pass enum members instead of raw literals. Caught by CI's type check, which I had run before adding the tests and not after — jest compiles the specs through babel, so the suite stayed green over a type error.	2026-08-31 08:34:05 -04:00
Danny Avila	c7f0a8f247	📬 feat: Move Agent Queued Turns Server-Side (#15370) ... * feat: move agent queued turns server-side * test: update queued turn composition coverage * fix: harden queued-turn lifecycle ownership * fix: centralize queued-turn lifecycle ownership * fix: close queued-turn lifecycle gaps * fix: unify queued turn lifecycle transitions * fix: fence queued-turn deletion admission * fix: preserve queued-turn lifecycle ownership * fix: close queued-turn reconciliation races * fix: reconcile quarantined queued-turn admissions * fix: annotate admission evidence default * fix: deepen queued turn lifecycle admission * fix: fence queued turn provider admission	2026-08-30 21:53:58 -04:00
Danny Avila	fcae1025c0	🧳 feat: Register Principal-Owned Code Environments (#15365) ... * feat: add principal-owned code environments * fix: address code environment CI coverage * fix: expose principal code environments in endpoint config * fix: reject missing code environment bodies * fix: harden code environment lifecycle * fix: revalidate principal code environments * fix: synchronize code environment authorization * fix: bind code environments to current principals * fix: fail closed on code ACL cache errors * fix: prevent code environment override shadowing * fix: preserve principal environment defaults * fix: suppress revoked code environment aliases * fix: fail closed on environment augmentation * fix: narrow code environment defaults * fix: isolate code environment fallback * style: sort code config imports	2026-08-30 19:33:19 -04:00
Danny Avila	7533d138fa	🧬 perf: Evolve Compaction Guidance on Warm Turns (#15371) ... * perf: evolve compaction guidance on warm turns * style: sort compaction adapter imports	2026-08-30 17:11:20 -04:00
Danny Avila	30124f21b2	🎻 refactor: Orchestrate Agent Runs Through a Request-Free Host (#15366) ... * refactor: decouple agent initialization from HTTP * refactor: centralize remote agent execution lifecycle * fix: preserve pre-settlement error rendering * fix: preserve request-backed tool loading * style: sort agent execution imports * fix: adapt public agent tool loaders	2026-08-30 15:38:57 -04:00
Danny Avila	c1cb591d49	🗿 feat: Add Attached Stateful Code Environments (#15352) ... * feat: add attached stateful code environments * fix: include code environment in lazy agent type * fix: harden stateful environment routing * fix: complete code environment route isolation * fix: declare code environment map type * fix: preserve configured code execution routes * fix: harden stateful environment updates * test: assert route-scoped sandbox readiness * fix: harden stateful environment lifecycle * fix: isolate migrated code sessions * test: assert route-qualified code sessions	2026-08-30 15:38:45 -04:00
Danny Avila	cd3768ed1f	🍵 feat: Continue Late Steers in Warm Agent Runs (#15357) ... * feat: continue late steers in warm agent runs * chore: bump agents sdk to v3.7.9 * fix: guard terminal steer admission	2026-08-30 11:54:17 -04:00
Danny Avila	2ae6c8aea9	🛎️ feat: Wake Agents on Background Tool Completion (#15350) ... * feat: wake agents for background tool completion * fix: isolate background completion contracts * fix: anchor background completion identity * fix: close background completion delivery gaps * fix: preserve manual background polling * fix: preserve legacy tool group identity * fix: close background wakeup identity gaps * test: type background wakeup enqueue mock * fix: preserve background completion identity * test: type activity phase fixture * test: mock phase media query * fix: arbitrate background result ownership * fix: retain tool step routing helper * test: expand phase groups before identity checks * fix: preserve background completion ownership * fix: bound durable background results * fix: type wakeup input budget export * chore: sort background handler imports * fix: persist timed-out background completions * fix: reconcile background completion ownership * fix: type background completion capabilities * fix: harden background completion terminalization * test: type missing completion evidence * fix: require evidence for completion retirement * chore: satisfy background completion static checks * test: cover automatic background completion wakeups * test: assert background wakeup agent identity * fix: wake capability-fenced trigger deliveries * test: keep capability shield fixture public * test: assert capability worker wakeup * fix: close background completion ownership gaps * test: satisfy completion lease static checks * fix: preserve artifact completion wakeups * fix: close background delivery recovery gaps * fix: simplify background receipt guidance * fix: recover dead background completion batches * fix: fence background completion recovery * fix: recheck background recovery ownership * fix: fence unpublished background continuations * test: await background message hydration	2026-08-30 08:56:41 -04:00
Danny Avila	e3ccaba5af	🛄 feat: Restore Compaction Guidance Across Continuations (#15356) ... * 🧭 feat: preserve compaction guidance across continuations * 🔧 fix: narrow persisted compaction fields	2026-08-30 08:13:24 -04:00
Danny Avila	70f735336d	🛎️ fix: Enroll Remote Agent Runs in the Generation Lifecycle (#15349) ... * fix: enroll remote agent runs in generation lifecycle * fix: close remote lifecycle ownership gaps * fix: close remote conversation drain races * fix: fence remote runs during conversation deletion * fix: reconcile remote deletion and settlement races * fix: complete owner deletion recovery * fix: preserve remote cleanup receipts * fix: consume deletion receipts before cleanup * fix: expose idempotent deletion option * chore: sort remote lifecycle imports	2026-08-30 07:14:13 -04:00
Dustin Healy	8fcab7e44f	🔄 fix: Recover Missing MCP Marketplace Catalogs (#15323) ... * fix: recover missing MCP marketplace catalogs * fix: make MCP catalog recovery passive * test: type MCP catalog recovery fixtures * fix: bound and back off passive MCP catalog recovery Passive recovery runs inline on `GET /api/mcp/tools` and its results are request-local by design, so every list request re-dialed the same cold servers with the default connection timeout. Three limits keep that cost proportional to what recovery can actually recover: - Cap the discovery timeout at 5s instead of inheriting the connection default (`initTimeout ?? 30s`); a server configured to connect faster keeps its own shorter limit. - Skip a server the config tier already marked `inspectionFailed`, leaving it to that tier's retry window rather than re-dialing it per request. - Skip a server whose declared `customUserVars` are unset, matching the gate `reinitMCPServer` applies for issue #10969 — connecting without them fails auth, so the attempt is spent for nothing. Servers that still fail discovery enter a one-minute per-process cooldown, which is what stops an unreachable server from being re-dialed by every subsequent list request. A server that recovers clears its own entry, and expired entries are swept at most once per window so the map stays bounded. Skipped servers render exactly as they did before recovery existed: present in the catalog with an empty tool list. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xr1Dabvdn1mzzyYgpJgU5B * fix: bound passive MCP recovery by deadline, key cooldowns by config Both follow-ups address the same mistake: recovery expressed its own request-level constraints in terms borrowed from other layers. `connectionTimeout` bounds one connection attempt, and `MCPConnectionFactory.discoverToolsInternal` spends it twice — once on the authenticated connection, then again in `attemptUnauthenticatedToolListing` — so capping it bounded no total this layer could reason about. Recovery now enforces its own wall-clock deadline per server with `withTimeout`, which holds however many attempts the factory makes; `connectionTimeout` is left to do only its own job, still honouring a shorter operator `initTimeout`. An attempt abandoned by the deadline disposes its own connection when it settles, and `Promise.race` keeps a handler on it, so a late rejection is not unhandled. A per-request budget now caps total recovery regardless of server count. A server is dialed only if the remaining budget can fund a full deadline; never dialing one is not evidence against it, so a skipped server records no cooldown and a later request reaches it once those ahead are cached or cooling down. Cooldown identity now includes the publication generation — the same effective-config identity the tool caches fence on — instead of just user and server name. Correcting a server's URL or transport keys a new entry, so the refetch the client issues on update is no longer skipped for up to a minute by the previous configuration's failure. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xr1Dabvdn1mzzyYgpJgU5B * refactor: keep passive MCP recovery stateless and bounded by its own work Reverts the cooldown, request budget and deadline race added in 257d5cf and fc3e3c9, and keeps only the three stateless limits. The tool cache refuses unfenced writes (`tools.ts`), and a discovery connection owns no publication generation and is disposed, so a recovered catalog cannot be retained by design. Those commits responded by building a cache-shaped memory in front of it — per-process failure state, a scheduling budget, an identity, an eviction sweep — and each round of review found another way that hand-rolled cache differed from a real one: wrong identity for configuration, wrong identity for credentials, no fairness across requests, and a limiter slot released while its network operation was still running. None of that machinery was asked for; all of it was compensation for a result the architecture does not allow keeping. Recovery is now stateless. It skips only what configuration alone proves pointless — a server the config tier already marked `inspectionFailed`, and one whose declared `customUserVars` are unset — and bounds the work itself rather than racing it, so a limiter slot is held for exactly as long as its network operation runs and the concurrency limit of three is real. The attempt timeout is not a compromise: recovery exists for a server that is reachable and authorized but whose catalog cache expired, and such a server answers tools/list well inside 1.5s. Anything slower cannot be rescued here, so failing fast costs nothing. The factory spends that value per attempt, so a server's ceiling is it times the attempts made; the constant documents that rather than hiding it behind a number tuned to today's attempt count. Consequences that were bugs are now gone by construction: every cold server is attempted on every request, so none is starved by those ahead of it, and correcting a server's configuration or credentials takes effect on the next refetch instead of waiting out a stale cooldown. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xr1Dabvdn1mzzyYgpJgU5B * fix: correct the inspection-failure skip and bound catalog fan-out Three fixes that belong to this layer; a fourth issue does not, and is described below. The `inspectionFailed` skip was too broad. `MCPServersInitializer` stores a YAML server that was unreachable at startup via `addServerStub`, which stamps `source: 'yaml'`, and only config-tier entries get the timed retry in `ensureSingleConfigServer`. Skipping every failed stub therefore hid a recoverable server from the marketplace permanently — the exact state this recovery exists to escape. It now defers only `source === 'config'`, matching what `reinitMCPServer` already does. Plugin auth is read only when some cold server actually declares `customUserVars`, and only for those servers. The common unauthenticated case no longer pays a MongoDB round trip whose result nothing can consume. Snapshot refreshes are now bounded by the same limiter as discovery. They are not local reads: both connection paths reach `fetchOrderedToolsSnapshot` and issue a real `tools/list`, so a cache reset across many servers previously burst unbounded outbound requests while discovery was capped at three. Not fixed here, because it cannot be: `connectionTimeout` does not bound discovery. It covers `connection.connect()` only, and `fetchToolsSnapshot` then applies its own `TOOLS_LIST_TIMEOUT_MS` (30s) to `tools/list`, so a server that connects fast and stalls while listing still holds its slot for that window. The factory also does not cancel a timed-out connect before starting the unauthenticated fallback. Bounding this end to end needs a deadline threaded through `MCPConnectionFactory` into both `connect()` and `fetchToolsSnapshot()`, which is a change to shared connection machinery rather than to this caller. The constant's comment now states what it does and does not bound instead of implying an end-to-end guarantee. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xr1Dabvdn1mzzyYgpJgU5B * fix: thread live-session OBO context into passive catalog discovery The merge of #15334 sources OBO tokens from the live OpenID session via request-boundary closures. Passive catalog recovery is a discovery call site too; without these options an OBO server whose stored token went stale fails recovery — the exact cold-catalog class this PR fixes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xr1Dabvdn1mzzyYgpJgU5B --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-30 06:55:30 -04:00
Danny Avila	fa913148fb	🔒 fix: Refresh MCP OBO Tokens From the Live OpenID Session (#15334) ... * 🧊 fix: Inline-refresh OpenID session tokens at MCP OBO call time Resolves the walk-away failure mode where MCP tool calls using OBO auth fail with "No valid OpenID access token is available for OBO exchange" after a user idles past their access-token lifetime. The strategy-time snapshot on `user.federatedTokens` could expire mid-stream before `resolveOboToken` ran, while `req.session.openidTokens` carried a still- valid (or refreshable) token that nothing read. - New OpenIDSessionRefresh service: per-user single-flighted closure that reads `req.session.openidTokens` at OBO time and inline-refreshes via `openid-client.refreshTokenGrant` when expired (30s skew), persisting via `req.session.save()`. No cookie writes (headers already flushed). - `resolveOboToken` gains a required UpstreamTokenProvider parameter (typed as `() => Promise<OIDCTokens | null>`, reusing the shared shape from @librechat/data-schemas). Compile-time guarantee that every call site is updated. - New `session_refresh_failed` OboTokenResolutionReason distinguishes "session expired and IdP rejected refresh" from "no upstream token ever existed." - `req` threaded through createMCPTool/createMCPTools/createToolInstance to construct the closure with captured request, plus fail-closed guards in MCPConnectionFactory.getOboTokens and MCPManager.callTool when the closure isn't plumbed. - Startup warning in MCPServersInitializer when OBO is configured but OPENID_REUSE_TOKENS is unset (the strategy populating user.federatedTokens is only registered under reuse, so OBO would fail every call without it). Tests: 16 new in OpenIDSessionRefresh.spec.js; obo.spec.ts extended for the new param + error reason; wiring smoke tests in MCPManager, MCPConnectionFactory, MCPServersInitializer, and MCP.spec.js. * 🛡️ fix: Harden OBO inline-refresh against token type and session edge cases - Token-preference asymmetry: live-token reuse and expires_at derivation now strictly gate on the access_token, not the id_token. Added a required `tokenPreference` parameter on isLiveSessionTokenStillValid, buildOIDCTokensFromSession, and createOpenIDSessionTokenProvider so every call site is explicit. Dropped the bogus id_token-exp fallback in performIdpRefresh — id_token TTL is governed by IdP session policy and would mark a short-lived access_token reusable past its real lifetime. - Missing req in /reinitialize route: the manual reconnect endpoint now forwards req into reinitMCPServer, so OBO servers can build a session-aware upstream-token closure instead of failing with missing_upstream_token. - Single-flight key collisions: composed key as tenantId:openidIssuer:openidId:sessionId via getSingleFlightKey. Concurrent calls in the same session still coalesce; separate sessions never share an in-flight refresh, preventing refresh-token rotation from breaking sibling sessions and preventing cross-tenant token crossover when distinct users share an IdP sub. - Opaque access token reuse): persist accessTokenExpiresAt (unix seconds, from tokenset.expires_in) on each refresh AND on initial login / SPA refresh in setOpenIDAuthTokens. New getAccessTokenExp helper falls back to it when the access token isn't a JWT, avoiding redundant inline refreshes for Microsoft Graph and Auth0 default audiences. - Log hygiene: the single-flight key (containing sessionId, openidId, openidIssuer, tenantId) is now SHA-256-hashed in the "Joining in-flight refresh" debug log. Preserves cross-line correlation via a 12-char prefix without leaking credential or PII material. Documented req.session.openidTokens shape contract via JSDoc typedef so the new accessTokenExpiresAt field has a discoverable home alongside the existing accessToken/idToken/refreshToken/expiresAt/lastRefreshedAt. Tests: OpenIDSessionRefresh.spec.js up to 30 passing (added coverage for opaque-token reuse, JWT-access-token-exp fallback, no-id_token-fallback regression, cross-session no-coalesce, persistence on refresh, and a guard against stale accessTokenExpiresAt carryover). AuthService.spec.js adds two cases covering accessTokenExpiresAt persistence on login. mcp.spec.js (route) gains a regression test asserting req flows into reinitMCPServer. * 🔍 fix: Detect OBO-only MCP admin config overrides Admin Config overlays for YAML-defined MCP servers compare only ADMIN_CONFIGURABLE_FIELDS to decide whether to lazy-init a config-tier override. The OBO config field was added after that fingerprint list, so an override that only added or changed `obo` was treated as unchanged YAML and skipped. Include `obo` in the admin-configurable field list and add a regression test for an OBO-only override. * 🔊 fix: Mock MCP OAuth timeout in SDK integration test MCPConnectionFactory.attemptToConnect reads mcpConfig.OAUTH_HANDLING_TIMEOUT when building the OAuth connection timeout. The SDK OAuth integration test mocked mcpConfig without that field, which made the timeout calculation produce NaN and caused the test to fail before the OAuth refresh/start path completed. Add OAUTH_HANDLING_TIMEOUT to the test mock. * ♻️ refactor: Pass OBO upstream-token closure into MCP instead of req Build the OpenID upstream-token provider at the request boundary and thread only the closure through MCP handling, so the MCP service layer no longer receives the raw Express request. The closure still reads/refreshes the live session at tool-call time, preserving the walk-away recovery. - Drop `req`/`capturedReq` from createMCPTools, createMCPTool, reconnectServer, createToolInstance, and reinitMCPServer; forward `upstreamTokenProvider` instead. Closure is constructed in loadTools, loadToolDefinitionsWrapper, and the reinitialize route, where req/res are in scope. - OBO: fall back to user.federatedTokens when the provider yields no live session, so OIDC remote-agent calls (verified bearer, no session) still work. - Inline refresh: mirror a rotated refresh token to the refreshToken cookie via a shared setRefreshTokenCookie helper, guarded by !res.headersSent (no-op on the streaming path; session copy stays authoritative). - Single-flight: hydrate a joining request's own session from the resolved tokens so a later OBO call doesn't replay a rotated-away refresh token. Addresses owner feedback and three review findings. * 🔒 fix: Recover OIDC refresh-token rotation after SSE OBO refresh When an inline OBO refresh rotates the OpenID refresh token after SSE headers have already been sent, the browser refreshToken cookie cannot be updated. Store a short-lived encrypted bridge from the stale cookie token to the rotated token so /api/auth/refresh can recover after express-session loss. Use the signed openid_user_id cookie to load user context for bridge validation, retry only on invalid_grant, and delete the bridge only after the bridged refresh succeeds. * 🔨 fix: hydrate joined OIDC refresh sessions with stable refresh tokens Update single-flight OIDC refresh joiners whenever refreshed access token state changes, even if the IdP keeps the refresh token unchanged. This prevents joined requests from retaining stale accessToken or accessTokenExpiresAt values and redundantly refreshing later in the same run. * 🌉 Persist OIDC refresh-token recovery bridges in MongoDB Store SSE OBO refresh-token recovery bridges in MongoDB instead of process-local memory so /api/auth/refresh can recover after worker restarts or cross-worker routing. Derive bridge expiry from REFRESH_TOKEN_EXPIRY so the recovery window matches the stale refreshToken cookie it repairs, and delete bridges after successful recovery. * 🤝 Coordinate OIDC inline refreshes across workers Add a short-lived Mongo-backed refresh-flight record so concurrent OBO refreshes for the same OpenID session do not redeem the same rotating refresh token on different workers. The winning worker performs the IdP refresh and stores an encrypted result; joiners wait for that result, hydrate their request session, and return without calling the IdP. * ⚓ Keep OpenID marker cookies aligned on inline refresh Refresh token_provider and openid_user_id with the same expiry as the rotated refreshToken cookie when an inline OBO refresh can still write headers. Share the marker-cookie writer with the normal OpenID auth refresh path so the fallback /api/auth/refresh branch continues to recognize valid OpenID refresh tokens after session expiry. * 🔑 fix: include refresh token in OIDC local refresh flight key Key the process-local OIDC refresh coalescing by the current session refresh token, matching the Mongo-backed flight key. This prevents a request with a newly rotated token from joining an older pending refresh and inheriting its failure/result. * 🌉 fix: store OIDC refresh bridge without cookie response Treat missing or non-cookie responses like headers-sent streaming responses during inline OIDC refresh. When the IdP rotates the refresh token and cookies cannot be written, persist a recovery bridge so a later /auth/refresh can recover after session expiry. * 🫙 fix: preserve stale OIDC cookie bridge key Track the refresh token last written to the browser cookie separately from the current session refresh token. When inline OIDC refreshes rotate tokens without a writable response, keep bridging from the browser-stale token directly to the latest session token. * 🙌 fix: keep OIDC bridge recovery success on cleanup failure Make refresh-token bridge cleanup best-effort after a bridged OIDC refresh succeeds. A transient delete failure now logs a warning but does not convert the already-refreshed session and cookies into a 403 response. * 📦 test: Exclude RefreshTokenBridge from tenant-isolation coverage Add RefreshTokenBridge to the tenant-isolation coverage allowlist because refresh bridge lookups run during unauthenticated OpenID refresh recovery. The controller first recovers user context from the signed OpenID marker cookie, then the bridge methods apply explicit user and tenant filters. Ambient tenant isolation would bind this recovery path to request-local tenant context that is not available at the point the stale cookie is being resolved * ⚡ Fix OpenID refresh flight retry and marker hydration Allow failed OpenID refresh flights to be reclaimed immediately instead of pinning transient errors. Preserve the browser refresh-token marker when joined refreshes hydrate session tokens from a shared flight result. Stabilize AuthService tests by isolating mocked module imports from prior suites. * 🛠️ fix: centralize OBO identity scoping Add shared auth identity helpers for app user ids, OpenID subjects, tenant ids, and normalized OpenID issuers. Thread a non-placeholder-visible OBO identity context from the real request user through MCP connection, tool-call, reinit, and refresh paths. Keep tenantId and openidIssuer out of createSafeUser so MCP user placeholders do not expose those fields. Scope OBO token cache and in-flight exchange keys by tenant, issuer, OpenID subject, scopes, and a SHA-256 hash of the upstream assertion. This prevents cross-tenant/cross-issuer collisions and avoids reusing tokens minted from stale rotated assertions. Use the shared identity helpers for OpenID refresh-flight keys and refresh-token bridge recovery records so related OBO refresh paths share the same identity normalization rules. The helper is intended for auth-boundary and credential-cache code, not as a blanket replacement for ordinary app user id ownership checks. * 🛠️ fix: preserve OIDC refresh-token sync on save failures Sync OpenID refresh-token cookie/bridge state before persisting the session so a transient session-store failure cannot lose an IdP-rotated refresh token. Also trigger sync when the session refresh token differs from the browser refresh-token marker, not only when the current grant rotates the token. This lets later writable refreshes repair stale browser cookies left behind by SSE refreshes. Route refresh bridge identity through the shared identity helper with the threaded OBO identity context, falling back to request/user context when needed. Add regression coverage for session-save failures, stale browser cookie repair, non-writable bridge storage, and shared-helper identity fallback. * 🛠️ fix: keep OIDC refresh bridge during recovery grace After successful bridged refresh recovery, re-store the stale-cookie bridge with a short grace TTL instead of deleting it immediately. This lets parallel /api/auth/refresh requests that already sent the stale browser cookie recover before they can observe the first response's Set-Cookie. Retarget the bridge to the refresh token returned by the bridged retry so B-to-C refresh-token rotation remains recoverable. The grace TTL is parsed with math() and defaults to 60s, which shrinks the replay window from the original REFRESH_TOKEN_EXPIRY bridge lifetime to the short recovery grace period. Remove the now-unused explicit bridge delete path from the service and data-schemas method surface. Add regression coverage for grace re-store, identity symmetry, retry failure behavior, and same-key upsert replacement. * 🛠️ fix: fail closed on OBO MCP user identity mismatch Add an OBO-specific guard before MCP tool execution that requires the effective invocation user and captured request user to both have ids and to match. This prevents OBO tool calls from falling back to a separate configurable.user_id identity after request-bound OBO context has already been captured. Keep the existing user id fallback behavior for non-OBO MCP calls. Tests cover mismatched OBO users, missing user ids, and the matching-user path ignoring a conflicting configurable.user_id. * 🛠️ fix: Guard OpenID bridge retry user identity Extract the shared OpenID refresh/user-resolution flow in AuthController so the normal refresh path and bridge-recovery retry use the same grant, claims, issuer, user lookup, and diagnostic logging code. Preserve the existing path-specific behavior: the normal path still owns migration updates and 401 login redirects, while the bridge retry still falls through to the existing 403 invalid-token response. Add a bridge-recovery guard that rejects retry results whose resolved user id differs from the signed openid_user_id cookie before issuing tokens or re-storing the grace bridge. Cover both the successful matching-user recovery and the mismatched-user rejection. * 🛠️ fix: type-safety polish on OBO data layer Replace refresh token bridge query/update Record<string, unknown> usage with typed Mongoose FilterQuery and UpdateQuery definitions. Harden OpenID marker cookie JWT expiry handling by converting refresh expiry milliseconds to integer seconds and rejecting invalid or non-positive durations. Add focused CSRF tests for fractional refresh expiry values and invalid expiry configuration. * 🛠️ fix: Bind OpenID session tokens to authenticated identity Stamp OpenID session token state with the LibreChat user id, OpenID subject, tenant id, and normalized issuer when tokens are stored. Fail closed before OBO inline token reuse/refresh when the session token identity does not match the current authenticated identity, preventing a stale or mixed Express session from supplying another user's upstream assertion. Also validate the normal /api/auth/refresh session-token reuse shortcut against the signed marker-cookie user before returning cached session tokens. Note: sessions created before this change carry no identity stamp and are treated as a mismatch. This is self-healing — the reuse path forces a full IdP refresh (which re-stamps the session) and the OBO path throws, surfacing as a one-time re-authentication for active OBO users at deploy time. The session re-stamps within one session lifetime (SESSION_EXPIRY, default 15 min). * 🛠️ fix: Recover OpenID refresh token drift Prefer the browser refresh-token cookie when it differs from the server-side OpenID session state, and force a real IdP refresh in that case instead of reusing stale session tokens. Store a short-lived refresh-token bridge when inline OBO refresh writes a rotated browser cookie but session persistence fails, so follow-up refreshes can still recover from the old token. Keep the bridge grace TTL centralized in RefreshTokenBridge so both recovery paths use the same env-backed value. Note: drift is measured against the last-synced browserRefreshToken marker, so the SSE path (intentionally stale cookie, authoritative session) does not false-positive. Sessions predating the marker have no browserRefreshToken; for those, drift falls back to comparing the cookie against the session refresh token and prefers the cookie on difference. This is the same self-healing pre-change-session window as the identity binding fix and re-syncs within one session lifetime. Tests cover cookie/session drift selection, reusable-session bypass on drift, bridge storage after session-save failure, and the shared bridge constant wiring. * 🛠️ fix: Harden OBO token caching and expiry handling Reject malformed OBO grant responses before writing them to the exchanged-token cache so a missing access_token cannot poison the cache. Store absolute expires_at values with cached OBO tokens and ignore legacy cache entries without usable expiry metadata. This keeps cached-token freshness based on the token’s real remaining lifetime instead of reusing the original relative expires_in on cache hits. Move OBO expiry normalization and skew helpers into packages/api and use them from both the JS exchange service and the TS MCP resolver. Apply a 30-second safety margin with a one- second floor for short-lived tokens, covered by direct helper tests and caller-level regression tests. Tests: - packages/api: npm run build - packages/api: npx jest src/mcp/oauth/expiry.spec.ts src/mcp/oauth/obo.spec.ts - api: npx jest server/services/OboTokenService.spec.js * 🛠️ fix: Harden OBO refresh-token bridge lookup and indexing Reuse getValidOpenIDReuseUserId for the bridge-recovery user lookup in refreshController instead of re-verifying openid_user_id inline. The shared helper enforces the JWT_REFRESH_SECRET presence check and a strict typeof payload.id === 'string' guard, rejecting tokens whose id claim is present but not a string (e.g. a numeric id) that the inline check accepted. Fail closed on issuer mismatch in getRefreshTokenBridge. Both the stored and the expected issuer are now normalized and compared for equality, so a bridge is recovered only when both sides agree (both absent, or both present and equal after normalization). Previously the check was skipped whenever the stored issuer was absent, allowing recovery across mismatched issuer context. Drop the unused {oldRefreshTokenHash, userId, tenantId, openidIssuer} index and the openidIssuer field on RefreshTokenBridgeQuery. The data-layer filter only queries the 3-field {oldRefreshTokenHash, userId, tenantId} index; the issuer is verified in application code, not the query. Hoist the repeated model accessor into getRefreshTokenBridgeModel. Note: issuer is now load-bearing for recovery. A bridge stored with an issuer recovers only when the lookup supplies a matching issuer; the recovery lookup reads user.openidIssuer via AUTH_REFRESH_USER_PROJECTION (an exclusion projection that retains the field). If a user's persisted openidIssuer is empty while the stored bridge has one, recovery fails closed (falls through to normal re-authentication) until the bridge TTLs out — no security regression. Tests cover invalid signed-cookie payloads bypassing the bridge, both asymmetric issuer-presence cases, issuer normalization before comparison, and an index-alignment assertion guarding against re-adding the dropped index. * 🛠️ fix: Degrade OBO discovery on token resolution failures Catch expected OboTokenResolutionError failures during MCP tool discovery and fall back to unauthenticated tool listing instead of aborting discovery. This keeps discovery aligned with the existing unauthenticated listing behavior while preserving unexpected errors as real failures. Also correct OBO tool-call freshness comment and tighten the OBO trust-check permissions type to the existing role permission shape. Tests: - npx jest src/mcp/__tests__/MCPConnectionFactory.test.ts --runInBand --coverage=false - npx jest src/mcp/oauth/obo.spec.ts --runInBand --coverage=false * 🛠️ fix: tighten OBO tool-call errors, bridge logging, and flight typing Move resolveToolCallUserId inside the tool-call try/catch so an OBO identity mismatch surfaces with serverName/toolName context and the standard tool-call-failed message instead of an opaque bare Error. Raise the refresh-token bridge lookup failure log from debug to warn so transient infrastructure failures on the unauthenticated /api/auth/refresh path are observable, and guard the message access against non-Error values. Replace the unknown+cast in isDuplicateKeyError with a hasErrorCode type predicate so the duplicate-key check reads error.code without an assertion. Preserve real math/isEnabled in the MCPConnectionFactory test mock (mock only processMCPEnv) so mcpConfig timeouts no longer resolve to NaN, fixing the TimeoutNaNWarning that masked slow OAuth retry behavior. * 🧪 fix: Restore the Flight Uniqueness Index and Buffer the Graph Cache TTL Two CI failures on the merge, both in suites this environment cannot run (their MongoDB binary download is blocked). `GraphApiService.spec.js` still asserted the unbuffered TTL. Graph tokens route through the same `getTokenCacheTtlMs` as the OBO and openidStrategy caches, so the entry now expires 30s before the credential does. `openidRefreshFlight.spec.ts` dropped the database between tests, which takes the indexes with it, and Mongoose builds them only once when the model is compiled. Whether the unique `key` index survived into a test was a race with that one-time build. Without it a second `create` inserts instead of raising a duplicate-key error, so every worker believes it won the flight — the mutual exclusion the file exists to prove. Indexes are now rebuilt after each drop, which also makes the reclaim and complete cases reach those paths for the right reason. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * fix: address OBO review findings * 🔐 fix: Install Bridge Indexes and Carry OBO Through Assistant Recovery Two findings from the Codex pass on d08c82f0d. The refresh-token bridge relied on Mongoose auto-indexing for both of its indexes, and `MONGO_AUTO_INDEX=false` is a supported deployment setting. A bridge holds an encrypted refresh token and the TTL index is the only thing that ever deletes one, so under that setting they would accumulate for the life of the collection while concurrent upserts lost the compound uniqueness the filter assumes. Installed before the first write, matching the flight methods and the session and schedule methods before them. `recoverServerTools`, the assistant create/update path that reruns `reinitMCPServer` when a referenced server's catalog and connection snapshot are both missing, was the last reinit site not carrying the upstream-token closure. For an OBO server the factory rejects the connection outright, so the assistant write failed with unavailable MCP definitions. It now builds the provider at that request boundary like the other entry points; assistant writes have no `res`, so a rotation there falls back to the recovery bridge. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * fix: harden OBO refresh coordination * ⏳ fix: Keep Elapsed Expiries Elapsed and Revoke the Superseded Session Two of the five findings from the Codex pass on fcdc15885 — the two that are defects in code this branch introduced rather than design questions about the bridge. `getSkewedTokenExpiresAtMs` floored every result at a second in the future, including an expiry the provider had already declared elapsed. An exchange answering `expires_in: 0` or a past `expires_at` was handed to the MCP connection stamped valid for another second, which only moves the failure downstream. The floor now applies to a lifetime that is still live, which is what it was for; an elapsed one stays elapsed so the caller rejects it. Same for the cache TTL, which falls back to the elapsed-credential floor. Bridge recovery left the stale token's durable Session behind. Only the token it recovered through was passed as `existingRefreshToken`, so that one's session was replaced while the token the browser actually presented kept its record until its original expiry. That record, with the marker cookie still bound to it, is what authorizes local image access for OpenID users — so a copy of the stale cookie outlived the rotation it had lost. Revoked explicitly on successful recovery. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * fix: close OBO refresh review findings * 🎟️ fix: Carry the Bridged Token Through a Non-Rotating Recovery Bridge recovery passes the browser's stale token as `existingRefreshToken` so the durable Session naming it is the record replaced. That also makes the stale token the fallback the installed session and the refresh cookie use when a tokenset carries no `refresh_token` of its own, which holds only while the recovery grant rotates. An IdP that answers that grant without rotating sends the browser back to the very token the bridge exists to retire: `storeOpenIDSession` installs and deletes the same stale record in one call, and the cookie is rewritten to a token the IdP already rejected — a sign-out on the next refresh. The grace bridge one line above already guards this with `|| bridgedRefreshToken`; the resolved tokenset now does the same, so leader and followers alike publish the recovered token. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * 🚫 fix: Reject an OBO Exchange That Returns an Expired Credential Preserving an elapsed expiry through the skew helper only helps if something acts on it, and nothing did: `MCPManager.callTool` checks the access token and nothing else before setting the Authorization header, so a credential the IdP declared spent still went downstream to fail there. It is rejected at the exchange now, where the reason is known, and retryably — the exchange itself worked, so a fresh grant can succeed. Completes the elapsed-expiry change in cb26a6f7d, which made the stamp honest without giving anyone a reason to look at it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * fix: close OpenID refresh review findings * fix: coordinate OpenID refresh entry points * chore: sort OpenID flight imports * fix: fence OpenID refreshes during logout * fix: close OpenID logout publication races * fix: narrow completed refresh flight * test: cover bridge cleanup failure after ownership loss The compensating delete in storeRefreshTokenBridgeWithLease swallows its own failure so the lease error stays the one the caller sees. Nothing asserted that, so removing the inner catch left every suite green while callers began receiving the cleanup error instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * fix: compensate OpenID bridges only on proven ownership loss The post-write lease assertion deletes the bridge it just published when it throws, but it threw for two different reasons: a coordination record that is no longer ours, and a coordination read that simply failed. Treating the second as the first destroys the only mapping from the token the browser still holds to the one the IdP already rotated to, so a transient Mongo error on the headers-already-sent path signed the user out. Tag the ownership error where the lease raises it and compensate only for that, preserving the bridge whenever ownership is merely undetermined. A preserved bridge stays behind the logout revocation fence, so the safe default costs nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * fix: reject spent OpenID refresh results Two ways a refresh could report success while handing back a credential nothing can use. An inline refresh carries the previous id_token forward when the IdP omits one on rotation, so tokenset.id_token is not necessarily freshly issued. setOpenIDAuthTokens applied its freshness guard only to the session copy and took tokenset.id_token unconditionally, so /refresh returned an expired bearer even though the grant produced a usable access token. Skip it only when it is provably expired: an id_token whose expiry cannot be read stays preferred, since access_token may be opaque or scoped to another audience. normalizeExpiresIn preserves a zero or negative lifetime rather than discarding it, so a grant declaring an already-spent access token still published, rotating the refresh token and returning a token every freshness check rejects. Each OBO call then repeated the grant. Reject an elapsed lifetime before publishing; an unknown lifetime still publishes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * fix: harden OpenID refresh publication * fix: keep identity and results intact through OpenID refresh cleanup Two follow-ons from the last round's fixes. Stripping an expired carried-forward id_token from the refresh result removed the only identity material a rotation without id_token leaves behind. The result is rebuilt by buildOIDCTokensFromSession, so it carries no provider claims() either, and getTokenClaims accepts only those two — bridge recovery failed with "no usable identity claims" before setOpenIDAuthTokens could hand back the fresh access token. The stripped token now travels in a non-enumerable marker, alongside the existing browser and predecessor markers, which identity resolution reads and the authentication response never sees. The lease drained a pending renewal by awaiting it in finally, so a transient coordination failure there threw from finally and replaced the operation's result. The refresh had already settled and published, so the caller saw a failure on credentials that had rotated. Proven ownership loss is recorded on ownershipLost and checked before the return, so the drain has nothing to add but noise; it now absorbs and logs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * refactor: fence OpenID recovery publication * fix: close OpenID publication transaction * refactor: make OpenID publication transactional * fix: satisfy OpenID publication type checks * fix: fence OpenID session publication * fix: authorize OpenID refresh publication * fix: bind OpenID replay generations * fix: fence OpenID response generations * fix: authorize OpenID token delivery * fix: linearize OpenID publication delivery * chore: sort OpenID refresh flight imports --------- Co-authored-by: J.C. Bartle <jcbartle@users.noreply.github.com> Co-authored-by: jbartle <jbartle@rand.org> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: jcbartle <7274202+jcbartle@users.noreply.github.com>	2026-08-29 23:55:58 -04:00
Danny Avila	10981b8637	🍱 feat: Guide Compaction With a Bounded Semantic Index (#15340) ... * 🧭 feat: guide compaction with semantic context * 🛡️ fix: fail closed on semantic intent collisions	2026-08-29 17:53:04 -04:00
Danny Avila	773127bff2	🎠 refactor: Route Every Event Actor Turn Through One Lifecycle (#15325) ... * refactor: unify Event Actor turn lifecycle * fix: retain Event Actor fence ownership * fix: preserve mixed-version actor suspension safety	2026-08-28 17:30:23 -04:00
Danny Avila	3fa33b740b	🛫 refactor: Promote Generation Protocol V2 Automatically (#15324) ... * refactor: promote generation protocol v2 automatically * fix: remove unused protocol import	2026-08-28 17:17:09 -04:00
Danny Avila	77c2a51cf3	🪧 fix: Advertise Detached Event Actor Support from the Generation Store (#15322) ... * fix: activate detached event actions automatically * fix: shield detached terminal generations * perf: share capability availability index * style: flatten capability status selection * fix: preserve mixed-version lifecycle shells * fix: complete detached action store adapters * fix: close mixed-version capability races * fix: honor legacy capability success	2026-08-28 16:33:23 -04:00
Danny Avila	487984193a	🪂 feat: Make Event Actor Detached Actions Durable (#15307) ... * feat: make Event Actor detached actions durable * fix: align detached actor checks with merged base * fix: close detached actor lifecycle gaps * fix: break detached action outcome cycle * fix: deepen detached action ownership * fix: fence detached launch handoffs * fix: preserve detached rollout fencing * test: type detached recovery failure * fix: close detached ownership handoffs * fix: stage detached action activation * fix: persist detached terminal retries * fix: fence detached durability ownership * fix: fence trigger lane publication * fix: fence detached transition generations * fix: retire failed detached predecessors * fix: close detached rollout consumers * test: type detached retry turns * fix: close detached ownership gaps * fix: fence detached action resumes * fix: fence prior-head event actor resumes * fix: preserve legacy event actor resumes * fix: align detached resume boundaries	2026-08-28 14:21:04 -04:00
Danny Avila	c06b09c945	🤖 feat: make Event Actor HITL durable (#15305) ... * 🤖 feat: make event actor HITL durable * 🤖 fix: break event actor outcome cycle * 🤖 fix: close durable actor terminal races * fix: harden durable event actor recovery proofs * fix: close event actor resume publication races	2026-08-28 08:05:11 -04:00
Danny Avila	6fba51a35a	🫆 feat: Fingerprint Agent Context for Zero-Read Warm Continuation (#15299) ... * feat: add context-fingerprinted warm continuation * fix: preserve manual skills across warm turns * perf: keep ordinary agent initialization neutral * fix: fingerprint complete agent topology * fix: close warm context replay gaps * fix: preserve lazy subagent memory scope * perf: fingerprint only model-bound memory * fix: project lazy memory capability * style: sort warm continuation imports * fix: make warm continuation reconstructable * fix: preserve lazy skill isolation * fix: retain scoped skill semantics	2026-08-27 23:01:14 -04:00
Danny Avila	a5fd00cd7a	🎼 feat: Unify Agent Turn Execution Behind One Immutable Plan (#15296) ... * feat: unify agent turn execution planning * fix: preserve automatic subagent activity handles * fix: gate subagent wakeups for ephemeral parents * fix: scope subagent wakeup guidance by agent	2026-08-27 22:26:46 -04:00
Danny Avila	f4efef4b3a	♟️ fix: Settle First Bound Actor Events (#15300)	2026-08-27 19:06:41 -04:00
Danny Avila	03fa98eb37	🕸️ feat: Complete Keenable Web Search Provider (#15288) ... * feat: wire Keenable web-search provider into config, schema, and UI Keenable landed as a search provider in @librechat/agents (#285, shipped in 3.2.58+), but LibreChat did not yet expose it. This adds the config/schema/UI glue so it can be selected, mirroring the existing Tavily provider. - data-provider: add `keenable` to SearchProvider type + SearchProviders enum, keenableApiKey/keenableApiUrl schema fields, and a keenableSearchOptions block (maxResults, site, attributionTitle, timeout). - data-schemas: register keenable in webSearchAuth.providers and default the key/URL placeholders in loadWebSearchConfig. - api/web: pass keenableSearchOptions through to the provider and handle Keenable's keyless model. Unlike other providers it authenticates with no key (the public endpoint), picking up an optional key/URL when set; the URL override is SSRF-preflighted like other user-provided URLs. - client: add Keenable to the provider dropdown with an optional API-key input. - docs: document KEENABLE_API_KEY/KEENABLE_API_URL in .env.example and a webSearch example in librechat.example.yaml. - tests: keyless + keyed auth resolution, config defaults, and schema parsing. * fix: ESLint no-unused-vars and clarify Keenable yaml example - Remove the now-unused RerankerTypes import in data-schemas web.ts (the lint job runs with --max-warnings 0 on changed files, so this latent warning failed CI once the file was touched). - Note in the librechat.example.yaml Keenable stanza that a scraper (and reranker) is still required for web search to load, and include a Firecrawl scraper in the example. * chore: fix import order drift (sort-imports) * feat: add Keenable as a keyless scraper and select it without a pinned provider The Keenable scraper landed in @librechat/agents#337, so wire the scraper category the same way the search provider already is: `scraperProvider: keenable` reads pages through Keenable's public fetch endpoint with no key (a key only lifts rate limits, and the endpoint is overridden with KEENABLE_FETCH_URL). Paired with `rerankerType: none` this makes a fully keyless web-search stack possible for the first time. Also closes the Codex finding on this PR: because none of Keenable's auth fields are required, the generic auth loop skips it whenever it isn't pinned, so a key submitted through the API-key dialog (which cannot pin a provider) left the providers category unauthenticated. Keenable is now selected in that case, gated on one of its values actually being present so installs that configured nothing keep their current behavior. The scraper gets the same fallback, additionally gated on Keenable being the resolved search provider, so it never silently scrapes for another provider. * fix: select the Keenable scraper from a supplied key, not only for Keenable search The API-key dialog submits credentials and cannot pin a provider, so choosing Keenable as the scraper while search stays on Serper/SearXNG/Tavily had no effect: the unpinned-scraper fallback required Keenable to also be the resolved search provider. A supplied Keenable value now triggers it as well, which is the only signal the dialog can send. The fallback still runs only when no keyed scraper authenticated, and with neither trigger the category stays unauthenticated, so a deployment that never configured Keenable is unaffected. Note the fully keyless choice still cannot be expressed through the dialog: Keenable's key is optional, so picking it with no key submits nothing at all. librechat.example.yaml now documents pinning scraperProvider: keenable for that case. * style: Sort Keenable imports * fix: Harden Keenable auth resolution * fix: Preserve Keenable selection intent * fix: Fail closed on invalid web search auth * fix: close keenable auth gaps * style: sort web auth imports * fix: preserve web search selection integrity * fix: isolate web search auth ordering * fix: silence expected credential misses * chore: bump agents sdk * fix: preserve web search preference ownership * fix: forward cleared Keenable endpoint * style: sort web search hook imports * fix: require intent for credential clears --------- Co-authored-by: Ilya Bogin <ilya.bogin@keenable.ai>	2026-08-27 14:29:20 -04:00
Danny Avila	8ba76507ae	🧷 fix: Verify Scheduled HITL State Before Pausing (#15284) ... * fix: require durable storage for scheduled HITL * fix: verify scheduled HITL checkpoints before pause * fix: narrow scheduled HITL capability checks * fix: honor ask tool filters in schedule preflight * fix: enforce durability across scheduled HITL resumes * fix: scope scheduled HITL admission to active tools * refactor: move scheduled HITL admission to TypeScript * fix: close scheduled HITL admission gaps * fix: bind scheduled pauses to current checkpoint * fix: cap pending actions at binding deadline * fix: reject expired pending actions * fix: guard approval deadlines atomically	2026-08-27 11:16:24 -04:00
Danny Avila	de59da9636	🎟️ refactor: Require Credentials for Local Image Access by Default (#15252) ... * 🔐 fix: Protect Local Image Access by Default * 🔐 fix: Scope Image Authorization to Active Sessions * 🧹 style: Format Image Authorization Checks * 🛡️ fix: Harden Image Avatar Authorization * 🧭 style: Sort Image Authorization Imports * 🔐 fix: Close Image Authorization Review Gaps * 🧭 fix: Normalize Stored Avatar Base Paths * 🏢 fix: Resolve Tenant Assistant Image Policy * 🛂 fix: Enforce Effective Image Access Policy * 🧹 style: Flatten Assistant Config Selection * 🧷 fix: Preserve Image Access Compatibility * 🪪 fix: Make Image Sessions Revocable * 🏗️ fix: Move Image Session Policy Into API	2026-08-27 09:55:27 -04:00
Danny Avila	41ba808e6f	🧅 feat: Reveal Subagent Thread History and Event Detail on Demand (#15283) ... * ✨ feat: Refine Subagent Thread Activity UX * test: align subagent completion lifecycle * chore: satisfy subagent panel static checks * fix: harden subagent activity pagination * fix: preserve paged subagent history * fix: surface unrecoverable activity boundaries * fix: rebase paged activity after live advances * fix: bound child activity history state * fix: preserve rebased child history order * fix: retain unavailable child history boundary * fix: fence child history cursor generations * fix: preserve subagent history continuity * fix: retain live turns during history rebase * fix: close subagent history edge cases * fix: harden subagent history phase transitions	2026-08-27 08:56:06 -04:00
Danny Avila	44edcbe014	🧾 feat: Store Durable Event Actor Receipts (#15265) ... * 🧾 feat: Store durable event actor receipts * 📊 fix: Scope actor delivery metrics * 🧱 fix: Close durable receipt recovery gaps * fix: serialize actor action admission * fix: retire terminal batch members * fix: close actor terminal recovery races * fix: reclaim lanes after terminal receipts * fix: close actor receipt recovery gaps * fix: preserve ambiguous legacy handling * fix: recover legacy actor outcomes * fix: admit leased actor actions * fix: serialize actor admission recovery * fix: close actor recovery races * fix: token-fence actor admissions * fix: preserve mixed-version actor fences * fix: harden actor rollout and metrics * fix: gate durable actor receipt rollout * fix: enforce base actor receipt rollout * fix: align experimental actor rollout	2026-08-27 06:00:43 -04:00
Danny Avila	68fc46a055	🪢 feat: Resume Bound Event Actors from Checkpoint Forks (#15227) ... * feat: resume event actors from checkpoint forks * fix: fence event actor checkpoint uncertainty * fix: satisfy event actor type contracts * fix: make actor reconciliation recoverable * fix: preserve event actor lifecycle transitions * fix: fence event actor lifecycle outcomes * fix: enforce event actor lifecycle ownership * fix: retain event actor settlement proof * 🔒 fix: Retain Event Actor Receipts Through Repair and Bound Their Journal Repair and compensation deleted the reconciliation row they resolved, which was the only durable proof that the invocation had already applied an external action. A delayed duplicate owner could then reacquire the same invocation id and repeat that action. Both resolutions now retire their receipt to `settled` and record how it settled, so the same-id tombstone survives; `history_repaired` and `action_compensated` still force a cold rebuild. Compensation undoes the effect without re-authorizing the delivery, so a legitimate retry must arrive under a new invocation id. A retried repair converges on its own receipt. Bound the journal so a long-lived actor cannot grow its conversation document without limit: a new fence is admitted only when no active lifecycle row exists, so a capped push can evict nothing but the oldest settled receipts. Stop shipping the unbounded source payload on every bound-child continuation. It rode the delivery body regardless of the feature flag while the sibling `fire` body deliberately sends event identity alone, so a large webhook payload could push a previously working delivery past the chat route's body limit. The actor binds an invocation from identity and never builds the prompt from it. Blank the positional token map on warm continuations. It is derived from the full DB history, while a warm run executes on checkpoint-restored state, so its indices address different messages and the pruner never recounts them — misattributing cached counts to the wrong messages in both directions. Keep the replaced-claim exit on its cleanup path when preserving reconciliation fails: the committing CAS already left a blocking row, so the failed status upgrade costs provenance, not safety. * 🧪 test: Pin the Warm Continuation's Map/Summary Asymmetry Give the warm-continuation client test a populated token map and a real cross-run summary so its assertions bite: the positional map must arrive blank (checkpoint-restored state no longer matches DB-derived indices, and the pruner never recounts a populated entry) while `initialSummary` must pass through unchanged — it rides the system tail and summarizes pre-boundary turns that were excluded from the very history the committed checkpoint was built from, so blanking it would silently drop context no warm run can recover. * ⚖️ fix: Honor Compensation in Settlement and Age-Bound the Receipt Journal A compensated receipt still tombstones its invocation id, but its external effect was explicitly undone — the terminal handler nonetheless replayed every settled lifecycle's stored action as authoritative and settled the public outcome as applied, telling an action-aware source the operation stands and suppressing the new-invocation retry compensation requires. The handler now settles a compensated invocation as failed with an explicit compensation error, overriding even fresh applied run evidence from a replayed generation; verified and repaired receipts continue to replay applied. Receipt eviction is now primarily age-based: a stale same-id owner is bounded by time, not by how many newer invocations settle, so the previous count-only slice let a high-rate actor evict a tombstone while its delayed duplicate owner could still wake and repeat the action. Admission prunes only settled receipts older than a retention window that dwarfs every generation, job, and delivery-retry lifetime, and the count cap is demoted to a raised document-size backstop. * 🔀 fix: Serialize Compensation Against Settlement and Never Evict Live Receipts The terminal handler read its lifecycle snapshot, verified history, and then settled the public outcome — so a compensation resolving the same receipt during that window lost: the handler settled applied from its stale snapshot and no retry could ever change the replay-identity-locked outcome. The receipt's status CAS is now the serialization point: verification resolves the receipt BEFORE settling, whichever transition wins determines the public outcome, and a crash between resolve and settle converges through the retained receipt's replay. The verified-replay probe requires the receipt's own resolution, so a compensated receipt can never satisfy a verification retry. This inverts the settle-before-receipt ordering deliberately: that ordering guarded proof that resolution used to delete, and the receipt now retains its full action proof through resolution. The document-size cap is no longer an eviction quota. A receipt inside its retention window is never discarded: when the journal holds a full cap of unexpired receipts, new invocations are refused fail-closed until receipts age out, making duplicate protection and document integrity simultaneous invariants instead of a rate-dependent trade. * 🎓 fix: Keep Skill-Bearing Event Actors on the Legacy Path Skill primes are spliced into the message list directly ahead of the newest message, and a warm continuation forwards only that newest message — so a checkpoint-restored actor would keep serving the prime bodies baked in at its last cold start and never observe an edited or newly attached skill. Until the actor head carries a context fingerprint that forces a cold rebuild when the agent's skill context changes, agents with always-apply or manual skill primes stay on the legacy path, which re-primes fresh bodies every turn: correct on every event, just never warm. * 📜 fix: Gate Fork Mode on the Skills Capability, Not Just Request-Time Primes History-derived re-priming was a third path into the same staleness class: an actor that previously invoked a skill carries no request-time prime arrays, yet primeInvokedSkills re-resolves that skill's current body from history each turn and the warm slice drops the reconstruction — leaving the checkpoint's old body active after edits. The fork gate now keys on the priming hook itself (present exactly when the skills capability is enabled) alongside the request-time arrays, so every skill-body path routes to the legacy rebuild until #15235's context fingerprint restores warm continuation for skill-bearing actors. * 🧾 fix: Capture Applied-Action Proof at Tool Execution, Not After sendMessage The executor read applied-action evidence from the run-step collection the instant sendMessage resolved, but that collection is populated asynchronously — an applied invocation could classify as actionless (runSteps still empty while the tool result already streamed), discarding its fork and stranding the actor cold while the terminal handler later settled the same delivery as applied from the persisted evidence. Authoritative proof is now recorded in graph context the moment the expected tool executes: the request-owned recorder observes the tool-end chain (which ToolNode dispatches synchronously with both input and output) and applies the same fences as run-step evidence — exact tool name with the MCP-suffixed form, the declared argument subset against the execution input, an error-free result, and the background non-execution receipt exclusion. readAppliedAction consults the receipt first; run-step inspection remains the fallback for paths that bypass the tool-end chain. Regression coverage reproduces the observed ordering: the real executor commits the head from the receipt while run steps are empty, warm-continues the next event, and never re-executes the action; recorder fences and the receipt-first controller wiring are covered separately. * 🎯 fix: Supply Execution Arguments to the Tool End Callback The live Vertex + MCP canary exposed a contract mismatch the synthetic fixtures hid: the ON_TOOL_EXECUTE execution path invoked its tool end callback with output only, while the action recorder must verify the declared argument subset against the execution input. The receipt never qualified, every turn fell back to cold history rebuilds, and the tournament advanced with zero actor heads and zero retained checkpoints while looking successful. The execution handler owns both halves at the same moment, so the fix is at the source rather than a correlation store: ToolEndCallbackData gains the executed call's input and every callback site passes tc.args. A handler-level regression drives the real createToolExecuteHandler and asserts the callback receives both fields; recorder regressions pin the production shapes — an output-only tool end must starve an argument-fenced receipt rather than trust an unfenced match, and still qualifies a name-only expected action. * 🕵️ fix: Mark Background Deliveries So They Cannot Impersonate Applied Actions The background-claim callback reports the ORIGINAL tool's name for artifact attribution on the poll turn that harvests a completed task. A name-only expected action could therefore be impersonated by work some earlier turn dispatched: the recorder would attribute that delivery to the current invocation and commit a head whose state never contained the invocation's own action. The run-step evidence path never had this hole — it sees the poll tool's name — so the recorder must match its provenance discipline. Delivery callbacks now carry an explicit backgroundDelivery marker set at the one site that rewrites the name, and the recorder ignores marked deliveries outright. Regressions pin both halves of the contract: the delivery callback must carry the marker with the poll call's arguments, and a marked delivery can never qualify even a name-only expected action. * 🧿 fix: Version Invalidations, Keep Evidence Ahead of Output Policy, Gate Detachable Actions Three closeout-round findings, each converted into an invariant. Every legacy-path invalidation now advances a durable epoch — including for headless and already cold-marked actors, where the marker alone leaves no CAS-visible trace — and the actor-head CAS requires the epoch observed at preparation. A concurrently prepared fork whose history predates an intervening legacy turn can no longer commit past it; the commit reports an ordinary conflict and journals. Execution identity is now emitted before post-execution output policy: when a side-effecting tool succeeds but its returned content is withheld by the output filter, the callback delivers an outputFiltered receipt with blank content — the recorder accepts it as proof (rejecting model-detached calls it cannot distinguish through the blank shape), the artifact path never sees it, and an applied action is no longer reclassified actionless and re-executed on retry. Background-capable expected actions stay off the fork path: dispatch returns a launch handle every evidence fence correctly rejects, and the completion is provenance-marked as another turn's work, so a fork would settle actionless before the external effect lands with nothing to stop a retry from dispatching it again. The gate mirrors the MCP-suffix name matching of the evidence path. * 🚧 fix: Seal the Whole Legacy Turn Behind a Second Epoch Advance The epoch fenced only the legacy turn's start: a fork preparing after the begin invalidation but before the turn's message persistence observed the new epoch and cold marker, rebuilt from history that did not yet contain the turn, and committed cleanly because nothing advanced the epoch again — making the incomplete rebuild authoritative and clearing the marker. Every legacy event turn now seals its invalidation at terminal persistence with a second epoch advance, on the success, replaced-claim, and error exits alike. Sealing deliberately carries no quiescence requirement — it must succeed while a fork fence is active, because that is exactly the mid-turn race it defeats — and a fork that already committed against the begin epoch is healed the same way: the seal re-marks the head cold, so the next event rebuilds with complete history. Seal failure never diverts the turn's own exit; the begin bump still fences everything prepared before the turn. * 🔗 fix: Replace the Best-Effort Epoch Bump With a Durable Legacy-Turn Fence The second epoch advance could not make a legacy turn atomic, and three findings shared that root cause: two conditional updates left a headless gap a fork could create the head inside; the error exit sealed before saveErrorTurn made the error history durable; and any crash or failure between persistence and sealing left an incomplete fork authoritative, because the seal was best-effort and its failure was swallowed. A legacy turn now carries one durable fence. A token is written before execution by a single update-pipeline write — no two-write gap, and the cold marker is applied only where a head exists via $cond/$$REMOVE. While the token is present no fork may prepare (the adapter refuses) or commit (the CAS requires its absence), because the turn's messages are not yet durable. One atomic write clears the exact token and advances the epoch once history is persisted — after saveErrorTurn on the error exit — and success, replacement, and error exits all route through it. Failure is now fail-closed rather than silent: a failed seal leaves the token set, which keeps blocking forks and is logged as such, and a fence abandoned by a crashed turn is reclaimed only once stale, advancing the epoch and marking any head cold so the next event rebuilds from whatever history actually survived. * fix: serialize legacy event actor turns * fix: close legacy actor fence ownership gaps * fix: preserve legacy actor persistence fences	2026-08-26 08:39:00 -04:00
Danny Avila	1489623fa3	🎬 test: Record-Once/Replay-Forever Model Fixtures for Mock E2E (#15210) ... * 🎬 test: Record-Once/Replay-Forever Model Fixtures for Mock E2E The mock e2e lane's only credential-free model is a hand-authored script: `fake-model.js` decides responses from ~60 `E2E_*` prompt markers. That covers scripted shapes well, but no scenario replays a *real* recorded provider conversation through the assembled chain, so real streaming shapes — provider chunk cadence, reasoning deltas, usage metadata — are only ever approximated. This adds a record-once/replay-forever tier alongside the marker routing. Record (`E2E_MODEL_FIXTURES=record`, needs a provider key): the run hook appends a LangChain callback handler to every agent context's `clientOptions.callbacks` instead of overriding the model, so the REAL provider streams while each invocation's `AIMessageChunk`s serialize to `e2e/fixtures/model-replay/<name>.jsonl` — text deltas, tool_call_chunks, reasoning kwargs, and genuine usage metadata. Only the latest human text is recorded for binding; system prompts and tool schemas never enter the fixture. Replay (default, keyless): `fake-model.js` consults `tryBindReplay` ahead of marker routing, binding a conversation whose prompt matches the next unconsumed invocation. The replaying model is not hand-assigned — it is registered as SDK provider `librechat-e2e-replay` via `registerProvider` and constructed through the SDK's own `initializeModel`, so registry lookup, constructor clientOptions, and real `bindTools` all run the way a live provider's would. Recorded chunks therefore stream through the same createRun → graph → SSE → persistence chain. Consumption is enforced rather than assumed: every invocation re-checks its prompt against the recording, an invocation past the end of the script throws, and a per-fixture ledger lets the spec assert at teardown that every recorded invocation and chunk was drained. Streaming incrementality is asserted from that ledger, not by sampling transient DOM, which is a race by construction. The credential-free profile is unchanged when not recording: the record provider and its selector entry are template markers that stay comments, and no existing spec's routing is touched (a fixture only binds on an exact prompt match; everything else falls through). Verified: record vs the real DeepSeek API 1 passed (15.2s); keyless replay 1 passed twice (11.4s, 12.7s) with the ledger fully drained (2/2 invocations, 10/10 chunks, no overruns or mismatches); app-load, completion, and chat 10 passed unchanged. * fix(e2e): rebind a replay fixture from the top for a new conversation The replay cursor is process-global while the web server outlives a Playwright retry, so a fully consumed fixture left the retry unable to bind its first prompt: it fell through to marker routing and failed deterministically, burning every configured CI retry. A partially consumed attempt failed the same way. Binding now restarts the fixture when the incoming prompt matches its first recorded invocation, resetting the ledger with the cursor so the new attempt is judged on its own consumption instead of accumulating the previous one's counts. Continuing an in-progress binding still outranks restarting, so a fixture whose opening prompt repeats later in the script advances rather than rewinding. The over-consumption guard is untouched — it fires inside the stream when the cursor passes the end, not at bind time. * fix(e2e): close three replay-lane gaps found in review Restart the recorder on a retry. Its state is process-global like the replay cursor, so a failed attempt that had already recorded invocations left the counter advanced: the retry appended 2/3 after 0/1, or kept the previous attempt's `error` line, and the fixture was unusable for replay. Recording now truncates and restarts when the opening prompt reappears, mirroring the replay side's rule and its caveat. Retain a consumed binding for the conversation that drove it. An extra user turn past the final recorded invocation found no next invocation and fell through to ordinary fake-model routing, so it was answered with a mock reply: the over-consumption guard never ran and the already-drained ledger still passed. Such a conversation is now recognized by its human turns opening with the fixture's recorded prompts, and stays bound so the stream raises the overrun. Continuing an in-progress binding still outranks restarting, which outranks retaining a consumed one, so a retry's fresh conversation rewinds rather than being read as an extra turn. Validate the fixture the recording actually wrote. Record mode honors `E2E_MODEL_FIXTURE_NAME`, but the spec always inspected the committed `deepseek-two-turn`; another name wrote elsewhere while the assertions read the pre-existing file, and because the prompts are fixed the stale answers could match and green a run that verified nothing it produced. * fix(e2e): make replay binding correct for tool and subagent fixtures Round two's retry and consumed-binding fixes both assumed one model invocation per user turn. A turn that calls a tool breaks that: the model is invoked again after the tool result under the same latest human message. Identify a retry by the conversation boundary, not the prompt. The recorder ran per invocation and truncated whenever the opening prompt reappeared, so a tool round trip looked like a retry and discarded the recorded tool-call invocation. Restart detection now sits in `installRecorder`, which runs once per `createRun`: a turn whose history holds no prior human message begins a conversation. Compare consumed bindings against user turns, not invocations. Several recorded invocations can share one prompt, so a one-to-one comparison could not recognize the originating conversation — invocations `[A, A, B]` against history `[A, B, C]` failed on both length and elements, and the extra turn fell through to the fake model with the drained ledger still passing. Fixtures now carry their collapsed turn sequence. Override the subagent model too. `graph.overrideModel` is not inherited by child executors, so a fixture recording a subagent call — record mode captures child invocations already — would leave the child on its configured provider: an underrun, and a real provider request in a lane that must stay keyless. Reject ambiguous prompt matches. Binding order followed filesystem enumeration, so a second fixture sharing a prompt could silently redirect a scenario to the wrong chunks and ledger; the spec's choice never reaches the server-side loop, so ambiguity fails instead of picking a winner. Fixture identity is the file name for the same reason — a recorded `meta.name` is descriptive, and trusting it let a copied fixture collapse onto another's registry key and ledger. Prove the recording is fresh. The spec removes the selected fixture before driving, so a run whose hook never installed the recorder fails instead of greening against a stale artifact whose answers still match these deterministic prompts. * fix(e2e): rewind a replay fixture at the conversation boundary Consecutive invocations can share a prompt — a tool call produces exactly that — so an attempt stopping mid-turn left the cursor on an invocation whose text still equalled the opening prompt. Matching the cursor first meant a retry's fresh conversation resumed after the tool call instead of rewinding, consuming the post-tool invocation and silently replaying a different script than was recorded. A conversation boundary now outranks an in-progress cursor: a fresh conversation whose prompt opens the fixture rewinds even when the cursor would have matched. Continuing still outranks restarting within a conversation, so a turn that calls a tool advances to its post-tool invocation rather than rewinding on its own repeated prompt. * fix(e2e): refuse cross-conversation binding and prove content streaming A fresh conversation could steal a partly consumed fixture's later turn. Only a conversation opening with the fixture's first prompt was treated as a boundary, so after `[A, B]` had consumed `A`, an unrelated new conversation whose first message was `B` matched the cursor, received the recorded second-turn response, and advanced the shared cursor without ever having driven `A`. A conversation start may now only rewind a partly consumed fixture, never continue it; continuation within a conversation is unaffected. The incrementality assertion counted empty frames. Providers emit empty initialization and usage-metadata chunks around the content deltas, so a total chunk count above one was satisfied by a single delta: the previous fixture's closing turn had four chunks and one content-bearing delta carrying the whole answer, and both modes stayed green without proving incremental assistant-content streaming at all. Fixtures now track content-bearing chunks separately, the closing prompt asks for prose rather than a number, and both modes require several content deltas on that turn. Re-recorded: the closing turn now carries 28 content deltas. * fix(e2e): scope record mode to the fixture spec `E2E_MODEL_FIXTURES=record` replaces the fake-model hook globally, so an unfiltered entry point such as `npm run e2e:mock` sent every spec under specs/mock to the paid real-provider endpoint, while each fresh conversation truncated and rewrote the one selected fixture — leaving an artifact from whichever scenario happened to run last. Record mode now matches only the fixture spec: an unfiltered recording run lists one test instead of 203. Replay mode is untouched and still collects the full suite. * 🪪 fix: Bind Replay Fixtures by Conversation, Not Prompt Text Prompt text was standing in for conversation identity, and three review rounds found the same class of defect underneath it: a tool call repeats a prompt across invocations, a retry repeats it across attempts, and a resumed run has neither prompt nor history because `createRun` is rebuilt with no messages while state comes from the checkpoint. Each fix in that space created the next gap. Thread the identity instead. `createRun` accepts a `conversationId` and passes it to the run hook, which the agents controller supplies at both call sites — the same value it already uses as the checkpointer's `thread_id`. The field is optional and the hook is env-gated, so nothing changes when the harness is not in use. Binding then collapses to ownership. A fixture is owned by the conversation that claimed it, and its cursor is authoritative wherever it stands: an extra turn reaches the over-consumption guard rather than falling through to the scripted fake model, and a resumed run keeps replaying with no prompt to match. A different conversation may claim the fixture only by opening it, which rewinds — what a Playwright retry looks like. Everything else is refused, so an unrelated conversation can no longer continue someone else's partly consumed script by repeating a later prompt. The prompt is still re-checked on every real turn; only a resume, which structurally carries no human message, is exempt. The previous text-and-history rules remain as a fallback when identity is absent. The recorder keys the same way: a new attempt is a new conversation, so a resume no longer truncates the fixture mid-turn and discards its tool-call invocation. Record summarization too. The summary provider runs on its own model with its own callback list, so a scenario crossing the context-pruning threshold recorded the agent's invocations but not the summariser's, leaving a fixture that could not reproduce the pruned context. * 🧾 fix: Harden Record Mode and Make the Rendered-Text Assertion Honest CI caught what local runs had not: the committed fixture was never replayed locally, because the record run overwrote it after the replay check rather than before. Re-recording and replaying in that order is what surfaced the rest of this. The DOM assertion compared raw recorded text against rendered markdown. The previous answer opened with `52.`, which Markdown renders as an ordered-list marker, so those characters never appear in the DOM and the match failed on all three CI attempts while replay itself was correct. The closing prompt now asks for prose beginning with a word, a leading enumerator is stripped before matching, and only a prose prefix is compared. Derived configs discarded the record-mode restriction. `config.redis.ts` and `config.mermaid.ts` spread this config and then replace `testMatch`, so `e2e:mock:redis` in record mode would still send its specs to the paid provider. A restriction expressed as an overridable value cannot hold, so record mode now refuses any config but the mock one. Superseded recording callbacks could write across a reset. A failed attempt with a provider call still in flight keeps its handler on the old graph; after the retry reset, that call would allocate an invocation from the new counter or append an `error` entry with a cleared mapping. Handlers now carry the recording generation they were installed for and ignore everything from an older one, and attachment dedupes against the current generation so a graph carried across a restart is not left with an inert handler. * 🚧 fix: Make Summarization an Explicit Boundary, Not a Half-Feature Recording summarization invocations without replaying them is worse than ignoring them. Replay routes the agent model and subagents only, so a recorded summarization entry takes a slot in the fixture sequence that replay never consumes, and the next primary call reads the summariser's chunks — a prompt mismatch or, worse, silently wrong content. The attachment was also aimed at the wrong shape: the SDK reads `summarizationConfig.parameters`, not `.parameters` nested under `.config`, so the previous attempt would have attached to nothing in a real run. Its test passed only because the test built the shape the code expected rather than the shape the SDK provides. Rather than ship a half-routed feature, recording now fails the moment summarization runs, naming the reason. Both shapes are guarded so the guard cannot miss the way the recorder did. Summarization fixtures need replay routing for that model before they can be supported. The derived-config guard added alongside it was itself broken: workers do not carry `--config`, and the argument lookup fell through to `process.argv[0]`, so every recording run aborted claiming the node binary was an unexpected config. The flag is now located explicitly and absence is treated as "not the process that parsed the CLI". * 🔒 fix: Close the -c Config Alias and Pin the Recorder's Fixture Name Playwright documents `-c` as an alias for `--config`, so record mode launched as `playwright test -c e2e/playwright.config.redis.ts` slipped past a guard that recognised only the long spelling. Both spellings and both `=` and space forms are now parsed. Accepting arbitrary fixture names also worked against the ambiguity check. This spec drives one fixed prompt pair, so recording under another name left two fixtures sharing those prompts; replay then refused to bind either and the keyless lane stopped working — a successful documented recording run could disable the suite it exists to serve. The spec now records only the fixture it owns and says so when asked for another. * 🔧 test: Record a Real Tool-Call Turn and Replay It Through the Tool Node The fixture format carried `tool_call_chunks` and the binding advanced through a turn's invocations, but nothing had recorded a real tool-calling conversation end to end — the path was covered only by hand-written synthetic fixtures, and it is the first one a new scenario would exercise. This records one: the provider calls the `remember_fact` MCP tool, the tool runs, and the model is invoked a second time with its result. That is the shape a single prompt cannot express — one user turn spanning several model invocations, all sharing one prompt — so it is what proves the turn-vs-invocation distinction the binding rules were built around. Replay drives the real tool node rather than replaying its output, so the tool executes again and the assertion checks its live result. Two fixtures now coexist, which the record path had to grow for: the config keeps an allowlist so an unknown name is still refused, record mode collects every replay spec, and each spec records only the fixture it owns and stands down for the others. MCP tools reach the model under a server-qualified name (`remember_fact_mcp_e2e-memory`); that qualification has changed before, so the assertions match the base name as a prefix rather than pinning the suffix. Verified: record 1 passed (15.6s, real API) then replay 1 passed (13.6s) against that fixture, ledger drained 2/2 invocations and 35/35 chunks; both replay specs together 2 passed; app-load, completion, chat and mcp-ephemeral 12 passed.	2026-08-25 18:29:55 -04:00
Danny Avila	290b8664d9	🧾 feat: Track Authoritative Agent Event Outcomes (#15213) ... * feat: track authoritative agent event outcomes * fix: isolate agent event outcome types * fix: declare agent event handler result * fix: simplify agent event status selection * fix: preserve authoritative event outcomes * fix: preserve terminal event evidence * test: use completed run-step envelope * test: scope deferred HITL question locator * fix: settle every agent event terminal path * style: sort terminal host action imports * fix: fence agent event terminal evidence * fix: recover agent event terminal settlement * fix: scope terminal retry hints by generation * fix: settle terminal host actions exactly	2026-08-25 18:20:57 -04:00
James Todaro	018775de07	🧾 fix: Honor Disabled Transactions on the Assistants Usage Path (#15100) ... * 🧾 fix: Honor Disabled Transactions on the Assistants Usage Path Thread the resolved transactions config through `recordUsage` from each of its callers, so `transactions.enabled: false` is honored on the assistants token spend path. * 🧾 fix: Thread the transactions config through the vision-request caller Address review: `ToolService.processVisionRequest` also records usage without the resolved config, and `recordUsage`'s documented return type did not match the function. * 🧾 fix: Set the resolved transactions config after the usage spread - provider usage could carry a `transactions` key that overwrote the trusted value - matches the ordering the other `recordUsage` callers already use	2026-08-25 08:30:12 -04:00
Danny Avila	862ebf3235	🪢 fix: Persist Failed Agent Turns Before Error Publication (#14118)	2026-08-25 06:50:46 -04:00
Danny Avila	ac2aef00f6	🫗 fix: Drain Quoted Excerpts Into Mid-Run Steering (#15175) ... * 🧭 fix: Carry Quoted Excerpts Through Mid-Run Steering "Add to chat" quote chips were dropped by every during-run steer path: the steer POST had no quotes concept, so a composer-origin steer left the chip staged (gluing onto the NEXT send) and a queued item steered into the live run lost its quotes silently. Quotes now ride the steer protocol end to end: - POST + admission: `quotes` on the steer body, normalized like the chat route's (getReferencedQuotes caps), part of the idempotency fingerprint only when present so pre-existing receipts still replay. - Injection: merged into the model-bound turn as Markdown blockquotes at both boundaries (text-only and media paths), mirroring prependQuotes. - Persistence + replay: the STEER content part stores `quotes` separately from the typed text; stampSteerPartMedia re-merges them per turn (even with resendFiles off) via the SDK's transient media stamp, with the quote block folded into the token budget. - UI: composer steers/interrupt-steers drain the chips (skill picks stay staged — they configure a NEW turn's run); SteerPart and the in-flight bubble render the same MessageQuotes reference blocks as user bubbles; queued/failed rows show a quote count; reconnect reseeds fall back to the server item's quotes when no local chip survives. - buildMessages keeps its zero-await path to the parallel context kickoff via a synchronous stamp-target probe. * 🧭 fix: Keep Quotes in the Client-Safe Steer Projection toPendingSteer is the projection behind resume-state pendingSteers, abort responses, and terminal leftover claims — dropping quotes there would lose them on exactly the recovery paths the reconnect reseed's server fallback relies on. * 🧪 test: In-Flight Steer Bubble Renders Carried Quotes * 🔁 fix: Re-Stage Quotes When a Pre-Quotes Replica Accepts the Steer Codex flagged the rolling-deploy window: an old replica 202s a quoted steer while dropping the excerpts, so the client cleared the chips for context the model never received. The 202 (fresh and receipt replay) now echoes quotesAccepted from the DURABLE item; a missing echo on a quote-bearing composer-origin steer re-stages the excerpts as composer chips — the pre-steer behavior, so they ride the next send instead of vanishing — and strips them from the surviving chip so a later terminal conversion cannot duplicate them. Queued-origin steers keep quotes on the item, whose restore paths already return it intact. The residual cross-version lost-ACK retry stays fail-closed as a 409 idempotency conflict (failed chip with retry controls). * 🔁 fix: Close the Remaining Cross-Version Quote-Loss Windows Codex round 2: - Send now of a quoted queued item against a pre-quotes replica now re-stages the excerpts too (the row is consumed and the words inject bare, so the composer is their only remaining home); the strip clears the chip's captured origin copy so reclaims and terminal conversions cannot duplicate them. - A quoted retry whose lost first ACK was accepted by a pre-quotes replica now REPLAYS that legacy receipt instead of 409ing: the stored fingerprint matching the quote-less hash of the same words proves the cross-version case, and the replayed 202's missing echo drives the re-stage. Different quotes against a quote-bearing receipt still conflict. - TSteerAppliedEvent.part gains the quotes field (typed SSE consumers). * 🧪 test: Drop the Stale Narrow SteerDrainOutput Alias The spec's local intersection re-declared injectedMessages with content: string, predating the SDK pin that declares the field natively (content: string | MessageContentComplex[]). Under CI's clean install the hook's BaseHookOutput is no longer assignable to that narrower alias; the plain PostToolBatchHookOutput is the correct type for every drain/boundary assertion. Verified against the published 3.6.16 dist and the local one. * 🔁 fix: Honor the Generation Owner's Quote Capability End to End Codex round 4: - steerQuotesCapable rides job metadata (createJob + HITL resume rewrite), mirroring preemptCapable's owner-recorded pattern: an upgraded admission replica no longer stores quotes — or claims them accepted — for a generation whose older owning drain would silently drop them at injection. The missing echo drives the client re-stage, and a later capable handover cannot double-deliver restored context. - Applied events reconcile dropped quotes: when a quote-less applied part settles a quote-bearing chip (the lost-202 ordering the ACK-echo path cannot see), resolveSteerChip and both reconnect settle paths re-stage the chip's excerpts before removing their only copy. mergeRestagedQuotes dedupe keeps every trigger idempotent for the same excerpts. * 🔁 fix: Re-Read Quote Capability at the Last Moment and Cap Restaged Chips Codex round 5: - A HITL resume rewrites steerQuotesCapable without changing the generation's createdAt, so the enqueue fence cannot see a capable-to-legacy handover landing during admission's awaits. Re-read the owner's flag immediately before item construction (paid for only by quote-bearing requests); the residual between re-read and enqueue commit matches preemptCapable's documented race. - mergeRestagedQuotes now respects the 10-quote contract with the staged chips winning: a restored tail that cannot ride the next send is dropped explicitly instead of rendering as a chip the submission would silently discard. MAX_QUOTE_COUNT moves to utils/steer as the single client source; QuoteButton imports it. * 🔁 fix: Steer Quote Coverage for Preflights, Memory, and Single-Scan Stamping Codex round 6: - Stored-message policy inspection now extracts steer-part quotes as quote fragments (path /content/N/quotes/M), so conversation import and shared link preflights inspect the newly persisted field exactly like top-level message.quotes. - The memory copy gets its own quote-merge stamp (text only, resendFiles false): formatAgentMessages ignores part.quotes, so without it a steer whose substance lives in its excerpt reached the chat model but never memory extraction. - collectSteerStampTargets replaces the boolean probe: buildMessages collects once and hands the targets to stampSteerPartMedia, keeping the zero-await fast path without scanning the history twice. * 🔁 fix: Redis Quote Plumbing, Conversion-Race Guard, and Quote-Bound Recovery Proof Codex round 7: - RedisJobStore.deserializeJob now restores steerQuotesCapable (the explicit mapper otherwise dropped it on every read, leaving quote steering inert in Redis deployments), with the round-trip spec extended. - Both Lua parked-steer projections (terminal close + generation replacement) forward item.quotes, matching toPendingSteer — a lost final no longer strips excerpts from durable recovery in Redis mode. - The no-echo restage reads the SURVIVING chip (reclaimRejectedChipQuotes): a terminal conversion that beat the delayed 202 already moved the quotes onto the queued follow-up, and re-staging them again double-delivered. Regression-tested with the conversion-before-ACK ordering. - RecoveredSteerPayload binds normalized, order-significant quotes (builder, validator, TS matcher, and the Lua decode+matcher): a stale client presenting the same recoverySteerId with altered or missing quotes cannot consume the parked source. Quote-less sources keep matching quote-less recoveries. * 🔁 fix: Execution-Bound Quote Capability with an Atomic Enqueue Predicate Codex round 8: - steerQuotesCapable becomes a transient assertion translated (at createJob and in ApprovalLifecycle.resolve) into steerQuotesExecutionId, valid only while it equals the LIVE providerExecutionId. A legacy replica winning a HITL resume rewrites the execution id without knowing the marker, so its stale assertion self-invalidates — a bare boolean could not be cleared by code that predates it. - The fenced enqueue evaluates that equality atomically (all three Redis scripts decode-and-strip like the existing preemptCapable normalization; both InMemory sites mirror it) and returns the persisted item, so the quotesAccepted echo reflects exactly what was stored even when a handover lands between admission's read and the commit. The last-moment re-read is gone — the transaction is the authority. - Tests: capable-resume re-binding, legacy-resume omit-not-clear invalidation, the admission-vs-handover race (capability read true, then execution rewritten before enqueue), and the Redis round-trip of the marker. * 🔁 fix: Full Redis Parking Coverage and Loss-Moment Quote Restaging Codex round 9: - The two remaining Redis parking projections (terminal status CAS and stale-running cleanup) forward item.quotes — every field-picked steer projection now carries them (audited: 2 Lua 'projected' + 2 Lua 'clientItem' + toPendingSteer). - The ordinary no-echo ACK no longer re-stages: the steer has not injected yet, so the quotes stay carried on the pending chip. A quote-less applied event re-stages them at the actual loss; a terminal leftover conversion carries them onto the recovered row, whose normal send delivers quotes on any server — re-staging at the ACK let that leftover auto-send bare text while the excerpts glued onto an unrelated draft. Only the settled receipt replay (already injected, no future event) reclaims immediately. * 🔁 fix: Legacy-Replayable Receipts with Separate Quote Identity Codex round 10: an upgraded-first receipt stored a quote-inclusive fingerprint no pre-quotes replica could recompute, so a lost-ACK retry routed through one 409'd already-accepted words with duplicate-send controls. The durable fingerprint reverts to the quote-independent 3-field hash — the one shape EVERY deployed version computes, replayable across a rolling deploy in both directions — and quote identity moves beside it as requestedQuotesFingerprint (of the REQUESTED quotes, pre any capability strip, so an incapable-owner acceptance still replays its own retries). Absent records (legacy-written or quote-less) accept any same-words retry, preserving the round-5 rule; present records must match exactly, keeping different-quotes clientSteerId reuse a 409 on quote-aware readers. Under the keep-on-chip client contract a legacy replay's missing echo is harmless — the excerpts stay carried on the pending chip. * 🧪 chore: Re-Trigger CI After Dropped Workflow Events	2026-08-24 22:29:13 -04:00
Danny Avila	5a8700643c	⚡ perf: Build the Memory Message Copy Only When Something Reads It (#15164) ... buildMessages formatted every history row twice per turn — a prompt copy and a memory copy — then discarded the entire memory payload unless some row carried fileContext, which is the rare case. The memory copy has exactly two consumers: that payload, and the canonical recount of a row, where it is content-identical to the prompt copy unless the row itself has fileContext. So the prompt copy is now the recount surface for context-free rows, a fileContext row builds its memory copy at recount time, and the full memory payload is assembled in a deferred pass — same formatting, same per-row merge order — only once a row has proven the payload will be kept. The common turn formats each row once instead of twice and no longer allocates a payload it throws away. Also forwards the run's useLegacyContent to formatAgentMessages as the new legacyContent option, inert on the current SDK release: once the SDK change ships, text history is emitted pre-flattened so the per-request legacy projection stops cloning every message and the context meter's identity-based count reuse holds across the projection.	2026-08-24 10:33:47 -04:00
Danny Avila	fc2b8584c4	📇 feat: Surface Event Child Activity Through a Bounded Parent Index (#15142) ... * feat: surface event-driven child activity * fix: keep child task aggregation documentdb-compatible * fix: address event activity review findings * test: provide markdown message context defaults * fix: report bounded child history truncation * fix: preserve current child activity state * fix: preserve durable event child activity * fix: handle missing task timestamps * fix: keep active event snapshots live * fix: preserve event activity across valid anchors * fix: close event child activity gaps * fix: preserve event activity across resume	2026-08-23 18:50:00 -04:00
Danny Avila	77cb72e50c	🧮 perf: Enable Agent Context Count Reuse (#15130) ... * perf: enable agent context count reuse * fix: declare token counter return type * fix: initialize cached token counters * style: sort token counter imports * fix: Keep cached token counts exact	2026-08-23 03:05:02 -04:00
Danny Avila	dd146ff74d	🧾 fix: Report Complete Agents API Usage (#15127) ... * fix: report complete agents api usage * fix: preserve invoked usage context * test: cover absent usage context * fix: type responses usage finalization * fix: preserve reasoning usage aliases * fix: declare reasoning usage alias	2026-08-23 02:37:09 -04:00