1583 commits

Author	SHA1	Message	Date
Danny Avila	aa7539909b	🪶 feat: Enable Standalone Skill Authoring (#15480) ... * feat: enable standalone skill authoring * test: align skill e2e flow with enable toggle * fix: preserve legacy skill state on version restore	2026-09-01 18:39:05 -04:00
Danny Avila	6195273429	🧲 fix: Anchor Memory Partitions to Accessible Agents (#15475) ... * fix: validate agent memory partitions * fix: authorize agent memory mutations * test: cover agent manager memory access * fix: centralize agent memory authorization * fix: harden agent memory authorization * fix: support synchronous role providers	2026-09-01 18:37:52 -04:00
Danny Avila	dccd7ae496	🧶 fix: Preserve Multi-Agent Tool Call History (#15468) ... * 🧭 fix: Preserve Multi-Agent Tool History * 🧰 fix: Preserve Lazy Subagent Tool History * 🧪 test: Enable Lazy Code History Fixture * 🧭 fix: Derive Effective Lazy Tool History * 🧪 fix: Tolerate Empty MCP Name Audits * 🧪 test: Mock Lazy MCP Name Audit * 🧭 fix: Expand Lazy Historical Tool Names * 🧱 fix: Derive Durable Wildcard History * 🧬 fix: Traverse Nested Wildcard History * 🔭 fix: Resolve Wildcard Server Boundaries * 🔐 fix: Persist MCP Call Identity * 🪪 fix: Stamp Resolved MCP Execution Identity * 🕸️ fix: Resolve Graph Member MCP Identity * 🧭 fix: Fall Back From Empty Graph Member ID * 🧭 fix: Preserve Aggregated MCP Identity * 🧭 fix: Harden MCP Identity Resolution * 🧭 test: Assert Concurrent Graph Limit Invariant * 🧭 fix: Centralize Lazy History Policy * 📦 chore: Bump `@librechat/agents` to v3.7.13 * 🧪 test: Align Content Filter Fixture With SDK History	2026-09-01 18:37:45 -04:00
Danny Avila	c8939c9730	🪄 fix: Reuse Downloaded Artifacts for Image Reads (#15454)	2026-09-01 10:55:53 -04:00
Danny Avila	8f6228095f	🖼️ fix: Stop Reporting a Failed Sandbox Image Read as an Unreadable Format (#15453) ... * 🖼️ fix: Stop Reporting a Failed Sandbox Image Read as an Unreadable Format `read_file` on a `/mnt/data` image collapsed every failure — rate limit, missing path, truncated transfer, transport error — into the binary-file hint, whose image branch reads as a permanent capability limit ("is an image file and cannot be read as text"). The model stopped retrying, and the operator had no way to tell which failure had actually occurred. The failure that surfaced it: the reader windows the bytes through `/exec` stdout, one call per window, against the Code API's per-user execution limiter (20 requests / 30s in the reference deployment). At a fixed 32KB window a 122KB chart cost 4 calls and a 1MB image cost 33 — more than a whole limiter window, so anything over ~640KB could never complete. A turn that generates several charts and reads them back exhausts the budget partway through and reports the 429s as unreadable images. - `handlers.ts`: failed reads name the real cause and the affordance that matches it — rate-limited waits and retries, a missing path lists the directory, a truncated transfer retries. The "cannot be read as text" phrasing is now reserved for bytes that are not a supported image. An image past the inline budget says to downscale and read the smaller copy rather than pointing at `bash_tool`, which cannot show an image. - `process.js`: the window is derived from the runner's stdout budget (48,960B vs 32,768B, so a 122KB chart is 3 calls and 1MB is 22), halves and re-reads the same offset when a runner truncates anyway (remembered per Code API, with the env remedy logged), waits out a rate limit it can afford instead of discarding the bytes already pulled, and stops at a 24-call ceiling so no read stalls a turn across limiter windows. - `utils/code.ts`: shared `getCodeApiRetryAfterMs` (header, then the `retry_after_seconds` body); `skillFiles.ts` drops its private copy. * ♻️ refactor: Move the Sandbox Image Reader Into TypeScript Addresses the review round on #15453. - P1: windowing, window sizing, the in-sandbox reader script, response parsing and byte assembly move to `packages/api/src/files/code/image.ts`, and the Code API rate-limit policy to `packages/api/src/utils/code.ts`. `process.js` keeps only the `/exec` transport it owns. Both new modules are tested against their real implementations (19 + 13 cases), including the round-trip ceiling, the changed-file guard and the shell-injection guard on the reader script. - P2: a configured stdout budget below the old 8KB window floor produced a window the runner could never emit, and narrowing was pinned to the same floor, so no retry could succeed. The window now aligns to the base64 group size instead, and the round-trip ceiling bounds what a small budget costs. - P2: `.env.example` documented a 32,768-byte default that no longer holds and never mentioned the stdout budget. It now describes `LIBRECHAT_CODE_SANDBOX_OUTPUT_MAX_SIZE` as the setting to tune, with the chunk override as the escape hatch. - P3: `process.spec.js` no longer reimplements the retry parser in its module mock. Behaviour is tested where it lives; the mock is a declared passthrough and the spec covers what the transport still owns — session forwarding, the profile header, parser hand-off and error logging. - P3: `getCodeApiRetryAfterMs` reads the response body through `in` narrowing rather than an `as` assertion. - P3: the window-sizing JSDoc no longer claims a multiple of 3 in the one case that is not — an explicit `LIBRECHAT_CODE_IMAGE_CHUNK_BYTES`. * 🎯 fix: Size Sandbox Image Reads to What the Limiter Can Deliver Addresses review round 2 on #15453. - Narrowing now halves the caller's own failing window instead of the shared learned value, and the learned value only moves down to the narrowest size any read needed. Reads run concurrently (one per tool call), so two reads failing at the same size were compounding into a window two steps smaller than the runner would have accepted, and persisting it for every later read. - Narrowing continues until a runner accepts a window or a floor below which no runner could serve an image at all. Two halvings stopped at 12,240 bytes — still ~16KB of base64 — so an 8-10KB stdout cap that was not configured up front stayed unreachable. - The round-trip ceiling is now one limiter window (20), and a read that cannot finish within it says so from the FIRST response rather than spending the whole budget rediscovering it: the first window reveals both the file size and what a call really delivers. A near-limit image no longer drains the turn's executions before failing, which is what made the wait budget look too small. - The round-trip message names the size this deployment can actually inline, so the model has a downscale target instead of a guess. - `python3: not found` from the sandbox reader was matching the generic "not found" in `isSandboxMissingFileError` and being reported as a missing image path, sending the model to `ls /mnt/data` while hiding a runner dependency. Image reads now classify only path-specific ENOENT forms as a missing file. * 📐 fix: Size a Truncated Read's Retry From the Cap It Revealed Addresses review round 3 on #15453. A ~1KB sandbox stdout cap is a real runner default, and the 1,536-byte narrowing floor sat above it: discovery stopped one step short and declared every image unreadable, even though small ones fit comfortably in the 576-byte window the configured path already derives. Narrowing now runs until no smaller window exists. It also no longer has to grope its way down: a truncated response emitted exactly as much as the runner allows, so its length measures the cap this deployment never declared, and the retry is derived from it directly — one round-trip instead of the eight blind halvings it takes to fall from ~49KB to ~1KB. Blind halving remains the fallback and bounds the result either way, so every retry is strictly smaller than the window that just failed.	2026-09-01 09:08:01 -04:00
Danny Avila	97e1e950ad	🧲 fix: Draw OpenID Refresh Lookups Into System Context (#15451) ... * fix: scope refresh user lookups to system context * fix: preserve refresh system context * fix: bind refresh recovery to user tenant	2026-09-01 02:02:21 -04:00
Danny Avila	224512851a	🗂️ fix: Resolve Code Interpreter Destination Collisions (#15449) ... * 🗂️ fix: Resolve Code Interpreter Destination Collisions Codeapi mounts each `/exec` input at a destination derived from its `name` and rejects the whole request when two entries land on one. LibreChat never enforced that: `primeFiles` sent `name: file.filename` verbatim, and the graph seed deduped by `(storage_session_id, id)` — identity, not destination. Two conversation files sharing a filename therefore killed code execution for the rest of the conversation. The rejected request never reaches the sandbox, so no result comes back to collapse the pair, and every later turn re-primes both and fails identically. Only code-generated outputs are covered by the `(filename, conversationId, context, tenantId)` partial unique index; uploads carry `context: message_attachment`, so one conversation can hold several records sharing a filename — two uploads of `image.png`, or an upload whose name a later execution wrote back in place. `packages/api/src/files/code/destinations.ts` now owns the name → destination mapping for every contributor to an `/exec` file list, mirroring codeapi's conflict rule including directory-prefix conflicts: - `primeFiles` claims per pushed file, disambiguating with a `-2`, `-3` counter and echoing the resolved path into the tool context, so the model is told where each file actually mounts. Claim order comes from immutable `createdAt`, not `getFiles`'s `updatedAt` sort, which usage accounting and re-upload both bump — order-dependent destinations would silently repoint a path an earlier turn had already handed the model. Newest wins the bare name, matching `ToolNode.updateCodeSession` and giving an in-place rewrite precedence over the original it superseded. - `seedCodeFilesIntoSessions` drops a later file whose destination is already claimed. Renaming there would invent a path nothing told the model about, and a conflict at that layer means one logical file arrived twice under different storage pointers. Fixes #15443 * 🧹 chore: Tolerate Holes in the Destination Priority Sort `primeFiles` sorts the raw `getFiles` result before its own per-entry guard runs, so the comparator has to survive a null row rather than throw ahead of the loop that was already defending against one. Also tightens the tool-context assertion in the upload-vs-output case, which was passing on a trailing space rather than on the path. * 🔁 fix: Address Codex Review on Destination Assignment Four findings from the review of `b0f61d0`. **Identity-derived suffixes instead of a collision counter.** A counter is assigned from whatever is in the set, so a third file arriving under the literal name a displaced file had been given pushed that file along, and code the model wrote in an earlier turn would silently start reading the newcomer. The fallback now hashes the file's own `file_id`, so a displaced file keeps its destination for the life of the conversation and only the winner of the bare name can change. A trailing counter still guards a hash collision so the search terminates on any input. **Rank by last content write, not record creation.** `processCodeOutput` claims one row per `(filename, conversationId)` and rewrites it in place, so a repeatedly written output keeps its original `createdAt` while recording the latest writer in `metadata.sourceDispatchedAt`. Ranking on `createdAt` alone handed the bare path to an upload the output had since been rewritten over. Recency is now `max(sourceDispatchedAt, createdAt)` — still never `updatedAt`, which usage accounting bumps without any content changing. **Advertise a displaced generated output.** Generated files are left out of the tool context because the model already knows what it wrote — which stops being true once a newer same-named file takes the bare path and the output mounts somewhere the model has never seen. Displaced outputs are now listed with the name they were written as, and displaced uploads name their original too. **Rank conversation files above an agent's private ones.** Each agent claims destinations over its own candidate set — the conversation's files plus its own — so a private file taking a bare name from a shared file in one agent but not in another left two agents advertising different paths for the same file into one shared mount namespace. Shared files now rank first, so the majority that every agent sees resolves identically. The private-versus-private case still drops at the merge and is documented there; closing it needs one assignment across contributors, made before any agent renders its context. * 📌 test: Pin Alias Ownership on an In-Place Rewrite Codex round 2 flagged that the identity alias still falls back to a counter when the alias itself is literally claimed, so a displaced file can move after all. The mechanism is real and my JSDoc overstated the guarantee, which this corrects. The behaviour stays as it is. The realistic way a file arrives under another file's alias is the model rewriting the displaced file in place: it was told `<stem>-<hash>.ext`, wrote back to it, and `processCodeOutput` registered an output under that literal name. Keeping the alias with the newest holder is what makes the model's familiar path resolve to its own edit; reserving it against the newcomer would hand that path back to the bytes it replaced. The test pins that ordering so a later change cannot invert it silently.	2026-09-01 00:55:54 -04:00
Marco Beretta	146664e0d1	🔐 fix: Redirect OAuth Browser Navigations Instead of Rendering JSON (#15440) ... * 🔐 fix: Redirect OAuth Browser Navigations Instead of Rendering JSON The /oauth/* routes are reached by top-level browser navigation, so a middleware that rejects one with a JSON body makes the browser render that body as the page. Hitting the login rate limit during a social sign-in dropped the user on a blank page showing: {"message": "Too many login attempts, please try again after 5 minutes."} Mark those requests with markOAuthNavigation and have the two middlewares that can reject them answer with a redirect to the login page carrying an error code the client localizes. The login limiter stays a single instance so the attempt budget is still shared with POST /api/auth/login, and that route keeps its JSON 429 for the SPA. * 🧪 fix: Address PR Review Bot Findings chatgpt-codex-connector: - Split the inline `type TranslationKeys` specifier out of the value import, per the standalone `import type` convention in CLAUDE.md. This also clears the Static checks import-sort failure. - Cover the new OAuth error branches in Login.spec.tsx: each mapped `?error=` code raises its localized toast and the query is cleaned, and unmapped codes including prototype members raise nothing.	2026-08-31 21:13:47 -04:00
Danny Avila	49e1e32f3a	🧺 fix: Funnel MCP Catalog Work Through a Shared Gate (#15429) ... * 🧯 fix: Bound MCP Catalog Work Across Requests * 🛑 fix: Reject Saturated MCP Discovery * fix: fairly bound MCP catalog work * test: type catalog snapshot mock arguments * test: assert snapshot deadline forwarding * fix: harden MCP catalog scheduling	2026-08-31 14:13:31 -04:00
Danny Avila	98eb3f8b1c	🧿 fix: Restore Shared IP Limits for Generation Retries (#15427) ... * fix: Preserve IP Limits for Generation Retries * test: Cover Scheduled Retry IP Limiting * style: format retry limiter route	2026-08-31 14:11:12 -04:00
Danny Avila	982a9a5141	🧴 fix: Strip LangChain URLs From Provider Errors (#15421) ... * 🧹 fix: Strip LangChain Attribution From User-Facing Provider Errors LangChain classifies provider failures by mutating `error.message` with a docs URL, and that message is forwarded verbatim into the error content part we persist, so users saw `docs.langchain.com/...` credited for a provider's own 429. The client only intercepted `MODEL_NOT_FOUND`, leaving every other code to render raw. Resolve the classification server-side instead: codes with localized copy become the typed payload the client already knows how to localize, and every other message keeps the provider's wording with the docs URL removed before it reaches the database. The untouched error still reaches the logs through `getSafeErrorMetadata`. The client keeps a code-driven lookup for messages persisted before this, and gains `com_error_model_rate_limit` alongside the existing not-found copy. The OpenAI-compatible and Responses controllers now share one provider-error helper rather than three copies of it, so the strip applies on those surfaces too. * 🛡️ fix: Harden Error Sanitization For Non-String Messages And Legacy Text Codex review follow-up. `stripLangChainTroubleshootingUrl` assumed a string, so an error-like object with a truthy non-string `message` threw `.replace is not a function` inside the error handler that was meant to report it — the template interpolation it replaced had coerced such values safely. It now coerces at the boundary, as does the code lookup. Legacy messages whose LangChain code has no localized copy still rendered the docs URL, which is exactly the historical case the client-side path exists to clean up. The rendered fallback is now stripped too. Both helpers move to `librechat-data-provider`, which is shared by the client and the backend, so one implementation serves both instead of a second regex drifting in the component.	2026-08-31 12:51:34 -04:00
Danny Avila	b7ae67307e	🕰️ fix: Anchor Temporal Variables to Turn Start (#15419)	2026-08-31 10:38:10 -04:00
Danny Avila	413906abc3	🖍️ fix: Honor Agent Image Detail Settings (#15407) ... * 🖼️ fix: Apply an Agent's Configured Image Detail The Agent Builder exposes an Image Detail slider and saves it to `model_parameters.imageDetail`, but nothing ever read it back: `encodeAndFormat` resolves the level from `req.body.imageDetail`, which the agents route never populates, so every image was encoded as `detail: "auto"`. Providers that reject `auto` — MiniMax's OpenAI-compatible API among them — fail every image upload through an agent, and the setting that would fix it has no effect. The root cause is a classification error rather than a missing argument. `extractLibreChatParams` splits LibreChat's own settings out of an agent's model parameters, and its key set is derived from the `librechat` group in `parameterSettings`. `imageDetail` sat in `baseDefinitions` instead, so it was neither extracted for LibreChat's use nor stripped from the options handed to the provider. It is now grouped with `resendFiles` and the other LibreChat-level parameters, which gives it the same path those already take: extracted during agent initialization, carried on the initialized agent, passed to the client, and read by `encodeAndFormat`. `AgentClient.getSaveOptions` already expected `options.imageDetail` — nothing had ever populated it. The request body still wins where it is set, so the classic endpoints keep resolving detail from conversation settings exactly as before. Fixes #15359 * 🔤 fix: Type Image Detail as Its Enum, Not a Bare String `TConversation.imageDetail` is the `ImageDetail` enum, so the extractor's return type and the initialized agent's field are now declared as that enum rather than widened to `string`, and the tests pass enum members instead of raw literals. Caught by CI's type check, which I had run before adding the tests and not after — jest compiles the specs through babel, so the suite stayed green over a type error.	2026-08-31 08:34:05 -04:00
Danny Avila	c7f0a8f247	📬 feat: Move Agent Queued Turns Server-Side (#15370) ... * feat: move agent queued turns server-side * test: update queued turn composition coverage * fix: harden queued-turn lifecycle ownership * fix: centralize queued-turn lifecycle ownership * fix: close queued-turn lifecycle gaps * fix: unify queued turn lifecycle transitions * fix: fence queued-turn deletion admission * fix: preserve queued-turn lifecycle ownership * fix: close queued-turn reconciliation races * fix: reconcile quarantined queued-turn admissions * fix: annotate admission evidence default * fix: deepen queued turn lifecycle admission * fix: fence queued turn provider admission	2026-08-30 21:53:58 -04:00
Danny Avila	fcae1025c0	🧳 feat: Register Principal-Owned Code Environments (#15365) ... * feat: add principal-owned code environments * fix: address code environment CI coverage * fix: expose principal code environments in endpoint config * fix: reject missing code environment bodies * fix: harden code environment lifecycle * fix: revalidate principal code environments * fix: synchronize code environment authorization * fix: bind code environments to current principals * fix: fail closed on code ACL cache errors * fix: prevent code environment override shadowing * fix: preserve principal environment defaults * fix: suppress revoked code environment aliases * fix: fail closed on environment augmentation * fix: narrow code environment defaults * fix: isolate code environment fallback * style: sort code config imports	2026-08-30 19:33:19 -04:00
Danny Avila	a9ccac8656	🧲 feat: Enable Secure Attached Environment Pairing (#15355) ... * feat: add secure code environment pairing * fix: satisfy code environment type checks * fix: secure code environment administration * fix: isolate code pairing control plane * fix: validate code pairing control responses * fix: secure code pairing transport * fix: validate code pairing wire format * fix: harden pairing secret lookup	2026-08-30 17:12:20 -04:00
Danny Avila	7533d138fa	🧬 perf: Evolve Compaction Guidance on Warm Turns (#15371) ... * perf: evolve compaction guidance on warm turns * style: sort compaction adapter imports	2026-08-30 17:11:20 -04:00
Danny Avila	30124f21b2	🎻 refactor: Orchestrate Agent Runs Through a Request-Free Host (#15366) ... * refactor: decouple agent initialization from HTTP * refactor: centralize remote agent execution lifecycle * fix: preserve pre-settlement error rendering * fix: preserve request-backed tool loading * style: sort agent execution imports * fix: adapt public agent tool loaders	2026-08-30 15:38:57 -04:00
Danny Avila	c1cb591d49	🗿 feat: Add Attached Stateful Code Environments (#15352) ... * feat: add attached stateful code environments * fix: include code environment in lazy agent type * fix: harden stateful environment routing * fix: complete code environment route isolation * fix: declare code environment map type * fix: preserve configured code execution routes * fix: harden stateful environment updates * test: assert route-scoped sandbox readiness * fix: harden stateful environment lifecycle * fix: isolate migrated code sessions * test: assert route-qualified code sessions	2026-08-30 15:38:45 -04:00
Danny Avila	cd3768ed1f	🍵 feat: Continue Late Steers in Warm Agent Runs (#15357) ... * feat: continue late steers in warm agent runs * chore: bump agents sdk to v3.7.9 * fix: guard terminal steer admission	2026-08-30 11:54:17 -04:00
Danny Avila	2ae6c8aea9	🛎️ feat: Wake Agents on Background Tool Completion (#15350) ... * feat: wake agents for background tool completion * fix: isolate background completion contracts * fix: anchor background completion identity * fix: close background completion delivery gaps * fix: preserve manual background polling * fix: preserve legacy tool group identity * fix: close background wakeup identity gaps * test: type background wakeup enqueue mock * fix: preserve background completion identity * test: type activity phase fixture * test: mock phase media query * fix: arbitrate background result ownership * fix: retain tool step routing helper * test: expand phase groups before identity checks * fix: preserve background completion ownership * fix: bound durable background results * fix: type wakeup input budget export * chore: sort background handler imports * fix: persist timed-out background completions * fix: reconcile background completion ownership * fix: type background completion capabilities * fix: harden background completion terminalization * test: type missing completion evidence * fix: require evidence for completion retirement * chore: satisfy background completion static checks * test: cover automatic background completion wakeups * test: assert background wakeup agent identity * fix: wake capability-fenced trigger deliveries * test: keep capability shield fixture public * test: assert capability worker wakeup * fix: close background completion ownership gaps * test: satisfy completion lease static checks * fix: preserve artifact completion wakeups * fix: close background delivery recovery gaps * fix: simplify background receipt guidance * fix: recover dead background completion batches * fix: fence background completion recovery * fix: recheck background recovery ownership * fix: fence unpublished background continuations * test: await background message hydration	2026-08-30 08:56:41 -04:00
Danny Avila	e3ccaba5af	🛄 feat: Restore Compaction Guidance Across Continuations (#15356) ... * 🧭 feat: preserve compaction guidance across continuations * 🔧 fix: narrow persisted compaction fields	2026-08-30 08:13:24 -04:00
Danny Avila	70f735336d	🛎️ fix: Enroll Remote Agent Runs in the Generation Lifecycle (#15349) ... * fix: enroll remote agent runs in generation lifecycle * fix: close remote lifecycle ownership gaps * fix: close remote conversation drain races * fix: fence remote runs during conversation deletion * fix: reconcile remote deletion and settlement races * fix: complete owner deletion recovery * fix: preserve remote cleanup receipts * fix: consume deletion receipts before cleanup * fix: expose idempotent deletion option * chore: sort remote lifecycle imports	2026-08-30 07:14:13 -04:00
Dustin Healy	8fcab7e44f	🔄 fix: Recover Missing MCP Marketplace Catalogs (#15323) ... * fix: recover missing MCP marketplace catalogs * fix: make MCP catalog recovery passive * test: type MCP catalog recovery fixtures * fix: bound and back off passive MCP catalog recovery Passive recovery runs inline on `GET /api/mcp/tools` and its results are request-local by design, so every list request re-dialed the same cold servers with the default connection timeout. Three limits keep that cost proportional to what recovery can actually recover: - Cap the discovery timeout at 5s instead of inheriting the connection default (`initTimeout ?? 30s`); a server configured to connect faster keeps its own shorter limit. - Skip a server the config tier already marked `inspectionFailed`, leaving it to that tier's retry window rather than re-dialing it per request. - Skip a server whose declared `customUserVars` are unset, matching the gate `reinitMCPServer` applies for issue #10969 — connecting without them fails auth, so the attempt is spent for nothing. Servers that still fail discovery enter a one-minute per-process cooldown, which is what stops an unreachable server from being re-dialed by every subsequent list request. A server that recovers clears its own entry, and expired entries are swept at most once per window so the map stays bounded. Skipped servers render exactly as they did before recovery existed: present in the catalog with an empty tool list. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xr1Dabvdn1mzzyYgpJgU5B * fix: bound passive MCP recovery by deadline, key cooldowns by config Both follow-ups address the same mistake: recovery expressed its own request-level constraints in terms borrowed from other layers. `connectionTimeout` bounds one connection attempt, and `MCPConnectionFactory.discoverToolsInternal` spends it twice — once on the authenticated connection, then again in `attemptUnauthenticatedToolListing` — so capping it bounded no total this layer could reason about. Recovery now enforces its own wall-clock deadline per server with `withTimeout`, which holds however many attempts the factory makes; `connectionTimeout` is left to do only its own job, still honouring a shorter operator `initTimeout`. An attempt abandoned by the deadline disposes its own connection when it settles, and `Promise.race` keeps a handler on it, so a late rejection is not unhandled. A per-request budget now caps total recovery regardless of server count. A server is dialed only if the remaining budget can fund a full deadline; never dialing one is not evidence against it, so a skipped server records no cooldown and a later request reaches it once those ahead are cached or cooling down. Cooldown identity now includes the publication generation — the same effective-config identity the tool caches fence on — instead of just user and server name. Correcting a server's URL or transport keys a new entry, so the refetch the client issues on update is no longer skipped for up to a minute by the previous configuration's failure. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xr1Dabvdn1mzzyYgpJgU5B * refactor: keep passive MCP recovery stateless and bounded by its own work Reverts the cooldown, request budget and deadline race added in 257d5cf and fc3e3c9, and keeps only the three stateless limits. The tool cache refuses unfenced writes (`tools.ts`), and a discovery connection owns no publication generation and is disposed, so a recovered catalog cannot be retained by design. Those commits responded by building a cache-shaped memory in front of it — per-process failure state, a scheduling budget, an identity, an eviction sweep — and each round of review found another way that hand-rolled cache differed from a real one: wrong identity for configuration, wrong identity for credentials, no fairness across requests, and a limiter slot released while its network operation was still running. None of that machinery was asked for; all of it was compensation for a result the architecture does not allow keeping. Recovery is now stateless. It skips only what configuration alone proves pointless — a server the config tier already marked `inspectionFailed`, and one whose declared `customUserVars` are unset — and bounds the work itself rather than racing it, so a limiter slot is held for exactly as long as its network operation runs and the concurrency limit of three is real. The attempt timeout is not a compromise: recovery exists for a server that is reachable and authorized but whose catalog cache expired, and such a server answers tools/list well inside 1.5s. Anything slower cannot be rescued here, so failing fast costs nothing. The factory spends that value per attempt, so a server's ceiling is it times the attempts made; the constant documents that rather than hiding it behind a number tuned to today's attempt count. Consequences that were bugs are now gone by construction: every cold server is attempted on every request, so none is starved by those ahead of it, and correcting a server's configuration or credentials takes effect on the next refetch instead of waiting out a stale cooldown. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xr1Dabvdn1mzzyYgpJgU5B * fix: correct the inspection-failure skip and bound catalog fan-out Three fixes that belong to this layer; a fourth issue does not, and is described below. The `inspectionFailed` skip was too broad. `MCPServersInitializer` stores a YAML server that was unreachable at startup via `addServerStub`, which stamps `source: 'yaml'`, and only config-tier entries get the timed retry in `ensureSingleConfigServer`. Skipping every failed stub therefore hid a recoverable server from the marketplace permanently — the exact state this recovery exists to escape. It now defers only `source === 'config'`, matching what `reinitMCPServer` already does. Plugin auth is read only when some cold server actually declares `customUserVars`, and only for those servers. The common unauthenticated case no longer pays a MongoDB round trip whose result nothing can consume. Snapshot refreshes are now bounded by the same limiter as discovery. They are not local reads: both connection paths reach `fetchOrderedToolsSnapshot` and issue a real `tools/list`, so a cache reset across many servers previously burst unbounded outbound requests while discovery was capped at three. Not fixed here, because it cannot be: `connectionTimeout` does not bound discovery. It covers `connection.connect()` only, and `fetchToolsSnapshot` then applies its own `TOOLS_LIST_TIMEOUT_MS` (30s) to `tools/list`, so a server that connects fast and stalls while listing still holds its slot for that window. The factory also does not cancel a timed-out connect before starting the unauthenticated fallback. Bounding this end to end needs a deadline threaded through `MCPConnectionFactory` into both `connect()` and `fetchToolsSnapshot()`, which is a change to shared connection machinery rather than to this caller. The constant's comment now states what it does and does not bound instead of implying an end-to-end guarantee. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xr1Dabvdn1mzzyYgpJgU5B * fix: thread live-session OBO context into passive catalog discovery The merge of #15334 sources OBO tokens from the live OpenID session via request-boundary closures. Passive catalog recovery is a discovery call site too; without these options an OBO server whose stored token went stale fails recovery — the exact cold-catalog class this PR fixes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xr1Dabvdn1mzzyYgpJgU5B --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-30 06:55:30 -04:00
Danny Avila	16c2bb4149	📬 feat: Establish Background Continuation Admission (#15348) ... * feat: establish background continuation admission * fix: preserve routed subagent wakeup controls * fix: probe configured subagent task store	2026-08-30 06:51:36 -04:00
Danny Avila	fa913148fb	🔒 fix: Refresh MCP OBO Tokens From the Live OpenID Session (#15334) ... * 🧊 fix: Inline-refresh OpenID session tokens at MCP OBO call time Resolves the walk-away failure mode where MCP tool calls using OBO auth fail with "No valid OpenID access token is available for OBO exchange" after a user idles past their access-token lifetime. The strategy-time snapshot on `user.federatedTokens` could expire mid-stream before `resolveOboToken` ran, while `req.session.openidTokens` carried a still- valid (or refreshable) token that nothing read. - New OpenIDSessionRefresh service: per-user single-flighted closure that reads `req.session.openidTokens` at OBO time and inline-refreshes via `openid-client.refreshTokenGrant` when expired (30s skew), persisting via `req.session.save()`. No cookie writes (headers already flushed). - `resolveOboToken` gains a required UpstreamTokenProvider parameter (typed as `() => Promise<OIDCTokens | null>`, reusing the shared shape from @librechat/data-schemas). Compile-time guarantee that every call site is updated. - New `session_refresh_failed` OboTokenResolutionReason distinguishes "session expired and IdP rejected refresh" from "no upstream token ever existed." - `req` threaded through createMCPTool/createMCPTools/createToolInstance to construct the closure with captured request, plus fail-closed guards in MCPConnectionFactory.getOboTokens and MCPManager.callTool when the closure isn't plumbed. - Startup warning in MCPServersInitializer when OBO is configured but OPENID_REUSE_TOKENS is unset (the strategy populating user.federatedTokens is only registered under reuse, so OBO would fail every call without it). Tests: 16 new in OpenIDSessionRefresh.spec.js; obo.spec.ts extended for the new param + error reason; wiring smoke tests in MCPManager, MCPConnectionFactory, MCPServersInitializer, and MCP.spec.js. * 🛡️ fix: Harden OBO inline-refresh against token type and session edge cases - Token-preference asymmetry: live-token reuse and expires_at derivation now strictly gate on the access_token, not the id_token. Added a required `tokenPreference` parameter on isLiveSessionTokenStillValid, buildOIDCTokensFromSession, and createOpenIDSessionTokenProvider so every call site is explicit. Dropped the bogus id_token-exp fallback in performIdpRefresh — id_token TTL is governed by IdP session policy and would mark a short-lived access_token reusable past its real lifetime. - Missing req in /reinitialize route: the manual reconnect endpoint now forwards req into reinitMCPServer, so OBO servers can build a session-aware upstream-token closure instead of failing with missing_upstream_token. - Single-flight key collisions: composed key as tenantId:openidIssuer:openidId:sessionId via getSingleFlightKey. Concurrent calls in the same session still coalesce; separate sessions never share an in-flight refresh, preventing refresh-token rotation from breaking sibling sessions and preventing cross-tenant token crossover when distinct users share an IdP sub. - Opaque access token reuse): persist accessTokenExpiresAt (unix seconds, from tokenset.expires_in) on each refresh AND on initial login / SPA refresh in setOpenIDAuthTokens. New getAccessTokenExp helper falls back to it when the access token isn't a JWT, avoiding redundant inline refreshes for Microsoft Graph and Auth0 default audiences. - Log hygiene: the single-flight key (containing sessionId, openidId, openidIssuer, tenantId) is now SHA-256-hashed in the "Joining in-flight refresh" debug log. Preserves cross-line correlation via a 12-char prefix without leaking credential or PII material. Documented req.session.openidTokens shape contract via JSDoc typedef so the new accessTokenExpiresAt field has a discoverable home alongside the existing accessToken/idToken/refreshToken/expiresAt/lastRefreshedAt. Tests: OpenIDSessionRefresh.spec.js up to 30 passing (added coverage for opaque-token reuse, JWT-access-token-exp fallback, no-id_token-fallback regression, cross-session no-coalesce, persistence on refresh, and a guard against stale accessTokenExpiresAt carryover). AuthService.spec.js adds two cases covering accessTokenExpiresAt persistence on login. mcp.spec.js (route) gains a regression test asserting req flows into reinitMCPServer. * 🔍 fix: Detect OBO-only MCP admin config overrides Admin Config overlays for YAML-defined MCP servers compare only ADMIN_CONFIGURABLE_FIELDS to decide whether to lazy-init a config-tier override. The OBO config field was added after that fingerprint list, so an override that only added or changed `obo` was treated as unchanged YAML and skipped. Include `obo` in the admin-configurable field list and add a regression test for an OBO-only override. * 🔊 fix: Mock MCP OAuth timeout in SDK integration test MCPConnectionFactory.attemptToConnect reads mcpConfig.OAUTH_HANDLING_TIMEOUT when building the OAuth connection timeout. The SDK OAuth integration test mocked mcpConfig without that field, which made the timeout calculation produce NaN and caused the test to fail before the OAuth refresh/start path completed. Add OAUTH_HANDLING_TIMEOUT to the test mock. * ♻️ refactor: Pass OBO upstream-token closure into MCP instead of req Build the OpenID upstream-token provider at the request boundary and thread only the closure through MCP handling, so the MCP service layer no longer receives the raw Express request. The closure still reads/refreshes the live session at tool-call time, preserving the walk-away recovery. - Drop `req`/`capturedReq` from createMCPTools, createMCPTool, reconnectServer, createToolInstance, and reinitMCPServer; forward `upstreamTokenProvider` instead. Closure is constructed in loadTools, loadToolDefinitionsWrapper, and the reinitialize route, where req/res are in scope. - OBO: fall back to user.federatedTokens when the provider yields no live session, so OIDC remote-agent calls (verified bearer, no session) still work. - Inline refresh: mirror a rotated refresh token to the refreshToken cookie via a shared setRefreshTokenCookie helper, guarded by !res.headersSent (no-op on the streaming path; session copy stays authoritative). - Single-flight: hydrate a joining request's own session from the resolved tokens so a later OBO call doesn't replay a rotated-away refresh token. Addresses owner feedback and three review findings. * 🔒 fix: Recover OIDC refresh-token rotation after SSE OBO refresh When an inline OBO refresh rotates the OpenID refresh token after SSE headers have already been sent, the browser refreshToken cookie cannot be updated. Store a short-lived encrypted bridge from the stale cookie token to the rotated token so /api/auth/refresh can recover after express-session loss. Use the signed openid_user_id cookie to load user context for bridge validation, retry only on invalid_grant, and delete the bridge only after the bridged refresh succeeds. * 🔨 fix: hydrate joined OIDC refresh sessions with stable refresh tokens Update single-flight OIDC refresh joiners whenever refreshed access token state changes, even if the IdP keeps the refresh token unchanged. This prevents joined requests from retaining stale accessToken or accessTokenExpiresAt values and redundantly refreshing later in the same run. * 🌉 Persist OIDC refresh-token recovery bridges in MongoDB Store SSE OBO refresh-token recovery bridges in MongoDB instead of process-local memory so /api/auth/refresh can recover after worker restarts or cross-worker routing. Derive bridge expiry from REFRESH_TOKEN_EXPIRY so the recovery window matches the stale refreshToken cookie it repairs, and delete bridges after successful recovery. * 🤝 Coordinate OIDC inline refreshes across workers Add a short-lived Mongo-backed refresh-flight record so concurrent OBO refreshes for the same OpenID session do not redeem the same rotating refresh token on different workers. The winning worker performs the IdP refresh and stores an encrypted result; joiners wait for that result, hydrate their request session, and return without calling the IdP. * ⚓ Keep OpenID marker cookies aligned on inline refresh Refresh token_provider and openid_user_id with the same expiry as the rotated refreshToken cookie when an inline OBO refresh can still write headers. Share the marker-cookie writer with the normal OpenID auth refresh path so the fallback /api/auth/refresh branch continues to recognize valid OpenID refresh tokens after session expiry. * 🔑 fix: include refresh token in OIDC local refresh flight key Key the process-local OIDC refresh coalescing by the current session refresh token, matching the Mongo-backed flight key. This prevents a request with a newly rotated token from joining an older pending refresh and inheriting its failure/result. * 🌉 fix: store OIDC refresh bridge without cookie response Treat missing or non-cookie responses like headers-sent streaming responses during inline OIDC refresh. When the IdP rotates the refresh token and cookies cannot be written, persist a recovery bridge so a later /auth/refresh can recover after session expiry. * 🫙 fix: preserve stale OIDC cookie bridge key Track the refresh token last written to the browser cookie separately from the current session refresh token. When inline OIDC refreshes rotate tokens without a writable response, keep bridging from the browser-stale token directly to the latest session token. * 🙌 fix: keep OIDC bridge recovery success on cleanup failure Make refresh-token bridge cleanup best-effort after a bridged OIDC refresh succeeds. A transient delete failure now logs a warning but does not convert the already-refreshed session and cookies into a 403 response. * 📦 test: Exclude RefreshTokenBridge from tenant-isolation coverage Add RefreshTokenBridge to the tenant-isolation coverage allowlist because refresh bridge lookups run during unauthenticated OpenID refresh recovery. The controller first recovers user context from the signed OpenID marker cookie, then the bridge methods apply explicit user and tenant filters. Ambient tenant isolation would bind this recovery path to request-local tenant context that is not available at the point the stale cookie is being resolved * ⚡ Fix OpenID refresh flight retry and marker hydration Allow failed OpenID refresh flights to be reclaimed immediately instead of pinning transient errors. Preserve the browser refresh-token marker when joined refreshes hydrate session tokens from a shared flight result. Stabilize AuthService tests by isolating mocked module imports from prior suites. * 🛠️ fix: centralize OBO identity scoping Add shared auth identity helpers for app user ids, OpenID subjects, tenant ids, and normalized OpenID issuers. Thread a non-placeholder-visible OBO identity context from the real request user through MCP connection, tool-call, reinit, and refresh paths. Keep tenantId and openidIssuer out of createSafeUser so MCP user placeholders do not expose those fields. Scope OBO token cache and in-flight exchange keys by tenant, issuer, OpenID subject, scopes, and a SHA-256 hash of the upstream assertion. This prevents cross-tenant/cross-issuer collisions and avoids reusing tokens minted from stale rotated assertions. Use the shared identity helpers for OpenID refresh-flight keys and refresh-token bridge recovery records so related OBO refresh paths share the same identity normalization rules. The helper is intended for auth-boundary and credential-cache code, not as a blanket replacement for ordinary app user id ownership checks. * 🛠️ fix: preserve OIDC refresh-token sync on save failures Sync OpenID refresh-token cookie/bridge state before persisting the session so a transient session-store failure cannot lose an IdP-rotated refresh token. Also trigger sync when the session refresh token differs from the browser refresh-token marker, not only when the current grant rotates the token. This lets later writable refreshes repair stale browser cookies left behind by SSE refreshes. Route refresh bridge identity through the shared identity helper with the threaded OBO identity context, falling back to request/user context when needed. Add regression coverage for session-save failures, stale browser cookie repair, non-writable bridge storage, and shared-helper identity fallback. * 🛠️ fix: keep OIDC refresh bridge during recovery grace After successful bridged refresh recovery, re-store the stale-cookie bridge with a short grace TTL instead of deleting it immediately. This lets parallel /api/auth/refresh requests that already sent the stale browser cookie recover before they can observe the first response's Set-Cookie. Retarget the bridge to the refresh token returned by the bridged retry so B-to-C refresh-token rotation remains recoverable. The grace TTL is parsed with math() and defaults to 60s, which shrinks the replay window from the original REFRESH_TOKEN_EXPIRY bridge lifetime to the short recovery grace period. Remove the now-unused explicit bridge delete path from the service and data-schemas method surface. Add regression coverage for grace re-store, identity symmetry, retry failure behavior, and same-key upsert replacement. * 🛠️ fix: fail closed on OBO MCP user identity mismatch Add an OBO-specific guard before MCP tool execution that requires the effective invocation user and captured request user to both have ids and to match. This prevents OBO tool calls from falling back to a separate configurable.user_id identity after request-bound OBO context has already been captured. Keep the existing user id fallback behavior for non-OBO MCP calls. Tests cover mismatched OBO users, missing user ids, and the matching-user path ignoring a conflicting configurable.user_id. * 🛠️ fix: Guard OpenID bridge retry user identity Extract the shared OpenID refresh/user-resolution flow in AuthController so the normal refresh path and bridge-recovery retry use the same grant, claims, issuer, user lookup, and diagnostic logging code. Preserve the existing path-specific behavior: the normal path still owns migration updates and 401 login redirects, while the bridge retry still falls through to the existing 403 invalid-token response. Add a bridge-recovery guard that rejects retry results whose resolved user id differs from the signed openid_user_id cookie before issuing tokens or re-storing the grace bridge. Cover both the successful matching-user recovery and the mismatched-user rejection. * 🛠️ fix: type-safety polish on OBO data layer Replace refresh token bridge query/update Record<string, unknown> usage with typed Mongoose FilterQuery and UpdateQuery definitions. Harden OpenID marker cookie JWT expiry handling by converting refresh expiry milliseconds to integer seconds and rejecting invalid or non-positive durations. Add focused CSRF tests for fractional refresh expiry values and invalid expiry configuration. * 🛠️ fix: Bind OpenID session tokens to authenticated identity Stamp OpenID session token state with the LibreChat user id, OpenID subject, tenant id, and normalized issuer when tokens are stored. Fail closed before OBO inline token reuse/refresh when the session token identity does not match the current authenticated identity, preventing a stale or mixed Express session from supplying another user's upstream assertion. Also validate the normal /api/auth/refresh session-token reuse shortcut against the signed marker-cookie user before returning cached session tokens. Note: sessions created before this change carry no identity stamp and are treated as a mismatch. This is self-healing — the reuse path forces a full IdP refresh (which re-stamps the session) and the OBO path throws, surfacing as a one-time re-authentication for active OBO users at deploy time. The session re-stamps within one session lifetime (SESSION_EXPIRY, default 15 min). * 🛠️ fix: Recover OpenID refresh token drift Prefer the browser refresh-token cookie when it differs from the server-side OpenID session state, and force a real IdP refresh in that case instead of reusing stale session tokens. Store a short-lived refresh-token bridge when inline OBO refresh writes a rotated browser cookie but session persistence fails, so follow-up refreshes can still recover from the old token. Keep the bridge grace TTL centralized in RefreshTokenBridge so both recovery paths use the same env-backed value. Note: drift is measured against the last-synced browserRefreshToken marker, so the SSE path (intentionally stale cookie, authoritative session) does not false-positive. Sessions predating the marker have no browserRefreshToken; for those, drift falls back to comparing the cookie against the session refresh token and prefers the cookie on difference. This is the same self-healing pre-change-session window as the identity binding fix and re-syncs within one session lifetime. Tests cover cookie/session drift selection, reusable-session bypass on drift, bridge storage after session-save failure, and the shared bridge constant wiring. * 🛠️ fix: Harden OBO token caching and expiry handling Reject malformed OBO grant responses before writing them to the exchanged-token cache so a missing access_token cannot poison the cache. Store absolute expires_at values with cached OBO tokens and ignore legacy cache entries without usable expiry metadata. This keeps cached-token freshness based on the token’s real remaining lifetime instead of reusing the original relative expires_in on cache hits. Move OBO expiry normalization and skew helpers into packages/api and use them from both the JS exchange service and the TS MCP resolver. Apply a 30-second safety margin with a one- second floor for short-lived tokens, covered by direct helper tests and caller-level regression tests. Tests: - packages/api: npm run build - packages/api: npx jest src/mcp/oauth/expiry.spec.ts src/mcp/oauth/obo.spec.ts - api: npx jest server/services/OboTokenService.spec.js * 🛠️ fix: Harden OBO refresh-token bridge lookup and indexing Reuse getValidOpenIDReuseUserId for the bridge-recovery user lookup in refreshController instead of re-verifying openid_user_id inline. The shared helper enforces the JWT_REFRESH_SECRET presence check and a strict typeof payload.id === 'string' guard, rejecting tokens whose id claim is present but not a string (e.g. a numeric id) that the inline check accepted. Fail closed on issuer mismatch in getRefreshTokenBridge. Both the stored and the expected issuer are now normalized and compared for equality, so a bridge is recovered only when both sides agree (both absent, or both present and equal after normalization). Previously the check was skipped whenever the stored issuer was absent, allowing recovery across mismatched issuer context. Drop the unused {oldRefreshTokenHash, userId, tenantId, openidIssuer} index and the openidIssuer field on RefreshTokenBridgeQuery. The data-layer filter only queries the 3-field {oldRefreshTokenHash, userId, tenantId} index; the issuer is verified in application code, not the query. Hoist the repeated model accessor into getRefreshTokenBridgeModel. Note: issuer is now load-bearing for recovery. A bridge stored with an issuer recovers only when the lookup supplies a matching issuer; the recovery lookup reads user.openidIssuer via AUTH_REFRESH_USER_PROJECTION (an exclusion projection that retains the field). If a user's persisted openidIssuer is empty while the stored bridge has one, recovery fails closed (falls through to normal re-authentication) until the bridge TTLs out — no security regression. Tests cover invalid signed-cookie payloads bypassing the bridge, both asymmetric issuer-presence cases, issuer normalization before comparison, and an index-alignment assertion guarding against re-adding the dropped index. * 🛠️ fix: Degrade OBO discovery on token resolution failures Catch expected OboTokenResolutionError failures during MCP tool discovery and fall back to unauthenticated tool listing instead of aborting discovery. This keeps discovery aligned with the existing unauthenticated listing behavior while preserving unexpected errors as real failures. Also correct OBO tool-call freshness comment and tighten the OBO trust-check permissions type to the existing role permission shape. Tests: - npx jest src/mcp/__tests__/MCPConnectionFactory.test.ts --runInBand --coverage=false - npx jest src/mcp/oauth/obo.spec.ts --runInBand --coverage=false * 🛠️ fix: tighten OBO tool-call errors, bridge logging, and flight typing Move resolveToolCallUserId inside the tool-call try/catch so an OBO identity mismatch surfaces with serverName/toolName context and the standard tool-call-failed message instead of an opaque bare Error. Raise the refresh-token bridge lookup failure log from debug to warn so transient infrastructure failures on the unauthenticated /api/auth/refresh path are observable, and guard the message access against non-Error values. Replace the unknown+cast in isDuplicateKeyError with a hasErrorCode type predicate so the duplicate-key check reads error.code without an assertion. Preserve real math/isEnabled in the MCPConnectionFactory test mock (mock only processMCPEnv) so mcpConfig timeouts no longer resolve to NaN, fixing the TimeoutNaNWarning that masked slow OAuth retry behavior. * 🧪 fix: Restore the Flight Uniqueness Index and Buffer the Graph Cache TTL Two CI failures on the merge, both in suites this environment cannot run (their MongoDB binary download is blocked). `GraphApiService.spec.js` still asserted the unbuffered TTL. Graph tokens route through the same `getTokenCacheTtlMs` as the OBO and openidStrategy caches, so the entry now expires 30s before the credential does. `openidRefreshFlight.spec.ts` dropped the database between tests, which takes the indexes with it, and Mongoose builds them only once when the model is compiled. Whether the unique `key` index survived into a test was a race with that one-time build. Without it a second `create` inserts instead of raising a duplicate-key error, so every worker believes it won the flight — the mutual exclusion the file exists to prove. Indexes are now rebuilt after each drop, which also makes the reclaim and complete cases reach those paths for the right reason. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * fix: address OBO review findings * 🔐 fix: Install Bridge Indexes and Carry OBO Through Assistant Recovery Two findings from the Codex pass on d08c82f0d. The refresh-token bridge relied on Mongoose auto-indexing for both of its indexes, and `MONGO_AUTO_INDEX=false` is a supported deployment setting. A bridge holds an encrypted refresh token and the TTL index is the only thing that ever deletes one, so under that setting they would accumulate for the life of the collection while concurrent upserts lost the compound uniqueness the filter assumes. Installed before the first write, matching the flight methods and the session and schedule methods before them. `recoverServerTools`, the assistant create/update path that reruns `reinitMCPServer` when a referenced server's catalog and connection snapshot are both missing, was the last reinit site not carrying the upstream-token closure. For an OBO server the factory rejects the connection outright, so the assistant write failed with unavailable MCP definitions. It now builds the provider at that request boundary like the other entry points; assistant writes have no `res`, so a rotation there falls back to the recovery bridge. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * fix: harden OBO refresh coordination * ⏳ fix: Keep Elapsed Expiries Elapsed and Revoke the Superseded Session Two of the five findings from the Codex pass on fcdc15885 — the two that are defects in code this branch introduced rather than design questions about the bridge. `getSkewedTokenExpiresAtMs` floored every result at a second in the future, including an expiry the provider had already declared elapsed. An exchange answering `expires_in: 0` or a past `expires_at` was handed to the MCP connection stamped valid for another second, which only moves the failure downstream. The floor now applies to a lifetime that is still live, which is what it was for; an elapsed one stays elapsed so the caller rejects it. Same for the cache TTL, which falls back to the elapsed-credential floor. Bridge recovery left the stale token's durable Session behind. Only the token it recovered through was passed as `existingRefreshToken`, so that one's session was replaced while the token the browser actually presented kept its record until its original expiry. That record, with the marker cookie still bound to it, is what authorizes local image access for OpenID users — so a copy of the stale cookie outlived the rotation it had lost. Revoked explicitly on successful recovery. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * fix: close OBO refresh review findings * 🎟️ fix: Carry the Bridged Token Through a Non-Rotating Recovery Bridge recovery passes the browser's stale token as `existingRefreshToken` so the durable Session naming it is the record replaced. That also makes the stale token the fallback the installed session and the refresh cookie use when a tokenset carries no `refresh_token` of its own, which holds only while the recovery grant rotates. An IdP that answers that grant without rotating sends the browser back to the very token the bridge exists to retire: `storeOpenIDSession` installs and deletes the same stale record in one call, and the cookie is rewritten to a token the IdP already rejected — a sign-out on the next refresh. The grace bridge one line above already guards this with `|| bridgedRefreshToken`; the resolved tokenset now does the same, so leader and followers alike publish the recovered token. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * 🚫 fix: Reject an OBO Exchange That Returns an Expired Credential Preserving an elapsed expiry through the skew helper only helps if something acts on it, and nothing did: `MCPManager.callTool` checks the access token and nothing else before setting the Authorization header, so a credential the IdP declared spent still went downstream to fail there. It is rejected at the exchange now, where the reason is known, and retryably — the exchange itself worked, so a fresh grant can succeed. Completes the elapsed-expiry change in cb26a6f7d, which made the stamp honest without giving anyone a reason to look at it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * fix: close OpenID refresh review findings * fix: coordinate OpenID refresh entry points * chore: sort OpenID flight imports * fix: fence OpenID refreshes during logout * fix: close OpenID logout publication races * fix: narrow completed refresh flight * test: cover bridge cleanup failure after ownership loss The compensating delete in storeRefreshTokenBridgeWithLease swallows its own failure so the lease error stays the one the caller sees. Nothing asserted that, so removing the inner catch left every suite green while callers began receiving the cleanup error instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * fix: compensate OpenID bridges only on proven ownership loss The post-write lease assertion deletes the bridge it just published when it throws, but it threw for two different reasons: a coordination record that is no longer ours, and a coordination read that simply failed. Treating the second as the first destroys the only mapping from the token the browser still holds to the one the IdP already rotated to, so a transient Mongo error on the headers-already-sent path signed the user out. Tag the ownership error where the lease raises it and compensate only for that, preserving the bridge whenever ownership is merely undetermined. A preserved bridge stays behind the logout revocation fence, so the safe default costs nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * fix: reject spent OpenID refresh results Two ways a refresh could report success while handing back a credential nothing can use. An inline refresh carries the previous id_token forward when the IdP omits one on rotation, so tokenset.id_token is not necessarily freshly issued. setOpenIDAuthTokens applied its freshness guard only to the session copy and took tokenset.id_token unconditionally, so /refresh returned an expired bearer even though the grant produced a usable access token. Skip it only when it is provably expired: an id_token whose expiry cannot be read stays preferred, since access_token may be opaque or scoped to another audience. normalizeExpiresIn preserves a zero or negative lifetime rather than discarding it, so a grant declaring an already-spent access token still published, rotating the refresh token and returning a token every freshness check rejects. Each OBO call then repeated the grant. Reject an elapsed lifetime before publishing; an unknown lifetime still publishes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * fix: harden OpenID refresh publication * fix: keep identity and results intact through OpenID refresh cleanup Two follow-ons from the last round's fixes. Stripping an expired carried-forward id_token from the refresh result removed the only identity material a rotation without id_token leaves behind. The result is rebuilt by buildOIDCTokensFromSession, so it carries no provider claims() either, and getTokenClaims accepts only those two — bridge recovery failed with "no usable identity claims" before setOpenIDAuthTokens could hand back the fresh access token. The stripped token now travels in a non-enumerable marker, alongside the existing browser and predecessor markers, which identity resolution reads and the authentication response never sees. The lease drained a pending renewal by awaiting it in finally, so a transient coordination failure there threw from finally and replaced the operation's result. The refresh had already settled and published, so the caller saw a failure on credentials that had rotated. Proven ownership loss is recorded on ownershipLost and checked before the return, so the drain has nothing to add but noise; it now absorbs and logs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SxKWxwqxAGckYpRsYTqx3F * refactor: fence OpenID recovery publication * fix: close OpenID publication transaction * refactor: make OpenID publication transactional * fix: satisfy OpenID publication type checks * fix: fence OpenID session publication * fix: authorize OpenID refresh publication * fix: bind OpenID replay generations * fix: fence OpenID response generations * fix: authorize OpenID token delivery * fix: linearize OpenID publication delivery * chore: sort OpenID refresh flight imports --------- Co-authored-by: J.C. Bartle <jcbartle@users.noreply.github.com> Co-authored-by: jbartle <jbartle@rand.org> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: jcbartle <7274202+jcbartle@users.noreply.github.com>	2026-08-29 23:55:58 -04:00
Danny Avila	10981b8637	🍱 feat: Guide Compaction With a Bounded Semantic Index (#15340) ... * 🧭 feat: guide compaction with semantic context * 🛡️ fix: fail closed on semantic intent collisions	2026-08-29 17:53:04 -04:00
Danny Avila	7a0061507e	🪪 fix: Admit Confirmed Generation Retries Before Message Limits (#15341) ... * fix: admit idempotent retries before message limits * fix: bound generation retry admission * fix: keep retry probe store-compatible * fix: exclude normalized resume routes * fix: preserve trusted retry exemptions * fix: bound retry claim admission * style: satisfy generation retry static checks	2026-08-29 13:41:26 -04:00
Danny Avila	773127bff2	🎠 refactor: Route Every Event Actor Turn Through One Lifecycle (#15325) ... * refactor: unify Event Actor turn lifecycle * fix: retain Event Actor fence ownership * fix: preserve mixed-version actor suspension safety	2026-08-28 17:30:23 -04:00
Danny Avila	3fa33b740b	🛫 refactor: Promote Generation Protocol V2 Automatically (#15324) ... * refactor: promote generation protocol v2 automatically * fix: remove unused protocol import	2026-08-28 17:17:09 -04:00
Danny Avila	77c2a51cf3	🪧 fix: Advertise Detached Event Actor Support from the Generation Store (#15322) ... * fix: activate detached event actions automatically * fix: shield detached terminal generations * perf: share capability availability index * style: flatten capability status selection * fix: preserve mixed-version lifecycle shells * fix: complete detached action store adapters * fix: close mixed-version capability races * fix: honor legacy capability success	2026-08-28 16:33:23 -04:00
Danny Avila	487984193a	🪂 feat: Make Event Actor Detached Actions Durable (#15307) ... * feat: make Event Actor detached actions durable * fix: align detached actor checks with merged base * fix: close detached actor lifecycle gaps * fix: break detached action outcome cycle * fix: deepen detached action ownership * fix: fence detached launch handoffs * fix: preserve detached rollout fencing * test: type detached recovery failure * fix: close detached ownership handoffs * fix: stage detached action activation * fix: persist detached terminal retries * fix: fence detached durability ownership * fix: fence trigger lane publication * fix: fence detached transition generations * fix: retire failed detached predecessors * fix: close detached rollout consumers * test: type detached retry turns * fix: close detached ownership gaps * fix: fence detached action resumes * fix: fence prior-head event actor resumes * fix: preserve legacy event actor resumes * fix: align detached resume boundaries	2026-08-28 14:21:04 -04:00
Danny Avila	01f5391ee3	🕰️ fix: Guard expires_in So a Token Response Cannot Outlive Its Credential (#15321) ... * 🕰️ fix: Guard `expires_in` So a Token Response Cannot Outlive Its Credential RFC 6749 §5.1 makes `expires_in` only RECOMMENDED, so a token response may legally omit it. Four sites derived a lifetime from the raw field, where `undefined * 1000` is `NaN`. `NaN` is not a short TTL, it is no TTL. `@keyv/redis` writes the key without `PX` because `NaN` is falsy, so the entry is stored in Redis with no expiration at all; the in-memory backend embeds `expires: NaN` and every check compares with `>`, always false against `NaN`. The namespace default does not stand in either, since Keyv applies it with `??=` and `NaN` is neither `null` nor `undefined`. The exchanged access token was therefore cached permanently at `openidStrategy.js` and `GraphApiService.js`, and once it genuinely expired the poisoned entry kept being served with no path to eviction. The same omission is sharper in `ActionService.js`, where `new Date(NaN).toISOString()` throws `RangeError: Invalid time value`. Both call sites are inside a `try`, so the failure surfaces as a generic "Failed to authenticate OAuth tool" that names nothing, and the refresh site falls through to `requestLogin()` on every attempt, looping with no exit. The rule had six hand-written homes and three of them were wrong, so it now has one. A new `packages/api/src/oauth/expiry.ts` normalizes `expires_in` to a positive finite number of seconds or nothing, and exposes the two shapes callers actually need: a cache TTL that falls back rather than returning `NaN`, and an absolute expiry that is absent rather than Invalid. The four unguarded sites adopt it, and the two ad-hoc guards in `openidStrategy.js` and `OboTokenService.js` are consolidated onto it. `createHandleOAuthToken` is folded in as well. Its guard already handled `null` and unparseable strings but admitted `NaN`, since `typeof NaN === 'number'` satisfied its first branch. The `mcp/oauth` sites are deliberately left alone: `tokens.ts` guards on truthiness and carries richer logic that reads a JWT access token's own expiry when the response omits one, and the file is being reworked in #13901. Closes #15318 Closes #15319 * 🕰️ fix: Address `expires_in` Guard Review Round 1 Preserve an explicitly elapsed lifetime instead of collapsing it into "unknown". `expires_in: 0` is the provider stating the credential is already dead, which is information; treating it as absent handed it the one-hour fallback in `createHandleOAuthToken` and dropped the expiry entirely in `ActionService`, so a credential declared expired could be used and retained for up to an hour. Both sites preserved that value before this branch, so the collapse was a regression introduced here. `normalizeExpiresIn` now returns any finite number, positive or not, and reports `undefined` only for a lifetime that is genuinely unusable. `getTokenExpiresAt` therefore yields a past timestamp for an elapsed lifetime, so callers refresh rather than guess. Cache TTLs cannot pass such a value through raw: Keyv reads a TTL of exactly `0` as "no expiry", turning a dead credential into the immortal entry this module exists to prevent. `getTokenCacheTtlMs` floors an elapsed lifetime at one millisecond, which expires immediately without ever writing an entry that outlives its credential. Parse numeric strings with `Number` rather than `parseInt`, which truncates a complete value such as `"3.6e3"` to `3` and would expire an hour-long credential after three seconds, re-exchanging against the identity provider on every request. An empty or blank string is rejected rather than read as zero, since `Number('')` is `0`. * 🕰️ fix: Bound `expires_in` to Lifetimes a Date Can Represent Parsing the complete numeric string last round made an overflow reachable that `parseInt` had been masking. `parseInt('1e13', 10)` was `1`; `Number('1e13')` is `1e13`, and `1e13` seconds is 1e16 ms, past the ECMAScript time value range of ±8.64e15. Every derived timestamp was therefore an Invalid Date whose `toISOString()` throws `RangeError: Invalid time value` — the exact failure this branch exists to remove, reintroduced by its own fix. The token model derives the same way at `packages/data-schemas/src/methods/token.ts:19`, so storage and authentication would fail with it. A lifetime is now reported as unusable unless it can still produce a valid `Date`. The bound is the time value range halved, leaving room for the `Date.now()` every derived timestamp adds. At roughly 137,000 years it rejects nothing a provider could mean: a one-year refresh token and even a hundred-year lifetime still pass through untouched, while `1e13`, `Number.MAX_SAFE_INTEGER` and `1e300` take the caller's fallback instead of poisoning a timestamp. The invariant tests now carry the overflow shapes rather than a fixed list of small ones, since a guard that only sees the inputs its author imagined is how the previous round's regression got in.	2026-08-28 10:47:09 -04:00
Danny Avila	04a7577821	🧵 fix: Retry Shared Links After Message Persistence Gaps (#15306) ... * fix: Retry shared links after message persistence gaps * chore: Sort share test imports * fix: Address share review feedback * fix: Normalize hydrated share target	2026-08-28 08:05:52 -04:00
Danny Avila	c06b09c945	🤖 feat: make Event Actor HITL durable (#15305) ... * 🤖 feat: make event actor HITL durable * 🤖 fix: break event actor outcome cycle * 🤖 fix: close durable actor terminal races * fix: harden durable event actor recovery proofs * fix: close event actor resume publication races	2026-08-28 08:05:11 -04:00
Danny Avila	6fba51a35a	🫆 feat: Fingerprint Agent Context for Zero-Read Warm Continuation (#15299) ... * feat: add context-fingerprinted warm continuation * fix: preserve manual skills across warm turns * perf: keep ordinary agent initialization neutral * fix: fingerprint complete agent topology * fix: close warm context replay gaps * fix: preserve lazy subagent memory scope * perf: fingerprint only model-bound memory * fix: project lazy memory capability * style: sort warm continuation imports * fix: make warm continuation reconstructable * fix: preserve lazy skill isolation * fix: retain scoped skill semantics	2026-08-27 23:01:14 -04:00
Danny Avila	a5fd00cd7a	🎼 feat: Unify Agent Turn Execution Behind One Immutable Plan (#15296) ... * feat: unify agent turn execution planning * fix: preserve automatic subagent activity handles * fix: gate subagent wakeups for ephemeral parents * fix: scope subagent wakeup guidance by agent	2026-08-27 22:26:46 -04:00
Danny Avila	f4efef4b3a	♟️ fix: Settle First Bound Actor Events (#15300)	2026-08-27 19:06:41 -04:00
Ravi Kumar L	a70bcf66a4	🪢 feat: show Langfuse session link in shared chats (#15273) ... * feat: show Langfuse sessions in shared chats * fix: wait for optional auth before loading shares * fix: require Langfuse access for shared session links * refactor: move shared Langfuse policy into API package * fix: wrap shared chat actions on mobile * fix: coordinate shared Langfuse link loading * perf: batch shared Langfuse capability checks	2026-08-27 14:29:57 -04:00
Danny Avila	03fa98eb37	🕸️ feat: Complete Keenable Web Search Provider (#15288) ... * feat: wire Keenable web-search provider into config, schema, and UI Keenable landed as a search provider in @librechat/agents (#285, shipped in 3.2.58+), but LibreChat did not yet expose it. This adds the config/schema/UI glue so it can be selected, mirroring the existing Tavily provider. - data-provider: add `keenable` to SearchProvider type + SearchProviders enum, keenableApiKey/keenableApiUrl schema fields, and a keenableSearchOptions block (maxResults, site, attributionTitle, timeout). - data-schemas: register keenable in webSearchAuth.providers and default the key/URL placeholders in loadWebSearchConfig. - api/web: pass keenableSearchOptions through to the provider and handle Keenable's keyless model. Unlike other providers it authenticates with no key (the public endpoint), picking up an optional key/URL when set; the URL override is SSRF-preflighted like other user-provided URLs. - client: add Keenable to the provider dropdown with an optional API-key input. - docs: document KEENABLE_API_KEY/KEENABLE_API_URL in .env.example and a webSearch example in librechat.example.yaml. - tests: keyless + keyed auth resolution, config defaults, and schema parsing. * fix: ESLint no-unused-vars and clarify Keenable yaml example - Remove the now-unused RerankerTypes import in data-schemas web.ts (the lint job runs with --max-warnings 0 on changed files, so this latent warning failed CI once the file was touched). - Note in the librechat.example.yaml Keenable stanza that a scraper (and reranker) is still required for web search to load, and include a Firecrawl scraper in the example. * chore: fix import order drift (sort-imports) * feat: add Keenable as a keyless scraper and select it without a pinned provider The Keenable scraper landed in @librechat/agents#337, so wire the scraper category the same way the search provider already is: `scraperProvider: keenable` reads pages through Keenable's public fetch endpoint with no key (a key only lifts rate limits, and the endpoint is overridden with KEENABLE_FETCH_URL). Paired with `rerankerType: none` this makes a fully keyless web-search stack possible for the first time. Also closes the Codex finding on this PR: because none of Keenable's auth fields are required, the generic auth loop skips it whenever it isn't pinned, so a key submitted through the API-key dialog (which cannot pin a provider) left the providers category unauthenticated. Keenable is now selected in that case, gated on one of its values actually being present so installs that configured nothing keep their current behavior. The scraper gets the same fallback, additionally gated on Keenable being the resolved search provider, so it never silently scrapes for another provider. * fix: select the Keenable scraper from a supplied key, not only for Keenable search The API-key dialog submits credentials and cannot pin a provider, so choosing Keenable as the scraper while search stays on Serper/SearXNG/Tavily had no effect: the unpinned-scraper fallback required Keenable to also be the resolved search provider. A supplied Keenable value now triggers it as well, which is the only signal the dialog can send. The fallback still runs only when no keyed scraper authenticated, and with neither trigger the category stays unauthenticated, so a deployment that never configured Keenable is unaffected. Note the fully keyless choice still cannot be expressed through the dialog: Keenable's key is optional, so picking it with no key submits nothing at all. librechat.example.yaml now documents pinning scraperProvider: keenable for that case. * style: Sort Keenable imports * fix: Harden Keenable auth resolution * fix: Preserve Keenable selection intent * fix: Fail closed on invalid web search auth * fix: close keenable auth gaps * style: sort web auth imports * fix: preserve web search selection integrity * fix: isolate web search auth ordering * fix: silence expected credential misses * chore: bump agents sdk * fix: preserve web search preference ownership * fix: forward cleared Keenable endpoint * style: sort web search hook imports * fix: require intent for credential clears --------- Co-authored-by: Ilya Bogin <ilya.bogin@keenable.ai>	2026-08-27 14:29:20 -04:00
Danny Avila	8ba76507ae	🧷 fix: Verify Scheduled HITL State Before Pausing (#15284) ... * fix: require durable storage for scheduled HITL * fix: verify scheduled HITL checkpoints before pause * fix: narrow scheduled HITL capability checks * fix: honor ask tool filters in schedule preflight * fix: enforce durability across scheduled HITL resumes * fix: scope scheduled HITL admission to active tools * refactor: move scheduled HITL admission to TypeScript * fix: close scheduled HITL admission gaps * fix: bind scheduled pauses to current checkpoint * fix: cap pending actions at binding deadline * fix: reject expired pending actions * fix: guard approval deadlines atomically	2026-08-27 11:16:24 -04:00
Danny Avila	de59da9636	🎟️ refactor: Require Credentials for Local Image Access by Default (#15252) ... * 🔐 fix: Protect Local Image Access by Default * 🔐 fix: Scope Image Authorization to Active Sessions * 🧹 style: Format Image Authorization Checks * 🛡️ fix: Harden Image Avatar Authorization * 🧭 style: Sort Image Authorization Imports * 🔐 fix: Close Image Authorization Review Gaps * 🧭 fix: Normalize Stored Avatar Base Paths * 🏢 fix: Resolve Tenant Assistant Image Policy * 🛂 fix: Enforce Effective Image Access Policy * 🧹 style: Flatten Assistant Config Selection * 🧷 fix: Preserve Image Access Compatibility * 🪪 fix: Make Image Sessions Revocable * 🏗️ fix: Move Image Session Policy Into API	2026-08-27 09:55:27 -04:00
Danny Avila	41ba808e6f	🧅 feat: Reveal Subagent Thread History and Event Detail on Demand (#15283) ... * ✨ feat: Refine Subagent Thread Activity UX * test: align subagent completion lifecycle * chore: satisfy subagent panel static checks * fix: harden subagent activity pagination * fix: preserve paged subagent history * fix: surface unrecoverable activity boundaries * fix: rebase paged activity after live advances * fix: bound child activity history state * fix: preserve rebased child history order * fix: retain unavailable child history boundary * fix: fence child history cursor generations * fix: preserve subagent history continuity * fix: retain live turns during history rebase * fix: close subagent history edge cases * fix: harden subagent history phase transitions	2026-08-27 08:56:06 -04:00
Danny Avila	8b1fcc0fc2	🌐 fix: Expose Gemini Models to Vertex AI Agents (#15234) ... * 🌐 fix: Expose Gemini Models to Vertex AI Agents * ♻️ refactor: Resolve Shared Vertex Model Catalogs * fix: Preserve Exact Vertex Model Catalogs * style: Format Agent Model Selection * test: Preserve Native FS in Stable Diffusion Spec	2026-08-27 06:50:23 -04:00
Danny Avila	44edcbe014	🧾 feat: Store Durable Event Actor Receipts (#15265) ... * 🧾 feat: Store durable event actor receipts * 📊 fix: Scope actor delivery metrics * 🧱 fix: Close durable receipt recovery gaps * fix: serialize actor action admission * fix: retire terminal batch members * fix: close actor terminal recovery races * fix: reclaim lanes after terminal receipts * fix: close actor receipt recovery gaps * fix: preserve ambiguous legacy handling * fix: recover legacy actor outcomes * fix: admit leased actor actions * fix: serialize actor admission recovery * fix: close actor recovery races * fix: token-fence actor admissions * fix: preserve mixed-version actor fences * fix: harden actor rollout and metrics * fix: gate durable actor receipt rollout * fix: enforce base actor receipt rollout * fix: align experimental actor rollout	2026-08-27 06:00:43 -04:00
Danny Avila	62a55213f0	🖇️ fix: Bind Model-Spec Authorization to the Loaded Agent (#15256)	2026-08-26 10:38:20 -04:00
Danny Avila	0c959deb99	🧢 fix: Cap Batched HITL Answers Before PII and Moderation (#15257)	2026-08-26 10:10:19 -04:00
Danny Avila	68fc46a055	🪢 feat: Resume Bound Event Actors from Checkpoint Forks (#15227) ... * feat: resume event actors from checkpoint forks * fix: fence event actor checkpoint uncertainty * fix: satisfy event actor type contracts * fix: make actor reconciliation recoverable * fix: preserve event actor lifecycle transitions * fix: fence event actor lifecycle outcomes * fix: enforce event actor lifecycle ownership * fix: retain event actor settlement proof * 🔒 fix: Retain Event Actor Receipts Through Repair and Bound Their Journal Repair and compensation deleted the reconciliation row they resolved, which was the only durable proof that the invocation had already applied an external action. A delayed duplicate owner could then reacquire the same invocation id and repeat that action. Both resolutions now retire their receipt to `settled` and record how it settled, so the same-id tombstone survives; `history_repaired` and `action_compensated` still force a cold rebuild. Compensation undoes the effect without re-authorizing the delivery, so a legitimate retry must arrive under a new invocation id. A retried repair converges on its own receipt. Bound the journal so a long-lived actor cannot grow its conversation document without limit: a new fence is admitted only when no active lifecycle row exists, so a capped push can evict nothing but the oldest settled receipts. Stop shipping the unbounded source payload on every bound-child continuation. It rode the delivery body regardless of the feature flag while the sibling `fire` body deliberately sends event identity alone, so a large webhook payload could push a previously working delivery past the chat route's body limit. The actor binds an invocation from identity and never builds the prompt from it. Blank the positional token map on warm continuations. It is derived from the full DB history, while a warm run executes on checkpoint-restored state, so its indices address different messages and the pruner never recounts them — misattributing cached counts to the wrong messages in both directions. Keep the replaced-claim exit on its cleanup path when preserving reconciliation fails: the committing CAS already left a blocking row, so the failed status upgrade costs provenance, not safety. * 🧪 test: Pin the Warm Continuation's Map/Summary Asymmetry Give the warm-continuation client test a populated token map and a real cross-run summary so its assertions bite: the positional map must arrive blank (checkpoint-restored state no longer matches DB-derived indices, and the pruner never recounts a populated entry) while `initialSummary` must pass through unchanged — it rides the system tail and summarizes pre-boundary turns that were excluded from the very history the committed checkpoint was built from, so blanking it would silently drop context no warm run can recover. * ⚖️ fix: Honor Compensation in Settlement and Age-Bound the Receipt Journal A compensated receipt still tombstones its invocation id, but its external effect was explicitly undone — the terminal handler nonetheless replayed every settled lifecycle's stored action as authoritative and settled the public outcome as applied, telling an action-aware source the operation stands and suppressing the new-invocation retry compensation requires. The handler now settles a compensated invocation as failed with an explicit compensation error, overriding even fresh applied run evidence from a replayed generation; verified and repaired receipts continue to replay applied. Receipt eviction is now primarily age-based: a stale same-id owner is bounded by time, not by how many newer invocations settle, so the previous count-only slice let a high-rate actor evict a tombstone while its delayed duplicate owner could still wake and repeat the action. Admission prunes only settled receipts older than a retention window that dwarfs every generation, job, and delivery-retry lifetime, and the count cap is demoted to a raised document-size backstop. * 🔀 fix: Serialize Compensation Against Settlement and Never Evict Live Receipts The terminal handler read its lifecycle snapshot, verified history, and then settled the public outcome — so a compensation resolving the same receipt during that window lost: the handler settled applied from its stale snapshot and no retry could ever change the replay-identity-locked outcome. The receipt's status CAS is now the serialization point: verification resolves the receipt BEFORE settling, whichever transition wins determines the public outcome, and a crash between resolve and settle converges through the retained receipt's replay. The verified-replay probe requires the receipt's own resolution, so a compensated receipt can never satisfy a verification retry. This inverts the settle-before-receipt ordering deliberately: that ordering guarded proof that resolution used to delete, and the receipt now retains its full action proof through resolution. The document-size cap is no longer an eviction quota. A receipt inside its retention window is never discarded: when the journal holds a full cap of unexpired receipts, new invocations are refused fail-closed until receipts age out, making duplicate protection and document integrity simultaneous invariants instead of a rate-dependent trade. * 🎓 fix: Keep Skill-Bearing Event Actors on the Legacy Path Skill primes are spliced into the message list directly ahead of the newest message, and a warm continuation forwards only that newest message — so a checkpoint-restored actor would keep serving the prime bodies baked in at its last cold start and never observe an edited or newly attached skill. Until the actor head carries a context fingerprint that forces a cold rebuild when the agent's skill context changes, agents with always-apply or manual skill primes stay on the legacy path, which re-primes fresh bodies every turn: correct on every event, just never warm. * 📜 fix: Gate Fork Mode on the Skills Capability, Not Just Request-Time Primes History-derived re-priming was a third path into the same staleness class: an actor that previously invoked a skill carries no request-time prime arrays, yet primeInvokedSkills re-resolves that skill's current body from history each turn and the warm slice drops the reconstruction — leaving the checkpoint's old body active after edits. The fork gate now keys on the priming hook itself (present exactly when the skills capability is enabled) alongside the request-time arrays, so every skill-body path routes to the legacy rebuild until #15235's context fingerprint restores warm continuation for skill-bearing actors. * 🧾 fix: Capture Applied-Action Proof at Tool Execution, Not After sendMessage The executor read applied-action evidence from the run-step collection the instant sendMessage resolved, but that collection is populated asynchronously — an applied invocation could classify as actionless (runSteps still empty while the tool result already streamed), discarding its fork and stranding the actor cold while the terminal handler later settled the same delivery as applied from the persisted evidence. Authoritative proof is now recorded in graph context the moment the expected tool executes: the request-owned recorder observes the tool-end chain (which ToolNode dispatches synchronously with both input and output) and applies the same fences as run-step evidence — exact tool name with the MCP-suffixed form, the declared argument subset against the execution input, an error-free result, and the background non-execution receipt exclusion. readAppliedAction consults the receipt first; run-step inspection remains the fallback for paths that bypass the tool-end chain. Regression coverage reproduces the observed ordering: the real executor commits the head from the receipt while run steps are empty, warm-continues the next event, and never re-executes the action; recorder fences and the receipt-first controller wiring are covered separately. * 🎯 fix: Supply Execution Arguments to the Tool End Callback The live Vertex + MCP canary exposed a contract mismatch the synthetic fixtures hid: the ON_TOOL_EXECUTE execution path invoked its tool end callback with output only, while the action recorder must verify the declared argument subset against the execution input. The receipt never qualified, every turn fell back to cold history rebuilds, and the tournament advanced with zero actor heads and zero retained checkpoints while looking successful. The execution handler owns both halves at the same moment, so the fix is at the source rather than a correlation store: ToolEndCallbackData gains the executed call's input and every callback site passes tc.args. A handler-level regression drives the real createToolExecuteHandler and asserts the callback receives both fields; recorder regressions pin the production shapes — an output-only tool end must starve an argument-fenced receipt rather than trust an unfenced match, and still qualifies a name-only expected action. * 🕵️ fix: Mark Background Deliveries So They Cannot Impersonate Applied Actions The background-claim callback reports the ORIGINAL tool's name for artifact attribution on the poll turn that harvests a completed task. A name-only expected action could therefore be impersonated by work some earlier turn dispatched: the recorder would attribute that delivery to the current invocation and commit a head whose state never contained the invocation's own action. The run-step evidence path never had this hole — it sees the poll tool's name — so the recorder must match its provenance discipline. Delivery callbacks now carry an explicit backgroundDelivery marker set at the one site that rewrites the name, and the recorder ignores marked deliveries outright. Regressions pin both halves of the contract: the delivery callback must carry the marker with the poll call's arguments, and a marked delivery can never qualify even a name-only expected action. * 🧿 fix: Version Invalidations, Keep Evidence Ahead of Output Policy, Gate Detachable Actions Three closeout-round findings, each converted into an invariant. Every legacy-path invalidation now advances a durable epoch — including for headless and already cold-marked actors, where the marker alone leaves no CAS-visible trace — and the actor-head CAS requires the epoch observed at preparation. A concurrently prepared fork whose history predates an intervening legacy turn can no longer commit past it; the commit reports an ordinary conflict and journals. Execution identity is now emitted before post-execution output policy: when a side-effecting tool succeeds but its returned content is withheld by the output filter, the callback delivers an outputFiltered receipt with blank content — the recorder accepts it as proof (rejecting model-detached calls it cannot distinguish through the blank shape), the artifact path never sees it, and an applied action is no longer reclassified actionless and re-executed on retry. Background-capable expected actions stay off the fork path: dispatch returns a launch handle every evidence fence correctly rejects, and the completion is provenance-marked as another turn's work, so a fork would settle actionless before the external effect lands with nothing to stop a retry from dispatching it again. The gate mirrors the MCP-suffix name matching of the evidence path. * 🚧 fix: Seal the Whole Legacy Turn Behind a Second Epoch Advance The epoch fenced only the legacy turn's start: a fork preparing after the begin invalidation but before the turn's message persistence observed the new epoch and cold marker, rebuilt from history that did not yet contain the turn, and committed cleanly because nothing advanced the epoch again — making the incomplete rebuild authoritative and clearing the marker. Every legacy event turn now seals its invalidation at terminal persistence with a second epoch advance, on the success, replaced-claim, and error exits alike. Sealing deliberately carries no quiescence requirement — it must succeed while a fork fence is active, because that is exactly the mid-turn race it defeats — and a fork that already committed against the begin epoch is healed the same way: the seal re-marks the head cold, so the next event rebuilds with complete history. Seal failure never diverts the turn's own exit; the begin bump still fences everything prepared before the turn. * 🔗 fix: Replace the Best-Effort Epoch Bump With a Durable Legacy-Turn Fence The second epoch advance could not make a legacy turn atomic, and three findings shared that root cause: two conditional updates left a headless gap a fork could create the head inside; the error exit sealed before saveErrorTurn made the error history durable; and any crash or failure between persistence and sealing left an incomplete fork authoritative, because the seal was best-effort and its failure was swallowed. A legacy turn now carries one durable fence. A token is written before execution by a single update-pipeline write — no two-write gap, and the cold marker is applied only where a head exists via $cond/$$REMOVE. While the token is present no fork may prepare (the adapter refuses) or commit (the CAS requires its absence), because the turn's messages are not yet durable. One atomic write clears the exact token and advances the epoch once history is persisted — after saveErrorTurn on the error exit — and success, replacement, and error exits all route through it. Failure is now fail-closed rather than silent: a failed seal leaves the token set, which keeps blocking forks and is logged as such, and a fence abandoned by a crashed turn is reclaimed only once stale, advancing the epoch and marking any head cold so the next event rebuilds from whatever history actually survived. * fix: serialize legacy event actor turns * fix: close legacy actor fence ownership gaps * fix: preserve legacy actor persistence fences	2026-08-26 08:39:00 -04:00
Danny Avila	f34a49007d	🔣 fix: Escape SPA Language Attribute (#15248)	2026-08-26 07:37:36 -04:00
Marco Beretta	0383030817	⚡ perf: Avoid Parallel Full and Paged Prompt Loading on Startup (#15031) ... Co-authored-by: Danny Avila <danny@librechat.ai>	2026-08-25 21:21:17 -04:00