572 commits

Author	SHA1	Message	Date
Danny Avila	773127bff2	🎠 refactor: Route Every Event Actor Turn Through One Lifecycle (#15325) ... * refactor: unify Event Actor turn lifecycle * fix: retain Event Actor fence ownership * fix: preserve mixed-version actor suspension safety	2026-08-28 17:30:23 -04:00
Danny Avila	3fa33b740b	🛫 refactor: Promote Generation Protocol V2 Automatically (#15324) ... * refactor: promote generation protocol v2 automatically * fix: remove unused protocol import	2026-08-28 17:17:09 -04:00
Danny Avila	77c2a51cf3	🪧 fix: Advertise Detached Event Actor Support from the Generation Store (#15322) ... * fix: activate detached event actions automatically * fix: shield detached terminal generations * perf: share capability availability index * style: flatten capability status selection * fix: preserve mixed-version lifecycle shells * fix: complete detached action store adapters * fix: close mixed-version capability races * fix: honor legacy capability success	2026-08-28 16:33:23 -04:00
Danny Avila	487984193a	🪂 feat: Make Event Actor Detached Actions Durable (#15307) ... * feat: make Event Actor detached actions durable * fix: align detached actor checks with merged base * fix: close detached actor lifecycle gaps * fix: break detached action outcome cycle * fix: deepen detached action ownership * fix: fence detached launch handoffs * fix: preserve detached rollout fencing * test: type detached recovery failure * fix: close detached ownership handoffs * fix: stage detached action activation * fix: persist detached terminal retries * fix: fence detached durability ownership * fix: fence trigger lane publication * fix: fence detached transition generations * fix: retire failed detached predecessors * fix: close detached rollout consumers * test: type detached retry turns * fix: close detached ownership gaps * fix: fence detached action resumes * fix: fence prior-head event actor resumes * fix: preserve legacy event actor resumes * fix: align detached resume boundaries	2026-08-28 14:21:04 -04:00
Danny Avila	c06b09c945	🤖 feat: make Event Actor HITL durable (#15305) ... * 🤖 feat: make event actor HITL durable * 🤖 fix: break event actor outcome cycle * 🤖 fix: close durable actor terminal races * fix: harden durable event actor recovery proofs * fix: close event actor resume publication races	2026-08-28 08:05:11 -04:00
Danny Avila	6fba51a35a	🫆 feat: Fingerprint Agent Context for Zero-Read Warm Continuation (#15299) ... * feat: add context-fingerprinted warm continuation * fix: preserve manual skills across warm turns * perf: keep ordinary agent initialization neutral * fix: fingerprint complete agent topology * fix: close warm context replay gaps * fix: preserve lazy subagent memory scope * perf: fingerprint only model-bound memory * fix: project lazy memory capability * style: sort warm continuation imports * fix: make warm continuation reconstructable * fix: preserve lazy skill isolation * fix: retain scoped skill semantics	2026-08-27 23:01:14 -04:00
Danny Avila	a5fd00cd7a	🎼 feat: Unify Agent Turn Execution Behind One Immutable Plan (#15296) ... * feat: unify agent turn execution planning * fix: preserve automatic subagent activity handles * fix: gate subagent wakeups for ephemeral parents * fix: scope subagent wakeup guidance by agent	2026-08-27 22:26:46 -04:00
Danny Avila	f4efef4b3a	♟️ fix: Settle First Bound Actor Events (#15300)	2026-08-27 19:06:41 -04:00
Danny Avila	03fa98eb37	🕸️ feat: Complete Keenable Web Search Provider (#15288) ... * feat: wire Keenable web-search provider into config, schema, and UI Keenable landed as a search provider in @librechat/agents (#285, shipped in 3.2.58+), but LibreChat did not yet expose it. This adds the config/schema/UI glue so it can be selected, mirroring the existing Tavily provider. - data-provider: add `keenable` to SearchProvider type + SearchProviders enum, keenableApiKey/keenableApiUrl schema fields, and a keenableSearchOptions block (maxResults, site, attributionTitle, timeout). - data-schemas: register keenable in webSearchAuth.providers and default the key/URL placeholders in loadWebSearchConfig. - api/web: pass keenableSearchOptions through to the provider and handle Keenable's keyless model. Unlike other providers it authenticates with no key (the public endpoint), picking up an optional key/URL when set; the URL override is SSRF-preflighted like other user-provided URLs. - client: add Keenable to the provider dropdown with an optional API-key input. - docs: document KEENABLE_API_KEY/KEENABLE_API_URL in .env.example and a webSearch example in librechat.example.yaml. - tests: keyless + keyed auth resolution, config defaults, and schema parsing. * fix: ESLint no-unused-vars and clarify Keenable yaml example - Remove the now-unused RerankerTypes import in data-schemas web.ts (the lint job runs with --max-warnings 0 on changed files, so this latent warning failed CI once the file was touched). - Note in the librechat.example.yaml Keenable stanza that a scraper (and reranker) is still required for web search to load, and include a Firecrawl scraper in the example. * chore: fix import order drift (sort-imports) * feat: add Keenable as a keyless scraper and select it without a pinned provider The Keenable scraper landed in @librechat/agents#337, so wire the scraper category the same way the search provider already is: `scraperProvider: keenable` reads pages through Keenable's public fetch endpoint with no key (a key only lifts rate limits, and the endpoint is overridden with KEENABLE_FETCH_URL). Paired with `rerankerType: none` this makes a fully keyless web-search stack possible for the first time. Also closes the Codex finding on this PR: because none of Keenable's auth fields are required, the generic auth loop skips it whenever it isn't pinned, so a key submitted through the API-key dialog (which cannot pin a provider) left the providers category unauthenticated. Keenable is now selected in that case, gated on one of its values actually being present so installs that configured nothing keep their current behavior. The scraper gets the same fallback, additionally gated on Keenable being the resolved search provider, so it never silently scrapes for another provider. * fix: select the Keenable scraper from a supplied key, not only for Keenable search The API-key dialog submits credentials and cannot pin a provider, so choosing Keenable as the scraper while search stays on Serper/SearXNG/Tavily had no effect: the unpinned-scraper fallback required Keenable to also be the resolved search provider. A supplied Keenable value now triggers it as well, which is the only signal the dialog can send. The fallback still runs only when no keyed scraper authenticated, and with neither trigger the category stays unauthenticated, so a deployment that never configured Keenable is unaffected. Note the fully keyless choice still cannot be expressed through the dialog: Keenable's key is optional, so picking it with no key submits nothing at all. librechat.example.yaml now documents pinning scraperProvider: keenable for that case. * style: Sort Keenable imports * fix: Harden Keenable auth resolution * fix: Preserve Keenable selection intent * fix: Fail closed on invalid web search auth * fix: close keenable auth gaps * style: sort web auth imports * fix: preserve web search selection integrity * fix: isolate web search auth ordering * fix: silence expected credential misses * chore: bump agents sdk * fix: preserve web search preference ownership * fix: forward cleared Keenable endpoint * style: sort web search hook imports * fix: require intent for credential clears --------- Co-authored-by: Ilya Bogin <ilya.bogin@keenable.ai>	2026-08-27 14:29:20 -04:00
Danny Avila	8ba76507ae	🧷 fix: Verify Scheduled HITL State Before Pausing (#15284) ... * fix: require durable storage for scheduled HITL * fix: verify scheduled HITL checkpoints before pause * fix: narrow scheduled HITL capability checks * fix: honor ask tool filters in schedule preflight * fix: enforce durability across scheduled HITL resumes * fix: scope scheduled HITL admission to active tools * refactor: move scheduled HITL admission to TypeScript * fix: close scheduled HITL admission gaps * fix: bind scheduled pauses to current checkpoint * fix: cap pending actions at binding deadline * fix: reject expired pending actions * fix: guard approval deadlines atomically	2026-08-27 11:16:24 -04:00
Danny Avila	de59da9636	🎟️ refactor: Require Credentials for Local Image Access by Default (#15252) ... * 🔐 fix: Protect Local Image Access by Default * 🔐 fix: Scope Image Authorization to Active Sessions * 🧹 style: Format Image Authorization Checks * 🛡️ fix: Harden Image Avatar Authorization * 🧭 style: Sort Image Authorization Imports * 🔐 fix: Close Image Authorization Review Gaps * 🧭 fix: Normalize Stored Avatar Base Paths * 🏢 fix: Resolve Tenant Assistant Image Policy * 🛂 fix: Enforce Effective Image Access Policy * 🧹 style: Flatten Assistant Config Selection * 🧷 fix: Preserve Image Access Compatibility * 🪪 fix: Make Image Sessions Revocable * 🏗️ fix: Move Image Session Policy Into API	2026-08-27 09:55:27 -04:00
Danny Avila	41ba808e6f	🧅 feat: Reveal Subagent Thread History and Event Detail on Demand (#15283) ... * ✨ feat: Refine Subagent Thread Activity UX * test: align subagent completion lifecycle * chore: satisfy subagent panel static checks * fix: harden subagent activity pagination * fix: preserve paged subagent history * fix: surface unrecoverable activity boundaries * fix: rebase paged activity after live advances * fix: bound child activity history state * fix: preserve rebased child history order * fix: retain unavailable child history boundary * fix: fence child history cursor generations * fix: preserve subagent history continuity * fix: retain live turns during history rebase * fix: close subagent history edge cases * fix: harden subagent history phase transitions	2026-08-27 08:56:06 -04:00
Danny Avila	44edcbe014	🧾 feat: Store Durable Event Actor Receipts (#15265) ... * 🧾 feat: Store durable event actor receipts * 📊 fix: Scope actor delivery metrics * 🧱 fix: Close durable receipt recovery gaps * fix: serialize actor action admission * fix: retire terminal batch members * fix: close actor terminal recovery races * fix: reclaim lanes after terminal receipts * fix: close actor receipt recovery gaps * fix: preserve ambiguous legacy handling * fix: recover legacy actor outcomes * fix: admit leased actor actions * fix: serialize actor admission recovery * fix: close actor recovery races * fix: token-fence actor admissions * fix: preserve mixed-version actor fences * fix: harden actor rollout and metrics * fix: gate durable actor receipt rollout * fix: enforce base actor receipt rollout * fix: align experimental actor rollout	2026-08-27 06:00:43 -04:00
Danny Avila	68fc46a055	🪢 feat: Resume Bound Event Actors from Checkpoint Forks (#15227) ... * feat: resume event actors from checkpoint forks * fix: fence event actor checkpoint uncertainty * fix: satisfy event actor type contracts * fix: make actor reconciliation recoverable * fix: preserve event actor lifecycle transitions * fix: fence event actor lifecycle outcomes * fix: enforce event actor lifecycle ownership * fix: retain event actor settlement proof * 🔒 fix: Retain Event Actor Receipts Through Repair and Bound Their Journal Repair and compensation deleted the reconciliation row they resolved, which was the only durable proof that the invocation had already applied an external action. A delayed duplicate owner could then reacquire the same invocation id and repeat that action. Both resolutions now retire their receipt to `settled` and record how it settled, so the same-id tombstone survives; `history_repaired` and `action_compensated` still force a cold rebuild. Compensation undoes the effect without re-authorizing the delivery, so a legitimate retry must arrive under a new invocation id. A retried repair converges on its own receipt. Bound the journal so a long-lived actor cannot grow its conversation document without limit: a new fence is admitted only when no active lifecycle row exists, so a capped push can evict nothing but the oldest settled receipts. Stop shipping the unbounded source payload on every bound-child continuation. It rode the delivery body regardless of the feature flag while the sibling `fire` body deliberately sends event identity alone, so a large webhook payload could push a previously working delivery past the chat route's body limit. The actor binds an invocation from identity and never builds the prompt from it. Blank the positional token map on warm continuations. It is derived from the full DB history, while a warm run executes on checkpoint-restored state, so its indices address different messages and the pruner never recounts them — misattributing cached counts to the wrong messages in both directions. Keep the replaced-claim exit on its cleanup path when preserving reconciliation fails: the committing CAS already left a blocking row, so the failed status upgrade costs provenance, not safety. * 🧪 test: Pin the Warm Continuation's Map/Summary Asymmetry Give the warm-continuation client test a populated token map and a real cross-run summary so its assertions bite: the positional map must arrive blank (checkpoint-restored state no longer matches DB-derived indices, and the pruner never recounts a populated entry) while `initialSummary` must pass through unchanged — it rides the system tail and summarizes pre-boundary turns that were excluded from the very history the committed checkpoint was built from, so blanking it would silently drop context no warm run can recover. * ⚖️ fix: Honor Compensation in Settlement and Age-Bound the Receipt Journal A compensated receipt still tombstones its invocation id, but its external effect was explicitly undone — the terminal handler nonetheless replayed every settled lifecycle's stored action as authoritative and settled the public outcome as applied, telling an action-aware source the operation stands and suppressing the new-invocation retry compensation requires. The handler now settles a compensated invocation as failed with an explicit compensation error, overriding even fresh applied run evidence from a replayed generation; verified and repaired receipts continue to replay applied. Receipt eviction is now primarily age-based: a stale same-id owner is bounded by time, not by how many newer invocations settle, so the previous count-only slice let a high-rate actor evict a tombstone while its delayed duplicate owner could still wake and repeat the action. Admission prunes only settled receipts older than a retention window that dwarfs every generation, job, and delivery-retry lifetime, and the count cap is demoted to a raised document-size backstop. * 🔀 fix: Serialize Compensation Against Settlement and Never Evict Live Receipts The terminal handler read its lifecycle snapshot, verified history, and then settled the public outcome — so a compensation resolving the same receipt during that window lost: the handler settled applied from its stale snapshot and no retry could ever change the replay-identity-locked outcome. The receipt's status CAS is now the serialization point: verification resolves the receipt BEFORE settling, whichever transition wins determines the public outcome, and a crash between resolve and settle converges through the retained receipt's replay. The verified-replay probe requires the receipt's own resolution, so a compensated receipt can never satisfy a verification retry. This inverts the settle-before-receipt ordering deliberately: that ordering guarded proof that resolution used to delete, and the receipt now retains its full action proof through resolution. The document-size cap is no longer an eviction quota. A receipt inside its retention window is never discarded: when the journal holds a full cap of unexpired receipts, new invocations are refused fail-closed until receipts age out, making duplicate protection and document integrity simultaneous invariants instead of a rate-dependent trade. * 🎓 fix: Keep Skill-Bearing Event Actors on the Legacy Path Skill primes are spliced into the message list directly ahead of the newest message, and a warm continuation forwards only that newest message — so a checkpoint-restored actor would keep serving the prime bodies baked in at its last cold start and never observe an edited or newly attached skill. Until the actor head carries a context fingerprint that forces a cold rebuild when the agent's skill context changes, agents with always-apply or manual skill primes stay on the legacy path, which re-primes fresh bodies every turn: correct on every event, just never warm. * 📜 fix: Gate Fork Mode on the Skills Capability, Not Just Request-Time Primes History-derived re-priming was a third path into the same staleness class: an actor that previously invoked a skill carries no request-time prime arrays, yet primeInvokedSkills re-resolves that skill's current body from history each turn and the warm slice drops the reconstruction — leaving the checkpoint's old body active after edits. The fork gate now keys on the priming hook itself (present exactly when the skills capability is enabled) alongside the request-time arrays, so every skill-body path routes to the legacy rebuild until #15235's context fingerprint restores warm continuation for skill-bearing actors. * 🧾 fix: Capture Applied-Action Proof at Tool Execution, Not After sendMessage The executor read applied-action evidence from the run-step collection the instant sendMessage resolved, but that collection is populated asynchronously — an applied invocation could classify as actionless (runSteps still empty while the tool result already streamed), discarding its fork and stranding the actor cold while the terminal handler later settled the same delivery as applied from the persisted evidence. Authoritative proof is now recorded in graph context the moment the expected tool executes: the request-owned recorder observes the tool-end chain (which ToolNode dispatches synchronously with both input and output) and applies the same fences as run-step evidence — exact tool name with the MCP-suffixed form, the declared argument subset against the execution input, an error-free result, and the background non-execution receipt exclusion. readAppliedAction consults the receipt first; run-step inspection remains the fallback for paths that bypass the tool-end chain. Regression coverage reproduces the observed ordering: the real executor commits the head from the receipt while run steps are empty, warm-continues the next event, and never re-executes the action; recorder fences and the receipt-first controller wiring are covered separately. * 🎯 fix: Supply Execution Arguments to the Tool End Callback The live Vertex + MCP canary exposed a contract mismatch the synthetic fixtures hid: the ON_TOOL_EXECUTE execution path invoked its tool end callback with output only, while the action recorder must verify the declared argument subset against the execution input. The receipt never qualified, every turn fell back to cold history rebuilds, and the tournament advanced with zero actor heads and zero retained checkpoints while looking successful. The execution handler owns both halves at the same moment, so the fix is at the source rather than a correlation store: ToolEndCallbackData gains the executed call's input and every callback site passes tc.args. A handler-level regression drives the real createToolExecuteHandler and asserts the callback receives both fields; recorder regressions pin the production shapes — an output-only tool end must starve an argument-fenced receipt rather than trust an unfenced match, and still qualifies a name-only expected action. * 🕵️ fix: Mark Background Deliveries So They Cannot Impersonate Applied Actions The background-claim callback reports the ORIGINAL tool's name for artifact attribution on the poll turn that harvests a completed task. A name-only expected action could therefore be impersonated by work some earlier turn dispatched: the recorder would attribute that delivery to the current invocation and commit a head whose state never contained the invocation's own action. The run-step evidence path never had this hole — it sees the poll tool's name — so the recorder must match its provenance discipline. Delivery callbacks now carry an explicit backgroundDelivery marker set at the one site that rewrites the name, and the recorder ignores marked deliveries outright. Regressions pin both halves of the contract: the delivery callback must carry the marker with the poll call's arguments, and a marked delivery can never qualify even a name-only expected action. * 🧿 fix: Version Invalidations, Keep Evidence Ahead of Output Policy, Gate Detachable Actions Three closeout-round findings, each converted into an invariant. Every legacy-path invalidation now advances a durable epoch — including for headless and already cold-marked actors, where the marker alone leaves no CAS-visible trace — and the actor-head CAS requires the epoch observed at preparation. A concurrently prepared fork whose history predates an intervening legacy turn can no longer commit past it; the commit reports an ordinary conflict and journals. Execution identity is now emitted before post-execution output policy: when a side-effecting tool succeeds but its returned content is withheld by the output filter, the callback delivers an outputFiltered receipt with blank content — the recorder accepts it as proof (rejecting model-detached calls it cannot distinguish through the blank shape), the artifact path never sees it, and an applied action is no longer reclassified actionless and re-executed on retry. Background-capable expected actions stay off the fork path: dispatch returns a launch handle every evidence fence correctly rejects, and the completion is provenance-marked as another turn's work, so a fork would settle actionless before the external effect lands with nothing to stop a retry from dispatching it again. The gate mirrors the MCP-suffix name matching of the evidence path. * 🚧 fix: Seal the Whole Legacy Turn Behind a Second Epoch Advance The epoch fenced only the legacy turn's start: a fork preparing after the begin invalidation but before the turn's message persistence observed the new epoch and cold marker, rebuilt from history that did not yet contain the turn, and committed cleanly because nothing advanced the epoch again — making the incomplete rebuild authoritative and clearing the marker. Every legacy event turn now seals its invalidation at terminal persistence with a second epoch advance, on the success, replaced-claim, and error exits alike. Sealing deliberately carries no quiescence requirement — it must succeed while a fork fence is active, because that is exactly the mid-turn race it defeats — and a fork that already committed against the begin epoch is healed the same way: the seal re-marks the head cold, so the next event rebuilds with complete history. Seal failure never diverts the turn's own exit; the begin bump still fences everything prepared before the turn. * 🔗 fix: Replace the Best-Effort Epoch Bump With a Durable Legacy-Turn Fence The second epoch advance could not make a legacy turn atomic, and three findings shared that root cause: two conditional updates left a headless gap a fork could create the head inside; the error exit sealed before saveErrorTurn made the error history durable; and any crash or failure between persistence and sealing left an incomplete fork authoritative, because the seal was best-effort and its failure was swallowed. A legacy turn now carries one durable fence. A token is written before execution by a single update-pipeline write — no two-write gap, and the cold marker is applied only where a head exists via $cond/$$REMOVE. While the token is present no fork may prepare (the adapter refuses) or commit (the CAS requires its absence), because the turn's messages are not yet durable. One atomic write clears the exact token and advances the epoch once history is persisted — after saveErrorTurn on the error exit — and success, replacement, and error exits all route through it. Failure is now fail-closed rather than silent: a failed seal leaves the token set, which keeps blocking forks and is logged as such, and a fence abandoned by a crashed turn is reclaimed only once stale, advancing the epoch and marking any head cold so the next event rebuilds from whatever history actually survived. * fix: serialize legacy event actor turns * fix: close legacy actor fence ownership gaps * fix: preserve legacy actor persistence fences	2026-08-26 08:39:00 -04:00
Danny Avila	1489623fa3	🎬 test: Record-Once/Replay-Forever Model Fixtures for Mock E2E (#15210) ... * 🎬 test: Record-Once/Replay-Forever Model Fixtures for Mock E2E The mock e2e lane's only credential-free model is a hand-authored script: `fake-model.js` decides responses from ~60 `E2E_*` prompt markers. That covers scripted shapes well, but no scenario replays a *real* recorded provider conversation through the assembled chain, so real streaming shapes — provider chunk cadence, reasoning deltas, usage metadata — are only ever approximated. This adds a record-once/replay-forever tier alongside the marker routing. Record (`E2E_MODEL_FIXTURES=record`, needs a provider key): the run hook appends a LangChain callback handler to every agent context's `clientOptions.callbacks` instead of overriding the model, so the REAL provider streams while each invocation's `AIMessageChunk`s serialize to `e2e/fixtures/model-replay/<name>.jsonl` — text deltas, tool_call_chunks, reasoning kwargs, and genuine usage metadata. Only the latest human text is recorded for binding; system prompts and tool schemas never enter the fixture. Replay (default, keyless): `fake-model.js` consults `tryBindReplay` ahead of marker routing, binding a conversation whose prompt matches the next unconsumed invocation. The replaying model is not hand-assigned — it is registered as SDK provider `librechat-e2e-replay` via `registerProvider` and constructed through the SDK's own `initializeModel`, so registry lookup, constructor clientOptions, and real `bindTools` all run the way a live provider's would. Recorded chunks therefore stream through the same createRun → graph → SSE → persistence chain. Consumption is enforced rather than assumed: every invocation re-checks its prompt against the recording, an invocation past the end of the script throws, and a per-fixture ledger lets the spec assert at teardown that every recorded invocation and chunk was drained. Streaming incrementality is asserted from that ledger, not by sampling transient DOM, which is a race by construction. The credential-free profile is unchanged when not recording: the record provider and its selector entry are template markers that stay comments, and no existing spec's routing is touched (a fixture only binds on an exact prompt match; everything else falls through). Verified: record vs the real DeepSeek API 1 passed (15.2s); keyless replay 1 passed twice (11.4s, 12.7s) with the ledger fully drained (2/2 invocations, 10/10 chunks, no overruns or mismatches); app-load, completion, and chat 10 passed unchanged. * fix(e2e): rebind a replay fixture from the top for a new conversation The replay cursor is process-global while the web server outlives a Playwright retry, so a fully consumed fixture left the retry unable to bind its first prompt: it fell through to marker routing and failed deterministically, burning every configured CI retry. A partially consumed attempt failed the same way. Binding now restarts the fixture when the incoming prompt matches its first recorded invocation, resetting the ledger with the cursor so the new attempt is judged on its own consumption instead of accumulating the previous one's counts. Continuing an in-progress binding still outranks restarting, so a fixture whose opening prompt repeats later in the script advances rather than rewinding. The over-consumption guard is untouched — it fires inside the stream when the cursor passes the end, not at bind time. * fix(e2e): close three replay-lane gaps found in review Restart the recorder on a retry. Its state is process-global like the replay cursor, so a failed attempt that had already recorded invocations left the counter advanced: the retry appended 2/3 after 0/1, or kept the previous attempt's `error` line, and the fixture was unusable for replay. Recording now truncates and restarts when the opening prompt reappears, mirroring the replay side's rule and its caveat. Retain a consumed binding for the conversation that drove it. An extra user turn past the final recorded invocation found no next invocation and fell through to ordinary fake-model routing, so it was answered with a mock reply: the over-consumption guard never ran and the already-drained ledger still passed. Such a conversation is now recognized by its human turns opening with the fixture's recorded prompts, and stays bound so the stream raises the overrun. Continuing an in-progress binding still outranks restarting, which outranks retaining a consumed one, so a retry's fresh conversation rewinds rather than being read as an extra turn. Validate the fixture the recording actually wrote. Record mode honors `E2E_MODEL_FIXTURE_NAME`, but the spec always inspected the committed `deepseek-two-turn`; another name wrote elsewhere while the assertions read the pre-existing file, and because the prompts are fixed the stale answers could match and green a run that verified nothing it produced. * fix(e2e): make replay binding correct for tool and subagent fixtures Round two's retry and consumed-binding fixes both assumed one model invocation per user turn. A turn that calls a tool breaks that: the model is invoked again after the tool result under the same latest human message. Identify a retry by the conversation boundary, not the prompt. The recorder ran per invocation and truncated whenever the opening prompt reappeared, so a tool round trip looked like a retry and discarded the recorded tool-call invocation. Restart detection now sits in `installRecorder`, which runs once per `createRun`: a turn whose history holds no prior human message begins a conversation. Compare consumed bindings against user turns, not invocations. Several recorded invocations can share one prompt, so a one-to-one comparison could not recognize the originating conversation — invocations `[A, A, B]` against history `[A, B, C]` failed on both length and elements, and the extra turn fell through to the fake model with the drained ledger still passing. Fixtures now carry their collapsed turn sequence. Override the subagent model too. `graph.overrideModel` is not inherited by child executors, so a fixture recording a subagent call — record mode captures child invocations already — would leave the child on its configured provider: an underrun, and a real provider request in a lane that must stay keyless. Reject ambiguous prompt matches. Binding order followed filesystem enumeration, so a second fixture sharing a prompt could silently redirect a scenario to the wrong chunks and ledger; the spec's choice never reaches the server-side loop, so ambiguity fails instead of picking a winner. Fixture identity is the file name for the same reason — a recorded `meta.name` is descriptive, and trusting it let a copied fixture collapse onto another's registry key and ledger. Prove the recording is fresh. The spec removes the selected fixture before driving, so a run whose hook never installed the recorder fails instead of greening against a stale artifact whose answers still match these deterministic prompts. * fix(e2e): rewind a replay fixture at the conversation boundary Consecutive invocations can share a prompt — a tool call produces exactly that — so an attempt stopping mid-turn left the cursor on an invocation whose text still equalled the opening prompt. Matching the cursor first meant a retry's fresh conversation resumed after the tool call instead of rewinding, consuming the post-tool invocation and silently replaying a different script than was recorded. A conversation boundary now outranks an in-progress cursor: a fresh conversation whose prompt opens the fixture rewinds even when the cursor would have matched. Continuing still outranks restarting within a conversation, so a turn that calls a tool advances to its post-tool invocation rather than rewinding on its own repeated prompt. * fix(e2e): refuse cross-conversation binding and prove content streaming A fresh conversation could steal a partly consumed fixture's later turn. Only a conversation opening with the fixture's first prompt was treated as a boundary, so after `[A, B]` had consumed `A`, an unrelated new conversation whose first message was `B` matched the cursor, received the recorded second-turn response, and advanced the shared cursor without ever having driven `A`. A conversation start may now only rewind a partly consumed fixture, never continue it; continuation within a conversation is unaffected. The incrementality assertion counted empty frames. Providers emit empty initialization and usage-metadata chunks around the content deltas, so a total chunk count above one was satisfied by a single delta: the previous fixture's closing turn had four chunks and one content-bearing delta carrying the whole answer, and both modes stayed green without proving incremental assistant-content streaming at all. Fixtures now track content-bearing chunks separately, the closing prompt asks for prose rather than a number, and both modes require several content deltas on that turn. Re-recorded: the closing turn now carries 28 content deltas. * fix(e2e): scope record mode to the fixture spec `E2E_MODEL_FIXTURES=record` replaces the fake-model hook globally, so an unfiltered entry point such as `npm run e2e:mock` sent every spec under specs/mock to the paid real-provider endpoint, while each fresh conversation truncated and rewrote the one selected fixture — leaving an artifact from whichever scenario happened to run last. Record mode now matches only the fixture spec: an unfiltered recording run lists one test instead of 203. Replay mode is untouched and still collects the full suite. * 🪪 fix: Bind Replay Fixtures by Conversation, Not Prompt Text Prompt text was standing in for conversation identity, and three review rounds found the same class of defect underneath it: a tool call repeats a prompt across invocations, a retry repeats it across attempts, and a resumed run has neither prompt nor history because `createRun` is rebuilt with no messages while state comes from the checkpoint. Each fix in that space created the next gap. Thread the identity instead. `createRun` accepts a `conversationId` and passes it to the run hook, which the agents controller supplies at both call sites — the same value it already uses as the checkpointer's `thread_id`. The field is optional and the hook is env-gated, so nothing changes when the harness is not in use. Binding then collapses to ownership. A fixture is owned by the conversation that claimed it, and its cursor is authoritative wherever it stands: an extra turn reaches the over-consumption guard rather than falling through to the scripted fake model, and a resumed run keeps replaying with no prompt to match. A different conversation may claim the fixture only by opening it, which rewinds — what a Playwright retry looks like. Everything else is refused, so an unrelated conversation can no longer continue someone else's partly consumed script by repeating a later prompt. The prompt is still re-checked on every real turn; only a resume, which structurally carries no human message, is exempt. The previous text-and-history rules remain as a fallback when identity is absent. The recorder keys the same way: a new attempt is a new conversation, so a resume no longer truncates the fixture mid-turn and discards its tool-call invocation. Record summarization too. The summary provider runs on its own model with its own callback list, so a scenario crossing the context-pruning threshold recorded the agent's invocations but not the summariser's, leaving a fixture that could not reproduce the pruned context. * 🧾 fix: Harden Record Mode and Make the Rendered-Text Assertion Honest CI caught what local runs had not: the committed fixture was never replayed locally, because the record run overwrote it after the replay check rather than before. Re-recording and replaying in that order is what surfaced the rest of this. The DOM assertion compared raw recorded text against rendered markdown. The previous answer opened with `52.`, which Markdown renders as an ordered-list marker, so those characters never appear in the DOM and the match failed on all three CI attempts while replay itself was correct. The closing prompt now asks for prose beginning with a word, a leading enumerator is stripped before matching, and only a prose prefix is compared. Derived configs discarded the record-mode restriction. `config.redis.ts` and `config.mermaid.ts` spread this config and then replace `testMatch`, so `e2e:mock:redis` in record mode would still send its specs to the paid provider. A restriction expressed as an overridable value cannot hold, so record mode now refuses any config but the mock one. Superseded recording callbacks could write across a reset. A failed attempt with a provider call still in flight keeps its handler on the old graph; after the retry reset, that call would allocate an invocation from the new counter or append an `error` entry with a cleared mapping. Handlers now carry the recording generation they were installed for and ignore everything from an older one, and attachment dedupes against the current generation so a graph carried across a restart is not left with an inert handler. * 🚧 fix: Make Summarization an Explicit Boundary, Not a Half-Feature Recording summarization invocations without replaying them is worse than ignoring them. Replay routes the agent model and subagents only, so a recorded summarization entry takes a slot in the fixture sequence that replay never consumes, and the next primary call reads the summariser's chunks — a prompt mismatch or, worse, silently wrong content. The attachment was also aimed at the wrong shape: the SDK reads `summarizationConfig.parameters`, not `.parameters` nested under `.config`, so the previous attempt would have attached to nothing in a real run. Its test passed only because the test built the shape the code expected rather than the shape the SDK provides. Rather than ship a half-routed feature, recording now fails the moment summarization runs, naming the reason. Both shapes are guarded so the guard cannot miss the way the recorder did. Summarization fixtures need replay routing for that model before they can be supported. The derived-config guard added alongside it was itself broken: workers do not carry `--config`, and the argument lookup fell through to `process.argv[0]`, so every recording run aborted claiming the node binary was an unexpected config. The flag is now located explicitly and absence is treated as "not the process that parsed the CLI". * 🔒 fix: Close the -c Config Alias and Pin the Recorder's Fixture Name Playwright documents `-c` as an alias for `--config`, so record mode launched as `playwright test -c e2e/playwright.config.redis.ts` slipped past a guard that recognised only the long spelling. Both spellings and both `=` and space forms are now parsed. Accepting arbitrary fixture names also worked against the ambiguity check. This spec drives one fixed prompt pair, so recording under another name left two fixtures sharing those prompts; replay then refused to bind either and the keyless lane stopped working — a successful documented recording run could disable the suite it exists to serve. The spec now records only the fixture it owns and says so when asked for another. * 🔧 test: Record a Real Tool-Call Turn and Replay It Through the Tool Node The fixture format carried `tool_call_chunks` and the binding advanced through a turn's invocations, but nothing had recorded a real tool-calling conversation end to end — the path was covered only by hand-written synthetic fixtures, and it is the first one a new scenario would exercise. This records one: the provider calls the `remember_fact` MCP tool, the tool runs, and the model is invoked a second time with its result. That is the shape a single prompt cannot express — one user turn spanning several model invocations, all sharing one prompt — so it is what proves the turn-vs-invocation distinction the binding rules were built around. Replay drives the real tool node rather than replaying its output, so the tool executes again and the assertion checks its live result. Two fixtures now coexist, which the record path had to grow for: the config keeps an allowlist so an unknown name is still refused, record mode collects every replay spec, and each spec records only the fixture it owns and stands down for the others. MCP tools reach the model under a server-qualified name (`remember_fact_mcp_e2e-memory`); that qualification has changed before, so the assertions match the base name as a prefix rather than pinning the suffix. Verified: record 1 passed (15.6s, real API) then replay 1 passed (13.6s) against that fixture, ledger drained 2/2 invocations and 35/35 chunks; both replay specs together 2 passed; app-load, completion, chat and mcp-ephemeral 12 passed.	2026-08-25 18:29:55 -04:00
Danny Avila	290b8664d9	🧾 feat: Track Authoritative Agent Event Outcomes (#15213) ... * feat: track authoritative agent event outcomes * fix: isolate agent event outcome types * fix: declare agent event handler result * fix: simplify agent event status selection * fix: preserve authoritative event outcomes * fix: preserve terminal event evidence * test: use completed run-step envelope * test: scope deferred HITL question locator * fix: settle every agent event terminal path * style: sort terminal host action imports * fix: fence agent event terminal evidence * fix: recover agent event terminal settlement * fix: scope terminal retry hints by generation * fix: settle terminal host actions exactly	2026-08-25 18:20:57 -04:00
James Todaro	018775de07	🧾 fix: Honor Disabled Transactions on the Assistants Usage Path (#15100) ... * 🧾 fix: Honor Disabled Transactions on the Assistants Usage Path Thread the resolved transactions config through `recordUsage` from each of its callers, so `transactions.enabled: false` is honored on the assistants token spend path. * 🧾 fix: Thread the transactions config through the vision-request caller Address review: `ToolService.processVisionRequest` also records usage without the resolved config, and `recordUsage`'s documented return type did not match the function. * 🧾 fix: Set the resolved transactions config after the usage spread - provider usage could carry a `transactions` key that overwrote the trusted value - matches the ordering the other `recordUsage` callers already use	2026-08-25 08:30:12 -04:00
Danny Avila	862ebf3235	🪢 fix: Persist Failed Agent Turns Before Error Publication (#14118)	2026-08-25 06:50:46 -04:00
Danny Avila	ac2aef00f6	🫗 fix: Drain Quoted Excerpts Into Mid-Run Steering (#15175) ... * 🧭 fix: Carry Quoted Excerpts Through Mid-Run Steering "Add to chat" quote chips were dropped by every during-run steer path: the steer POST had no quotes concept, so a composer-origin steer left the chip staged (gluing onto the NEXT send) and a queued item steered into the live run lost its quotes silently. Quotes now ride the steer protocol end to end: - POST + admission: `quotes` on the steer body, normalized like the chat route's (getReferencedQuotes caps), part of the idempotency fingerprint only when present so pre-existing receipts still replay. - Injection: merged into the model-bound turn as Markdown blockquotes at both boundaries (text-only and media paths), mirroring prependQuotes. - Persistence + replay: the STEER content part stores `quotes` separately from the typed text; stampSteerPartMedia re-merges them per turn (even with resendFiles off) via the SDK's transient media stamp, with the quote block folded into the token budget. - UI: composer steers/interrupt-steers drain the chips (skill picks stay staged — they configure a NEW turn's run); SteerPart and the in-flight bubble render the same MessageQuotes reference blocks as user bubbles; queued/failed rows show a quote count; reconnect reseeds fall back to the server item's quotes when no local chip survives. - buildMessages keeps its zero-await path to the parallel context kickoff via a synchronous stamp-target probe. * 🧭 fix: Keep Quotes in the Client-Safe Steer Projection toPendingSteer is the projection behind resume-state pendingSteers, abort responses, and terminal leftover claims — dropping quotes there would lose them on exactly the recovery paths the reconnect reseed's server fallback relies on. * 🧪 test: In-Flight Steer Bubble Renders Carried Quotes * 🔁 fix: Re-Stage Quotes When a Pre-Quotes Replica Accepts the Steer Codex flagged the rolling-deploy window: an old replica 202s a quoted steer while dropping the excerpts, so the client cleared the chips for context the model never received. The 202 (fresh and receipt replay) now echoes quotesAccepted from the DURABLE item; a missing echo on a quote-bearing composer-origin steer re-stages the excerpts as composer chips — the pre-steer behavior, so they ride the next send instead of vanishing — and strips them from the surviving chip so a later terminal conversion cannot duplicate them. Queued-origin steers keep quotes on the item, whose restore paths already return it intact. The residual cross-version lost-ACK retry stays fail-closed as a 409 idempotency conflict (failed chip with retry controls). * 🔁 fix: Close the Remaining Cross-Version Quote-Loss Windows Codex round 2: - Send now of a quoted queued item against a pre-quotes replica now re-stages the excerpts too (the row is consumed and the words inject bare, so the composer is their only remaining home); the strip clears the chip's captured origin copy so reclaims and terminal conversions cannot duplicate them. - A quoted retry whose lost first ACK was accepted by a pre-quotes replica now REPLAYS that legacy receipt instead of 409ing: the stored fingerprint matching the quote-less hash of the same words proves the cross-version case, and the replayed 202's missing echo drives the re-stage. Different quotes against a quote-bearing receipt still conflict. - TSteerAppliedEvent.part gains the quotes field (typed SSE consumers). * 🧪 test: Drop the Stale Narrow SteerDrainOutput Alias The spec's local intersection re-declared injectedMessages with content: string, predating the SDK pin that declares the field natively (content: string | MessageContentComplex[]). Under CI's clean install the hook's BaseHookOutput is no longer assignable to that narrower alias; the plain PostToolBatchHookOutput is the correct type for every drain/boundary assertion. Verified against the published 3.6.16 dist and the local one. * 🔁 fix: Honor the Generation Owner's Quote Capability End to End Codex round 4: - steerQuotesCapable rides job metadata (createJob + HITL resume rewrite), mirroring preemptCapable's owner-recorded pattern: an upgraded admission replica no longer stores quotes — or claims them accepted — for a generation whose older owning drain would silently drop them at injection. The missing echo drives the client re-stage, and a later capable handover cannot double-deliver restored context. - Applied events reconcile dropped quotes: when a quote-less applied part settles a quote-bearing chip (the lost-202 ordering the ACK-echo path cannot see), resolveSteerChip and both reconnect settle paths re-stage the chip's excerpts before removing their only copy. mergeRestagedQuotes dedupe keeps every trigger idempotent for the same excerpts. * 🔁 fix: Re-Read Quote Capability at the Last Moment and Cap Restaged Chips Codex round 5: - A HITL resume rewrites steerQuotesCapable without changing the generation's createdAt, so the enqueue fence cannot see a capable-to-legacy handover landing during admission's awaits. Re-read the owner's flag immediately before item construction (paid for only by quote-bearing requests); the residual between re-read and enqueue commit matches preemptCapable's documented race. - mergeRestagedQuotes now respects the 10-quote contract with the staged chips winning: a restored tail that cannot ride the next send is dropped explicitly instead of rendering as a chip the submission would silently discard. MAX_QUOTE_COUNT moves to utils/steer as the single client source; QuoteButton imports it. * 🔁 fix: Steer Quote Coverage for Preflights, Memory, and Single-Scan Stamping Codex round 6: - Stored-message policy inspection now extracts steer-part quotes as quote fragments (path /content/N/quotes/M), so conversation import and shared link preflights inspect the newly persisted field exactly like top-level message.quotes. - The memory copy gets its own quote-merge stamp (text only, resendFiles false): formatAgentMessages ignores part.quotes, so without it a steer whose substance lives in its excerpt reached the chat model but never memory extraction. - collectSteerStampTargets replaces the boolean probe: buildMessages collects once and hands the targets to stampSteerPartMedia, keeping the zero-await fast path without scanning the history twice. * 🔁 fix: Redis Quote Plumbing, Conversion-Race Guard, and Quote-Bound Recovery Proof Codex round 7: - RedisJobStore.deserializeJob now restores steerQuotesCapable (the explicit mapper otherwise dropped it on every read, leaving quote steering inert in Redis deployments), with the round-trip spec extended. - Both Lua parked-steer projections (terminal close + generation replacement) forward item.quotes, matching toPendingSteer — a lost final no longer strips excerpts from durable recovery in Redis mode. - The no-echo restage reads the SURVIVING chip (reclaimRejectedChipQuotes): a terminal conversion that beat the delayed 202 already moved the quotes onto the queued follow-up, and re-staging them again double-delivered. Regression-tested with the conversion-before-ACK ordering. - RecoveredSteerPayload binds normalized, order-significant quotes (builder, validator, TS matcher, and the Lua decode+matcher): a stale client presenting the same recoverySteerId with altered or missing quotes cannot consume the parked source. Quote-less sources keep matching quote-less recoveries. * 🔁 fix: Execution-Bound Quote Capability with an Atomic Enqueue Predicate Codex round 8: - steerQuotesCapable becomes a transient assertion translated (at createJob and in ApprovalLifecycle.resolve) into steerQuotesExecutionId, valid only while it equals the LIVE providerExecutionId. A legacy replica winning a HITL resume rewrites the execution id without knowing the marker, so its stale assertion self-invalidates — a bare boolean could not be cleared by code that predates it. - The fenced enqueue evaluates that equality atomically (all three Redis scripts decode-and-strip like the existing preemptCapable normalization; both InMemory sites mirror it) and returns the persisted item, so the quotesAccepted echo reflects exactly what was stored even when a handover lands between admission's read and the commit. The last-moment re-read is gone — the transaction is the authority. - Tests: capable-resume re-binding, legacy-resume omit-not-clear invalidation, the admission-vs-handover race (capability read true, then execution rewritten before enqueue), and the Redis round-trip of the marker. * 🔁 fix: Full Redis Parking Coverage and Loss-Moment Quote Restaging Codex round 9: - The two remaining Redis parking projections (terminal status CAS and stale-running cleanup) forward item.quotes — every field-picked steer projection now carries them (audited: 2 Lua 'projected' + 2 Lua 'clientItem' + toPendingSteer). - The ordinary no-echo ACK no longer re-stages: the steer has not injected yet, so the quotes stay carried on the pending chip. A quote-less applied event re-stages them at the actual loss; a terminal leftover conversion carries them onto the recovered row, whose normal send delivers quotes on any server — re-staging at the ACK let that leftover auto-send bare text while the excerpts glued onto an unrelated draft. Only the settled receipt replay (already injected, no future event) reclaims immediately. * 🔁 fix: Legacy-Replayable Receipts with Separate Quote Identity Codex round 10: an upgraded-first receipt stored a quote-inclusive fingerprint no pre-quotes replica could recompute, so a lost-ACK retry routed through one 409'd already-accepted words with duplicate-send controls. The durable fingerprint reverts to the quote-independent 3-field hash — the one shape EVERY deployed version computes, replayable across a rolling deploy in both directions — and quote identity moves beside it as requestedQuotesFingerprint (of the REQUESTED quotes, pre any capability strip, so an incapable-owner acceptance still replays its own retries). Absent records (legacy-written or quote-less) accept any same-words retry, preserving the round-5 rule; present records must match exactly, keeping different-quotes clientSteerId reuse a 409 on quote-aware readers. Under the keep-on-chip client contract a legacy replay's missing echo is harmless — the excerpts stay carried on the pending chip. * 🧪 chore: Re-Trigger CI After Dropped Workflow Events	2026-08-24 22:29:13 -04:00
Danny Avila	5a8700643c	⚡ perf: Build the Memory Message Copy Only When Something Reads It (#15164) ... buildMessages formatted every history row twice per turn — a prompt copy and a memory copy — then discarded the entire memory payload unless some row carried fileContext, which is the rare case. The memory copy has exactly two consumers: that payload, and the canonical recount of a row, where it is content-identical to the prompt copy unless the row itself has fileContext. So the prompt copy is now the recount surface for context-free rows, a fileContext row builds its memory copy at recount time, and the full memory payload is assembled in a deferred pass — same formatting, same per-row merge order — only once a row has proven the payload will be kept. The common turn formats each row once instead of twice and no longer allocates a payload it throws away. Also forwards the run's useLegacyContent to formatAgentMessages as the new legacyContent option, inert on the current SDK release: once the SDK change ships, text history is emitted pre-flattened so the per-request legacy projection stops cloning every message and the context meter's identity-based count reuse holds across the projection.	2026-08-24 10:33:47 -04:00
Danny Avila	fc2b8584c4	📇 feat: Surface Event Child Activity Through a Bounded Parent Index (#15142) ... * feat: surface event-driven child activity * fix: keep child task aggregation documentdb-compatible * fix: address event activity review findings * test: provide markdown message context defaults * fix: report bounded child history truncation * fix: preserve current child activity state * fix: preserve durable event child activity * fix: handle missing task timestamps * fix: keep active event snapshots live * fix: preserve event activity across valid anchors * fix: close event child activity gaps * fix: preserve event activity across resume	2026-08-23 18:50:00 -04:00
Danny Avila	77cb72e50c	🧮 perf: Enable Agent Context Count Reuse (#15130) ... * perf: enable agent context count reuse * fix: declare token counter return type * fix: initialize cached token counters * style: sort token counter imports * fix: Keep cached token counts exact	2026-08-23 03:05:02 -04:00
Danny Avila	dd146ff74d	🧾 fix: Report Complete Agents API Usage (#15127) ... * fix: report complete agents api usage * fix: preserve invoked usage context * test: cover absent usage context * fix: type responses usage finalization * fix: preserve reasoning usage aliases * fix: declare reasoning usage alias	2026-08-23 02:37:09 -04:00
Danny Avila	2018c70040	🧫 test: Lock Subagent File Context Propagation (#15126)	2026-08-23 02:06:57 -04:00
Danny Avila	c2aa688d73	🖥️ feat: Stream PTC Inner Tool Calls as a CLI-Style Trace (#15115) ... * 🖥️ feat: Stream PTC Inner Tool Calls as a CLI-Style Trace Programmatic tool calling runs a whole program inside the sandbox, and the tool calls that program makes open no run step of their own. The card showed one running spinner for the entire execution, with no sign of what the code was doing. Emit a new `on_ptc_tool_call` step event for each inner invocation — once on dispatch, once on settle — and render them under the code as a terminal-style trace: status glyph, tool identity, argument preview, duration, with a failure message printed under the call that produced it. The seam is the tool map the sandbox bridge resolves inner calls against. `instrumentPtcToolMap` proxies `invoke` on each entry and leaves every other property (name, schema, mcp) passing straight through, so nothing about execution changes and emission failures can never fail a tool call. Client state is a per-tool-call Recoil atom keyed like the sandbox-starting and subagent atoms — live for the session, cleared on conversation switch so a finished program's trace stays readable. * 🩹 fix: Address Codex Review on the PTC Tool Trace Five findings, all confirmed against the source before fixing. Scope the trace atoms to a message occurrence. The hook already documents that providers repeat a tool_call_id across turns and even within one message, and `call_id` restarts at :0 for every outer call — so two programs sharing `call_0` merged into one card. Key by (response message id, tool call id) via `ptcTraceKey`, mirroring `subagentProgressKey`; the event's `runId` already carries the message id and the card reads its own from MessageContext. Prune unsettled rows on resume. Inner calls are not content parts, so the resume snapshot cannot rebuild them, and `trackReplayEvent` only persists OAuth events — a call that settled during a disconnect left a spinner that never resolved. Settled rows are real history and stay. Make the argument preview budget-aware. Iterate keys rather than entries so the budget check can actually skip work, and clip against a bounded window so a multi-megabyte value is never collapsed in full to build a 40-character preview. Catch the resumable emission promise. The synchronous try/catch around the emitter cannot observe a rejected `emitChunk`, so a failing transport raised an unhandled rejection per event instead of dropping telemetry. Announce completion to assistive technology. The check glyph is decorative and a fast call renders no duration, so a settled row previously announced no outcome; each row now carries an sr-only status and the visible cell that duplicated it is hidden. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MBRQUsPXSvDBnjrSDoXWHP * 🧹 fix: Repair CI Failures on the PTC Tool Trace Two failures on the previous head, both mine. `Tests: api (shard 2/3)` — 46 failures in `initialize.spec.js`, all `TypeError: createPtcProgressEmitter is not a function`. The suite mocks the callbacks module with an object literal, and wiring the new emitter into `initialize.js` without adding it there left the factory undefined at call time. Added it alongside `createAttachmentEmitter`, plus an assertion that it receives the same generation fence as every other resumable emitter — a stale epoch would leak one run's inner calls into the next. `Static checks` — import-order drift in `PtcToolTrace.tsx` and `handlers.ts`, repaired with `scripts/sort-imports.mts`. ESLint and Prettier both passed, so only the dedicated check caught it. `openai.js` and `responses.js` never take the emitter, so their specs were unaffected; verified the initialize mock now covers every name the module destructures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MBRQUsPXSvDBnjrSDoXWHP * 🔐 fix: Address Second Codex Review on the PTC Tool Trace Three of five findings actioned; two answered on the thread. Respect tool-argument PII filtering (P1). Inner calls never reach `filteredToolArgumentsResult` — the sandbox bridge invokes them directly — so the trace was the one path putting their values on the wire in a deployment that had configured `filters.toolArguments.pii`. When any of the name / arguments / output fields are filtered, the emitter now omits both the argument preview and the failure message, which routinely quotes the argument that caused it. Name, status and duration still report. Drop the light/dark-specific background (P1). `dark:bg-transparent` stepped outside the semantic roles and would lose the intended separation under a custom theme. The pane now sets no background at all and inherits the card's surface, which resolves to the same color the override produced in both default themes and stays correct when a theme reassigns its roles. Bound the live trace (P2). A program looping over a large collection made every event copy an ever-growing array and rendered a row per call. The trace now keeps a rolling tail of 100 rows and counts what it evicted, surfaced as "+N earlier calls" so the cap is never silent. A settle whose row is gone — evicted, or pruned across a resume gap — no longer reappears out of order. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MBRQUsPXSvDBnjrSDoXWHP * ✅ test: Keep PTC Trace Tests Aligned With Caller-Capability Filtering Left out of the merge commit by a staging slip; without them `handlers.spec.ts` fails on the merged tree. `#15105` restricts the PTC tool map to tools whose `allowed_callers` admit code execution, so the existing trace test's registry entry — which declared none, defaulting to `direct` — was filtered out before the instrumentation could see it. Declare the fixture `code_execution`. Add a guard for the resolution itself: a `direct`-only tool must never appear in the instrumented map. Tracing wraps the eligible map, and this fails if a later change reorders that and lets the trace widen what the sandbox reaches. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MBRQUsPXSvDBnjrSDoXWHP * 🛡️ fix: Close Name Disclosure and Follow the PTC Trace Tail Two findings from the third Codex pass on `17a9ec9`. Redact filtered inner-tool names (P1). The previous gate suppressed argument and failure previews but the event still carried `name` verbatim, so a deployment whose `filters.toolArguments.pii.fields` includes `name` could see a blocked identifier disclosed through the trace — the one path inner calls take, since they never reach `filteredToolArgumentsResult`. Inner tool names are now inspected once per PTC call with the same `extractToolArgumentContent` + `inspectContent` pair the executor uses; any that trip the policy are left unwrapped, so they still execute and emit nothing. An un-inspectable name fails closed. Follow the trace tail (P2). The row list is a 200px scroller that never moved, so once a program exceeded the viewport the card sat on the oldest calls while live activity accumulated below the fold. Reuse `useFollowScroll` — the hook the code and command panes already use — which pins to the tail while calls are running and yields the moment the reader scrolls up. The host card threads its disclosure state so a collapsed pane is never scrolled invisibly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MBRQUsPXSvDBnjrSDoXWHP * 📌 fix: Pin the PTC Trace Through Its Final Settle The fourth Codex pass on `4bf68e1`, one P2 finding. `useFollowScroll` returned early whenever `active` was false, so the one change it most needed to follow was the one it skipped. A failing inner call settles by appending its error line in the same commit that clears the last running row: the content grows and the stream ends together, and the pin that would have revealed that line never fired. On an expanded, bottom-pinned pane the failure — the row a reader most wants — stayed below the fold. The falling edge of `active` now pins too, but only when the content changed with it. Ending a stream on its own still leaves the pane where the reader left it, which is what the existing contract promises and what the sibling code and command panes rely on; a reader who has scrolled up is untouched either way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MBRQUsPXSvDBnjrSDoXWHP * 🔌 fix: Keep PTC Calls That Outlive a Reconnect Fifth Codex pass on `085a83f`; one of its two findings. Pruning rows across a resume gap deleted every `running` row, but a stream gap is not proof the call ended. A call still executing across the reconnect settles normally on the restored live stream — and `applyPtcToolCall` drops a settle whose row is gone, by design, so an evicted row cannot reappear out of order. The call therefore vanished from the trace despite having run, which is worse than the spinner the pruning existed to prevent. Rows are now marked `interrupted` instead of removed. A call whose settle was genuinely lost in the gap reports that honestly rather than spinning forever, and one that survives the gap settles onto the row it opened, reporting its real outcome and duration. `interrupted` is a client-side conclusion, so it widens the row status locally and leaves the wire contract alone. Two cases added: the gap marks rather than drops, and a post-reconnect settle lands on its marked row; plus a render case for the new outcome. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MBRQUsPXSvDBnjrSDoXWHP --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-23 01:18:14 -04:00
Danny Avila	1de88e7e91	📨 feat: Continue Bound Child Agents from Events (#15112) ... * feat: add authenticated agent event ingress * style: sort agent ingress imports * fix: harden agent event ingress * fix: bind event provenance to API keys * fix: inspect event input with legacy PII filters * fix: scope event status reads to source keys * fix: bind event status reads to remote sources * feat: add bound event-driven child turns * fix: harden event-bound child continuations * fix: satisfy event binding type contracts * fix: close event actor lifecycle races * fix: harden event actor dispatch continuity * fix: fence event actor resume lifecycle * fix: bind event actor state to lifecycle * fix: preserve cascade write outcomes * test: type cascade failure injection * style: sort cascade test imports * fix: harden event child lifecycle boundaries * fix: make event cleanup retryable * fix: annotate event retention clock * fix: reconcile partial cascade metadata * fix: recheck event binding expiry on resume * fix: fence event actors by retention deadline * fix: close event actor lifecycle races * fix: harden event child lease acquisition * fix: lazy-load event child lease adapter	2026-08-23 01:15:57 -04:00
Danny Avila	8f9fae0a6e	🛂 fix: Preserve Legacy Assistant Attribution (#15118)	2026-08-22 16:21:45 -04:00
Danny Avila	89494d45fd	🚦 fix: Restrict Programmatic Tool Execution Maps (#15105) ... * fix: restrict programmatic tool execution maps * chore: bump `@librechat/agents` to v3.6.10 * fix: honor live programmatic caller projections * style: sort caller capability imports * test: expect caller projection loader argument * chore: bump agents sdk to v3.6.11 * refactor: use SDK caller projection type * style: sort agent handler imports	2026-08-22 11:02:09 -04:00
Danny Avila	67b7b441b2	🛂 feat: Filter Model-Bound Content by Source (#14425) ... * feat: introduce optional content protection seam * feat: enforce source-aware content filters * feat: complete source-aware content enforcement * test: activate skill file-text fail-close fixtures * fix: harden source-aware content filters * fix: harden model-bound content filtering * fix: preserve legacy filters and generated files * fix: inspect shared scalar metadata * test: align mocks with current dev dependencies * feat: add persisted content filter safeguards * feat: complete source-aware content filter enforcement * fix: move resume content preflight into TypeScript * fix: close content inspection edge cases * fix: harden content protection boundaries * fix: complete content protection safeguards * test: align persisted memory filter coverage * fix: reconcile content protection with current dev * fix: reconcile content protection with latest dev * fix: close content protection review gaps * fix: enforce source-aware provider boundaries * fix: preserve legacy PII preflight semantics * test: stabilize stored branch preflight fixture * fix: defer agent writes until protected model admission * perf: harden source-aware model-bound filtering * fix: canonicalize provider lineage before validation * fix: satisfy model-bound callback type checks * perf: Bound content protection filtering work * fix: Bound submission array traversal * fix: Stabilize bounded content snapshots * fix: Scope model-bound traversal overflows * fix: Preserve scoped content inspection * fix: Accumulate aggregate traversal scopes * fix: centralize content policy boundaries * test: align deferred tool policy context * test: align controller policy mocks * style: normalize content protection imports * fix: close content policy review gaps * fix: narrow active skill policy config * fix: address content protection review boundaries * fix: retain exact provenance overflow sentinel * fix: preserve literal and scoped provenance updates * fix: narrow persisted edit provenance * fix: isolate exact overflow attribution * fix: centralize stored prompt protection * fix: fail closed on incomplete transcript evidence * fix: align canonical transcript routing * refactor: centralize content policy preflights * fix: isolate upload policy error typing * style: sort policy preflight imports * refactor: centralize content policy boundaries	2026-08-21 22:43:32 -04:00
Danny Avila	8ae94afa91	🪡 fix: Thread Parent Message ID Through MCP Request-Scoped Bodies (#15095) ... * fix: Unify MCP request-scoped headers * fix: address request-scoped MCP review findings * test: preserve request scope on status errors * fix: treat authorized on-demand MCP servers as ready * refactor: separate MCP readiness from connection state * fix: preserve on-demand MCP readiness labels * test: satisfy OpenAI conversation ownership guard * fix: keep MCP action predicates boolean * fix: close deferred MCP request context gaps * fix: preserve on-demand MCP configuration actions * fix: fail closed on unavailable MCP parent context * test: complete MCP connecting-state mocks * fix: preserve missing MCP parent on continuations * fix: align native MCP request identities * fix: preserve edited MCP parent identity * test: use scoped Agent initializer fixture * test: expose MCP request body helper * fix: preserve MCP turn identity across resume * style: sort stream metadata imports * fix: carry normalized MCP identity to execution	2026-08-21 16:33:01 -04:00
Dustin Healy	f02ce63d57	✂️ fix: Strip Redundant Server-Name Prefixes from MCP Tool Keys (#14732) ... * ✂️ fix: Strip Redundant Server-Name Prefixes from MCP Tool Keys MCP servers that prefix every tool with their own name produce model-facing keys that embed the server twice once the _mcp_<server> suffix is appended, pushing long tool names past provider 64-character function-name limits. Tool keys now drop a leading <normalizedServerName>_ prefix (case-insensitive, skipped when a sibling tool already owns the stripped name). The original upstream name is recorded as serverToolName on the cached definition and is always what tool calls send to the server, and runtime lookups also try the stripped spelling of persisted pre-strip keys so existing agents keep resolving. * 🩹 fix: Keep Stripped MCP Tool Keys Provider-Safe and Collision-Free Assistant writers submit catalog entries verbatim, so the internal serverToolName mapping is now removed from provider-facing definitions before they reach create/update payloads. Prefix stripping is collision-guarded over the resulting name set rather than raw siblings only, which also covers case-variant prefixed pairs under the case-insensitive match. Assistant payload healing now rewrites a pre-strip persisted key to the stripped catalog key when that key actually exists in the loaded definitions, and legacy agent references keep their persisted spelling as the runtime instance name so per-tool options stay applied while the upstream call still uses the matched entry's raw name. * 🧷 fix: Harden Stripped MCP Tool Keys Against Heal, Collision, and Cache Edges The pre-strip heal now resolves the key boundary against both raw and normalized server spellings, mapping back to the raw name for the shadow and membership guards, so keys persisted after server-name normalization heal too. Collision detection iterates to a fixpoint so a fallback to a raw name cannot silently collide with another sibling's stripped result, and a stripped remainder equal to a synthetic marker (wildcard or server pin) is never produced. MCP catalog cache slices are versioned so replicas that predate serverToolName never read stripped entries during a rolling deploy; stale slices expire on their own. * 🔎 fix: Resolve Pre-Strip Keys in Event-Driven Definitions and Reinspect Persisted Catalogs The event-driven definitions loader now tries the stripped spelling of a persisted key when the exact lookup misses, keeping the persisted name so it matches the runtime instance, which stops legacy agents from failing initialization with expected tools unavailable. The registry storage schema version is bumped so followers rebuild persisted toolFunctions instead of republishing pre-strip definitions into the versioned catalog namespace. The assistants heal also fails closed when a normalized-suffix reference lands on a contested server-name slot, since rewriting persisted data must not bind an ambiguous reference to the tie-break winner. * 🛰️ fix: Reserve the Synthetic OAuth Name and Heal User-Owned Server Keys A stripped remainder equal to oauth would make the client stream handlers treat a real tool call as a synthetic authentication prompt, so it joins the reserved remainders alongside the wildcard and pin markers. The assistants heal now audits the FULL accessible server set on every run instead of operator config names only, since assistants reference user-owned servers whose catalogs the definitions loader already resolves; an unavailable audit still skips healing entirely. * 🧬 fix: Verify Upstream Identity for Legacy Keys and Reserve Sibling Raw Names Stripped results now reserve every sibling's raw name even when that sibling itself strips, so a stripped key can never shadow another tool's pre-rollout persisted references within the same snapshot. Every legacy fallback (runtime lookup, event-driven definitions, assistants heal) accepts a stripped-spelling match only when the entry's recorded serverToolName proves the same upstream tool, so a stale key for a removed tool degrades to unavailable instead of calling a different sibling. To keep that identity visible to the heal, assistant tool definitions retain serverToolName and the controllers sanitize entries through toProviderToolDefinition at the provider submission boundary instead. The agent editor migrates pre-strip persisted ids the same identity-verified way, with the upstream name exposed on the MCP tools payload. * 🧭 fix: Heal Wildcard Tool Options and Reserve the OAuth Namespace Wildcard-expanded catalogs rename stripped tools without any agent.tools entry to preserve the spelling, so buildToolClassification now aliases persisted pre-strip tool_options keys onto the current instance names in place, identity-gated on the definition's recorded upstream name and never overriding an explicit entry. Both loading modes flow through it: instances carry mcpServerToolName from createToolInstance and event-driven definitions thread serverToolName from the catalog. stripServerNamePrefix also reserves the entire oauth namespace rather than the exact name, since the client stream handlers classify every oauth-prefixed key as a synthetic authentication call. * 🛡️ fix: Derive the Full Reserved Namespace and Heal Approval Policies The reservation guard now covers every namespace consumers classify by prefix: the wildcard and server-pin markers alongside oauth, plus the server-scoped mcp_ pluginKey namespace that pre-strip keys could never enter. Stripping also never produces a key whose isActionTool classification differs from the raw key's, since a server whose normalized name contains _action_ would otherwise see a real MCP tool routed down the OpenAPI action path past MCP authorization. Admin toolApproval globs written against upstream tool naming keep applying: pattern lists are healed at run wiring with the current names of tools whose pre-strip spelling matches, list-level so deny, ask, and allow precedence is unchanged and a non-matching deny can no longer fail open. The MCP tools wire type also declares serverToolName end to end. * 🪪 fix: Alias Both Key Spellings for Approval Policies and Hook Matchers Identity aliases are now collected once at tool classification, in both directions: a stripped instance aliases its pre-strip spelling and a legacy-named instance aliases its current catalog spelling, with the current name recorded on legacy matches by the runtime lookup and the event-driven definitions loader alike. The aliases ride the agent config through both loading modes, so approval pattern healing applies to deny rules written against either spelling, closing the bypass where a rule targeting the current name missed an unedited agent's legacy instance. Programmatic approval hook matchers get the same treatment: each hook is additionally registered under an anchored exact-name pattern for tools whose other spelling its regex matches, keeping the admin's matcher semantics intact while argument, user, and tenant specific deny or ask decisions keep executing for renamed tools. * 🔁 fix: Alias Tool Options in Both Spelling Directions Options aliasing now consumes the same bidirectional alias pairs as policy healing and hook matchers, so options the editor migrated to the current catalog spelling still reach a legacy-named instance retained by an unedited agent.tools entry. The previous serverToolName-only derivation skipped exactly that case since the legacy key equals the instance name there. * 🤝 fix: Reserve the Agent Handoff Namespace Before Stripping The client renders any lc_transfer_to_ prefixed call as an agent handoff and the background and intent passes exclude such names, so a stripped remainder inside that namespace would misclassify a real upstream tool. It joins the mcp_ pluginKey namespace as a bare-prefix reservation, which pre-strip keys could never enter. * ⚡ fix: Reuse the Loader's Server Snapshot and Index the Editor Catalog getAssistantToolDefinitions now returns the accessible-server snapshot from the same merged registry read that resolved the catalogs, and the heal consumes it instead of repeating the app-config and registry round trips on the assistant write path; without a snapshot the heal still fetches and fails closed as before. The agent editor's id migration uses a memoized tool_id map, so the per-key form heal does constant-time lookups instead of scanning the catalog per option. --------- Co-authored-by: Danny Avila <danny@librechat.ai>	2026-08-21 14:30:29 -04:00
Yorgos K	9f8d71a3c5	🪢 fix: Preserve Response Identity and Branch During Resumable SSE Sync (#14788) ... * fix(client): preserve resumable response identity Fixes #14787 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(client): align resumable sync regressions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(client): clarify resumable response ownership Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(client): preserve resumed regeneration ordering Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(client): cover missing resumed response row Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(client): preserve resume identity on page reload * fix(client): replace reassigned resume placeholder * fix(client): preserve content during response id handoff * fix(client): limit resume placeholder handoff * fix(client): preserve resume display metadata * fix(client): reconcile resume metadata in one pass * fix(client): reconcile preliminary resume user * fix(client): restore regenerated branch on early abort * test(client): cover external regeneration resume * fix(client): preserve regeneration history on errors * fix(client): replace reused regeneration error ids * fix(client): preserve exact-id regeneration rollback --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Danny Avila <danny@librechat.ai>	2026-08-21 14:25:03 -04:00
Danny Avila	8c14f03432	🗂️ feat: Scope Scheduled Chats to Chat Projects (#15056) ... * 🗂️ feat: Scope Scheduled Chats to Chat Projects Adds an optional chat-project destination to a schedule, plus the operator config to require one — or to pin every scheduled run to a specific project. Feature: - `chatProjectId` on the schedule row, accepted on create/update, projected on the wire, and carried into the run's conversation through the durable trigger envelope's `run` context. - `interface.schedules.requireProject` refuses schedules that are not filed under a project; `interface.schedules.projectId` pins every run to one project and implies the requirement. - Dialog gains a project picker (required when configured, a read-only row when pinned); the card shows the destination and the new disabled reasons. Invariants: - ONE resolver (`resolveScheduleProjectId`) decides the destination for the write handler, the fire path, and the wire projection alike, and an operator pin OUTRANKS the stored id in all three. Tightening the config therefore redirects — or stops — existing schedules instead of grandfathering where their runs land. A pin implies `requireProject` for the same reason: without it, a row created before the pin would keep firing with no project at all. - Create/fire precheck symmetry, mirroring `resolveAgentFireAccess`: a write this handler accepts is one the next fire also accepts. Any edit leaving a schedule ENABLED re-validates its EFFECTIVE (possibly stored) project, like the existing stored-agent and cadence-floor rechecks. A DISABLING edit skips the requirement, or a schedule auto-disabled for `project_required` could never be turned off. - Fire-time enforcement auto-disables rather than filing runs loose, matching agent_deleted: new `project_required` (requirement raised after creation) and `project_deleted` (gone, or pinned to a project this owner does not have) reasons, both refused BEFORE a billed generation is dispatched, and both advancing so a schedule can never wedge on the occurrence. - `computeCreateDigest` appends the field only when present, so a payload without a project digests byte-identically to one from before this change — a create retried across the upgrade still matches its own row instead of reading as key reuse. - Project reads are scoped to the owner, so ownership and existence are the same lookup; a read ERROR propagates instead of failing closed, so a Mongo blip retries the fire rather than auto-disabling the schedule. The trigger idempotency key hashes principal/event/target and never `envelope.run`, so the added run field cannot destabilize delivery identity. * 🗜️ fix: Keep the Schedule Dialog Inside Its Height Budget The project picker landed as a new ROW in the schedule dialog, which broke the e2e edit spec: `md:overflow-visible` turns off the template's scrolling from `md` up, so the dialog's content must fit the viewport. The extra row pushed the footer's Save button below a 720x1280 window, where Playwright reported a visible, enabled button it could never click — 226 scroll-into-view retries and a 2-minute timeout, on all three attempts. Measured against `dev` at 1280x720 (Save button's bottom edge, viewport 720): dev 688 (32px slack) project row ~790 (off-screen, CI failure) 3-column row 704 (16px slack — half the budget spent) this commit 686 (34px slack, 2px better than dev) The identity row is now three columns — name, agent, project — and its caption moved out of the agent cell to sit full width beneath the row: at a third of the dialog that sentence wraps an extra line, and the row is the tallest thing competing for the budget. The caption is grouped with the row rather than left to the form's own 4-unit rhythm, which spent more height on the gap than the caption occupies. The e2e spec now asserts the button is in the viewport before clicking it, so the next field that overflows this dialog says so in one line instead of a two-minute timeout on a visible element. FOLLOWUPS.md records what the planned dialog controls (multi-day weekly, timezone, attachments) need first: give `ControlCombobox` the `portalElement` prop `Dropdown` already has, portal the popovers into the dialog content, and let the form scroll again. * 🧹 fix: Address Codex Review on Scheduled Chat Project Scope Four P2 findings, all real. Unreachable clearing path (ScheduleDialog). The picker only held live projects, so `com_ui_schedule_project_none` was a PLACEHOLDER — nothing selectable. Once a schedule had a project the owner could never take it away, leaving the server's `chatProjectId: null` path reachable only by API. The picker now carries a real "No project" option whenever a project is optional, and omits it when one is required, where there is nothing valid to select. Placeholder shown for a real project (ScheduleDialog). A stored or pinned project outside the first loaded page had no name in the paged map, and the combobox renders its placeholder for an empty display value — telling the owner a scoped schedule had no project. That one project is now read by id, with the raw id as a last resort: a poor label, but an honest one. Project policy skipped at the resume boundary (service.ts). `claimScheduleResume` re-applied the schedules gate, the revision fence, the kill switch and SCHEDULES:USE, but not the project policy this PR added. Approving a paused run whose project was deleted — or whose owner now sits under a requirement or a pin it no longer satisfies — billed a continuation the very next scheduled fire would refuse and auto-disable the schedule for. The effective-project resolution now runs there too, refused before the lease and the capacity slot so a policy refusal costs nothing and leaves no state to unwind. NOTE: agent access and balance are still not rechecked on resume; that gap predates this PR and is left alone. Per-card project derivation (ScheduleCard). Every card ran the projects hook and rebuilt the full option array, name map, and one icon element per project, to use a single name — O(schedules x projects) per render and per project-list refresh. The hook is split: `useChatProjectNames` (map only, for the panel, which resolves every card's name once and passes it down) and `useChatProjectPicker` (options and pagination, for the dialog's one combobox). The panel skips the query entirely until some schedule actually has a scope. Tests: four at the resume boundary (verified failing without the gate) and four in the dialog spec. The picker selection in one existing test now goes through the search field — the popover's VIRTUALIZED renderer sizes its window from a scroll height jsdom always reports as 0, so with three options it materialized only two. Full schedules e2e re-run green against the rebuilt client. * 🎯 fix: Settle Project-Policy Refusals and Keep Create Retries Idempotent Second Codex round, four P2s. Three were consequences of the resume gate added in the previous commit, which was half-built: it admitted where it should not and stranded the run where it refused. Project policy moves from `claimScheduleResume` into `isScheduleLive`'s `policy` branch. Both entry points consult that branch FIRST, and both already route its refusal through abort-and-settle — so a policy stop now settles the occurrence instead of answering a bare 409 while the job stays `requires_action`, the card keeps reading "Needs approval", and every retry repeats the same 409 until expiry. No change to resume.js: the existing branch does the work. The rule is deliberately NARROW. It refuses only where no valid destination is left — the requirement is on with nothing satisfying it, or the schedule's own project is gone (which also unset it on the conversation). It does NOT refuse because an operator's pin moved: the paused conversation cannot be rebound (`chatProjectId` is excluded from the resume context and the continuation reuses the same conversationId), so refusing would strand a pending approval over a destination it can never reach, for a pin that governs only where the NEXT run lands — which the fire path already redirects. Create retries are idempotent again. Project policy had been applied BEFORE the `clientRequestId` replay lookup, so a raised requirement, a deleted project, or a moved pin could answer 400 for a create that already committed — pushing the client to rotate its key and create a DUPLICATE schedule, the exact failure the key exists to prevent. Policy now applies only to a genuinely new insert, and the digest is computed from the CLIENT's payload rather than the resolved destination, so today's policy can no longer re-digest a genuine retry into a mismatch. An explicit `chatProjectId: null` under a pin is refused rather than silently resolved to the pin. The payload contract defines `null` as clearing the scope; answering 201 while filing under the pin reported success for the opposite of what was asked. Only an OMITTED field takes the pin silently. Tests: five on the policy branch (both refusals verified failing without it, plus guards that a moved pin and a live project still admit) and two on the handlers (the pinned explicit clear, and a committed create recovered by retry after the policy tightened — verified answering 400 without the reordering). * 🧭 fix: Converge the Stored Project on the Destination a Fire Resolved Third Codex round, four P2s. The root confusion behind the resume findings: an operator pin outranks the stored id at fire time, `fireSchedule` sends the pin in the trigger envelope, and the row keeps its old value. The row therefore LIED about where that occurrence's conversation went, and every later re-validation — the resume boundary above all — checked a project the conversation was never filed under. A schedule storing A, pinned to B, with B later deleted and A still live, was admitted for resume into a conversation that had just been unscoped. Fixed at the source rather than at each reader: a fire that resolves a destination different from the stored one writes it back, claim-token fenced like every other worker-side write and deliberately WITHOUT a configRevision bump — this is the server reconciling itself to policy, not an owner edit, and a bump would fence an in-flight occurrence off its own run. Written only AFTER the destination validates, so an unusable pin never lands in the row, and best-effort: the envelope already carries the right destination, so a failed write costs accuracy on a later recheck, never the run itself. The wire projection already reported the pin, so this also stops the row and the UI disagreeing. An explicit `chatProjectId: null` under a pin is now refused on the DISABLING edit path too. The pin check and the requirement are independent rules, and folding them together let `{enabled: false, chatProjectId: null}` skip the pin check entirely, unset the row, and answer with a wire projection still naming the pin. Only the requirement is waived for a disabling edit. The dialog no longer requires a project for an edit that leaves a schedule DISABLED. The server waives the requirement there precisely so a row auto-disabled for `project_required` can still be renamed or tidied up; requiring it in the form made that unreachable, and an owner with no projects could not edit the stopped schedule at all. FOLLOWUPS.md records the two residual gaps with their exact triggers: the sub-second deletion race inside the resume claim window (which needs the effective project persisted per OCCURRENCE plus a distinct policy conflict routed through abort-and-settle), and the fact that convergence happens only when a schedule fires. Tests: four on convergence (pin written, no write when unchanged, no write for a destination that failed validation, fire survives a failed write), two on the disabling-edit pin rules, two in the dialog. Schedules e2e re-run green. * 🔑 fix: Keep an Explicit Project Clear Out of an Omitted Field's Digest `computeCreateDigest` appended `chatProjectId` on `!= null`, so an OMITTED field and an explicit `null` produced the same digest. Because the replay lookup and `matchesCreateIntent` deliberately run before project policy, a request could reuse a pinned create's `clientRequestId` while explicitly sending `chatProjectId: null` and receive 201 describing the pinned row — success reported for the opposite of what it asked, and the pinned-clear refusal the normal create path applies never reached. Now `!== undefined`: an omitted field still digests byte-identically to a payload from before project scope existed, so a create in flight across the upgrade still matches its own row, while an explicit clear is a distinct intent and digests differently. A pre-scope client never sent the field at all, so nothing legacy can carry an explicit null. * 📍 fix: Validate a Paused Run Against the Project Its Own Occurrence Used The schedule-wide convergence from the previous commit was not enough, and the reason is the single-active run index: it covers `status: 'started'` only, so a PAUSED run does not block the next occurrence. While run 1 sat paused in project A, a pin move plus one later fire rewrote the schedule row to B — and the resume policy then validated B while run 1's conversation was still filed under A. Delete A and the continuation was admitted into a conversation that had just been unscoped. That window lasts as long as the pause, not the sub-second race the previous commit documented. The reservation now records the destination THIS occurrence used, and `isScheduleLive` validates that record when given the occurrence's `scheduledFor` — which `resume.js` already reads two lines above the call. No new conflict type and no settlement-path surgery: the refusal rides the abort-and-settle branch that check already has. An ABSENT record falls back to the schedule-level resolution rather than reading as unscoped. A pre-scope occurrence, or one whose row is gone, must never be treated as evidence to stop a run — the fallback keeps legacy paused runs behaving exactly as they do today. Schedule-wide convergence stays: it keeps the row honest for the UI and for every check that has no occurrence in hand. FOLLOWUPS.md now describes the one remaining gap accurately — a deletion inside the claim window, which needs a distinct policy conflict routed through abort-and-settle rather than the bare 409 an `inactive` conflict produces. Tests: three on occurrence-vs-row precedence (the decisive one verified failing without the lookup), two on what the reservation records, and the resume controller spec now pins `scheduledFor` in the policy call. Schedules e2e re-run green. * 🏷️ fix: Tell a Deliberately Unscoped Occurrence From an Unrecorded One The occurrence fallback added in the previous commit conflated two different absences. A post-upgrade run that deliberately went unscoped omitted the field exactly like a row written before the field existed, so both took the fallback — and a paused unscoped run was then validated against the schedule's CURRENT project. Under a requirement or a pin added while it sat paused, that admitted a billed continuation into a conversation satisfying no present policy. The reservation now ALWAYS records its decision, `null` for unscoped, and the read reports `recorded` from key PRESENCE rather than truthiness. Only an unknown record — a pre-scope row, or no row at all — falls back to the schedule-level resolution; a recorded null is the genuinely unscoped occurrence it says it is, and is refused once a project becomes required. The distinction rests entirely on a stored `null` surviving as a present key while a never-written field stays absent, so that is asserted against real Mongo rather than assumed: if it ever stopped holding, unscoped runs would silently start being validated against the schedule's current project again. Tests: two against mongodb-memory-server (recorded null vs never-written vs missing row), plus refusal of a recorded-unscoped occurrence under a new requirement and the preserved fallback for a pre-scope one. Schedules e2e green. * 🔁 fix: Validate an Initial Scheduled Start Against Its Own Occurrence The initial-start policy check in `request.js` called `isScheduleLive(..., { policy: true })` without `scheduledFor`, so it fell back to the schedule-level resolution even though the run row already carries the occurrence's recorded scope. An occurrence reserved unscoped, with a pin introduced while its loopback request sat queued, was therefore admitted against the new pin — producing a billed unscoped conversation under a requirement it does not satisfy, from an envelope already built without a project. `scheduledFor` was already in scope there. Passing it makes the initial start and the resume validate the same way: against the destination the occurrence itself recorded.	2026-08-21 11:09:04 -04:00
Danny Avila	c7e355b219	🛑 fix: Confirm Scheduled Stops and Separate Terminal Scheduler Failure from Startup (#15051) ... * 🛑 fix: Confirm Scheduled Stops and Separate Terminal Scheduler Failure from Startup Fixes #15042, fixes #15043. `resume.js` inferred a confirmed stop from the ABSENCE of `failureReason`, but `abortJob` had four `success: false` paths that returned no reason at all. Those settled the occurrence as `interrupted` and pruned the checkpoint on aborts that never landed — including one where a REPLACEMENT generation owned the conversation, which pruned the successor's checkpoint. Every `success: false` return now names itself (`job_not_found`, `already_settled` added alongside the existing `generation_replaced` / `job_still_active`), and a single canonical `isStopConfirmed` predicate decides whether durable state may be settled. `already_settled` confirms a stop — `awaitProviderDrain` has proven the provider segment can no longer persist — so a permanently terminal generation is not answered with a retry loop. Separately, a schedule engine that failed to arm advertised its permanent outage as a transient 503 with `Retry-After`, so a client obeying it would poll forever. Readiness is now tri-state (`starting` / `armed` / `unavailable`): the retry contract applies only while arming is genuinely pending, and a failed arm returns a terminal `SCHEDULES_UNAVAILABLE` with no `Retry-After` and an error-level log. * 🏷️ fix: Declare Schedule Write Gate Return Types `--isolatedDeclarations` requires an explicit return type on the exported factory and on the middleware it returns (TS9007). Adds a named `ScheduleWriteGate` type matching the existing `ShareMiddleware` shape.	2026-08-20 18:33:51 -04:00
Danny Avila	c5276fc63d	⏱️ feat: Run Scheduled Chats Through Durable Agent Triggers (#14939) ... * feat: Scheduled Chats — agent-centric scheduled runs creating real conversations Squash of the full review-hardened branch (PR 14540, supersedes 14373) onto latest dev, preserving the exact verified tree. History prior to this commit lived on the pre-squash branch; every invariant below survived 25 Codex review rounds plus two external audit rounds (R26) with regression tests that fail without their fixes. Feature: - Schedules CRUD + side-panel UI (cadence dialog, run cards, Run Now), roles/ permissions (SCHEDULES:USE), interface.schedules availability, per-user limits and capacity slots, timezone-aware cadence with DST-conservative floors and misfire grace. - Engine: single-process claim/fire loop with leases, loopback POST dispatch (signed schedule-fire JWT claims, per-occurrence idempotency key inside the route's clientRequestId charset), overlap/balance/capacity/duplicate skip policies with auto-disable streaks (too_many_failures, insufficient_balance), reconciliation from retained terminal-job evidence, erasure sweep. - Scheduled runs create real conversations through the resumable agents chat path: HITL pauses surface on the card (requires_action), resumes re-apply the fire boundary's admission policy (revision fence, enabled, global kill switch, SCHEDULES:USE, availability) before continuing a billed generation. Correctness invariants (the audit surface): - Settlement discipline: every persistence-producing write happens-before a run's terminal outcome write; Stop/complete/pause race through single-winner terminal CAS claims (dev's TerminalJobClaim substrate) with retained, completedAt-less evidence for scheduled fires plus an owner-intended outcome stamp (scheduleOutcome) the reconciler prefers over re-derived success — round-tripped through the Redis hash mapper. - Swallowed generation failures (client error content parts) classify to error/skipped_balance instead of success on both initial and resumed paths; stale stamps are refreshed evidence-first when persistence plus the Mongo outcome write both fail. - Abort honesty: delivery judged by generation ownership and the CAS's actual from-status; republication escalates to the transport's acknowledged variant; the Stop route settles only genuinely paused runs. - Account deletion: one-way barrier (deletionRequestedAt) with auth-cache tombstone-before-stamp, boundary rechecks across all auth strategies, quiesce of scheduled + interactive work with durable per-stream abort fences (positive-evidence acknowledgement only), owner-side finalization markers for post-terminal billed writes, deferred-deletion sweep (explicitly ensured partial index) that completes cascades autonomously. Remote OpenAI-compatible/ Responses requests are documented as outside the quiesce and tracked in issue 14594. - Store compatibility: finalization markers optional on the legacy IJobStore contract with coherent degradation (registration fails -> synchronous title fallback; count reads 0). * fix: fence terminal response persistence from deletion; retain stale-pause evidence Two blockers from the third external review round. Terminal persistence visible to account deletion: - The finalization marker was registered only when post-terminal TITLE work was possible, but every persistence-owning terminal CAS opens the same window: the claim drops the job out of the active set BEFORE the response save (and the background user-message/convo saves), so a deletion quiesce landing there saw neither an active job nor a marker and could cascade while the admitted request could still recreate messages. Both controllers now register the marker before every persistence-owning claim — the fresh-turn path and the HITL resume — and release it only after their pending saves (and any post-terminal title) have landed, on success and failure paths alike. The TTL bounds a crash. - settleAbortFence no longer clears a complete/error fence while `terminalPersistencePending` is set: terminal at the CAS is not settled while the owner is still persisting. The job facade now surfaces the flag. - The marker trio is REQUIRED by the runtime store contract (assertJobStoreV2 refuses a store without it at configure time, keeping the failure loud and deterministic) while remaining optional on the legacy public IJobStore type for source compatibility. The silent degrade path from the previous round is gone — it was not deletion-safe. Stale-pause recovery retains scheduled evidence: - Three crash/timeout recovery paths — ApprovalLifecycle.failStalePausePersistence and the InMemory/Redis stale-pause cleanups — unconditionally stamped `completedAt`, putting a scheduled fire's failed-pause error terminal on the short completed TTL. A Mongo outage longer than that TTL erased the evidence and the reconciler recovered the run as `interrupted` instead of `error`. All three now follow the controller-observed path from the previous round: scheduled jobs omit `completedAt` (retained-evidence TTL) and stamp the error outcome. The PR description now explicitly narrows the deletion guarantee for the remote OpenAI-compatible/Responses paths (tracked in issue 14594). * fix: deletion-fence marker protocol — generation-scoped, atomic, fail-closed, all terminal paths One consolidated pass over the finalization-marker mechanism, per the fourth external review round. The invariant it establishes: NO persistence-owning terminal CAS runs without a durable, generation-qualified marker covering the window it opens, and every consumer treats a pending terminal as unsettled. - Generation-scoped markers. Entries were keyed (userId, streamId), so a Stop-superseded generation finishing late could clear the marker its replacement registered on the same conversation. Marker fields are now qualified by the generation's createdAt; clears must present the same identity, and an unqualified legacy clear cannot drop a qualified entry. - Atomic Redis registration. HSET-then-EXPIRE loses the fresh marker when the user's existing hash expires between the two commands (or the process dies there); registration is now a single Lua script carrying both. - Fail closed everywhere. Registration failure (after one retry) now REFUSES the terminal CAS instead of proceeding uncovered: the completion claim throws into the error path, the error path skips completeJob and leaves the job ACTIVE — deletion-visible by itself, recovered by the stale-running reaper — and abortJob returns a new retryable `fence_unavailable` failure the Stop route answers with 503 and the deletion quiesce treats as an unacknowledged stop (fence kept). The previous round's log-and-proceed is gone. - Every terminal path enrolled. abortJob now owns its window (register before the abort CAS, clear in its finally — the Stop route's checkpoint prune and partial save run inside beforePublish, between CAS and publication); the interactive and background generation-error paths register before their completeJob; the resume controller's error finalization registers before its completeJob. A lost or thrown claim releases the marker after pending saves flush instead of holding the user's deletion behind the TTL. - Every pending terminal unsettled. settleAbortFence defers on terminalPersistencePending for ALL statuses — including `aborted`, whose route-side persistence the previous guard missed. Also: a direct Redis regression for scheduled stale-pause retention (the P2 test gap), and the PR description no longer claims to carry every commit. * fix: lease-token lifecycle fences — same-generation isolation, admission fence, undelivered-Stop retention, legacy abort enrollment Fifth external review round; four P1s, handled as the requested consolidated lifecycle-fence pass. - Lease tokens. Marker fields were (streamId, createdAt), shared by every contender on the same generation — completion and Stop, or two racing Stops — so a losing contender's cleanup erased the winner's still-live marker. Every registrant now carries a unique lease token in the field and may only ever clear its own lease; unqualified legacy clears cannot touch qualified entries. - Admission fence. Authentication can pass before the deletion barrier goes up, and the durable createJob is several async steps later — a deletion quiesce in that window saw neither an active job nor a marker and could cascade before the admitted request created its job. The controller now registers an admission lease and THEN rereads the deletion barrier: the ordering guarantees either this request observes the barrier (403, lease released, slot/claim cleanup) or the quiesce observes the lease and defers. Held until createJob is durable; released on every refusal and initialization-error path. Fail closed when the lease itself cannot be registered (503 retryable). - Undelivered-Stop retention. abortJob released its lease in a finally even when delivery AND publication had provably failed — the job reads terminal (invisible to active-set scans), a user Stop writes no durable abort fence, and the remote owner keeps generating and will persist its abort-catch writes whenever the signal finally lands. The lease is now retained in exactly that case, and each resignal attempt heartbeats a fresh lease so the fence outlives the TTL for as long as delivery is still being driven. The abort-winning turn's own loser-side pending saves are additionally fenced in the controller catch (best-effort — those writes are already in flight). - Legacy abort enrollment. abortMiddleware (assistants abort route fallback for non-assistants endpoints) awaited abortJob and then spent usage and saved the stopped response AFTER the abort's lease was released. Both writes now run inside `beforePublish`, between the abort CAS and publication, covered by the same lease as every other abort. Barrier tests, each verified to fail without its fix: same-generation lease isolation (store), racing two-Stop loser cleanup (manager, stale-read forced CAS race), undelivered-Stop lease retention, resignal heartbeat, admission refusal with lease-before-reread ordering plus release-on-durable-create, and legacy-abort persistence inside beforePublish. * fix: heartbeat-backed owner-lifecycle leases close the settlement handoff races Sixth external review round: the remaining P1 interleavings were one structural problem — lease handoffs that were not atomic — resolved as the requested consolidated lifecycle-lease pass. - Quiesce reads leases BEFORE the active-job scan. The admission-lease -> durable -job handoff is only atomic against a reader in the OPPOSITE order of the writer: writers hold the lease strictly until the job is active-set visible, so leases-first shows every interleaving either the lease or the job. Jobs-first allowed a request to create its job after the scan and release its lease before the count — hiding both, cascading, and letting the new generation persist into a deleted account. - The abort acknowledgement is fenced by the owner-lifecycle lease. Redis ACKed the moment the owner's AbortController tripped; the stopping side released its lease on that ACK while the owner's asynchronous abort-catch persistence was still ahead. The transport now awaits a manager-installed pre-ACK hook that registers a DETERMINISTIC owner lease (exactly one owner exists per generation, and determinism is what lets the signal-time registrant and the owner's catch-side release agree across processes) before the acknowledgement is persisted or published; an owned same-replica abort bridges to the same lease before tripping its local controller. Both generation-owner catches (fresh turn, resume) release it once their writes land. - A failed replacement handoff no longer orphans the predecessor. The atomic replacement removes it from active storage, and an unconfirmed handoff terminalizes the replacement too — leaving nothing a quiesce could discover while the predecessor's provider may still be generating. Its owner lease is now retained at the point the receipt fails delivery; the owner replica renews it through the pre-ACK fence when the signal finally lands. - Leases HEARTBEAT while held. The five-minute store TTL only bounds a crashed holder; live persistence — a stalled save, a long deferred title — must never outlive its own fence. holdUserFinalization registers and renews every minute until released; the controllers' completion/error/admission leases all hold. The undelivered-Stop retention moved to a deterministic `stop` lease that every resignal attempt renews and the first successful one clears (no more opaque leases accumulating to TTL), and a THROWN abort transition releases the contender lease instead of leaking it. - The user-document abort-fence mutations now invalidate the auth user-doc cache, matching every other user-doc write. Barrier tests, each verified to fail without its fix: quiesce lease-scan ordering (plus the observed-lease defer), pre-ACK fence ordering at the transport, owned-abort owner-lease bridging, replacement-handoff predecessor retention, held-lease heartbeat past the TTL, and failed-then-successful resignal reaping the retained stop lease. * fix: one manager-owned owner-lease span across every abort delivery path Seventh external review round; four lifecycle-fence gaps, closed by making the owner-lifecycle lease a single manager-owned, heartbeat-held span. - Fail-closed acknowledgements. The pre-ACK hook registered a one-shot lease and the transport ACKed even when it failed; a same-replica owned abort likewise swallowed registration failure. The hook now acquires a HELD owner lease (heartbeat until the owner's catch releases it via releaseOwnerLease) and a rejection SUPPRESSES the acknowledgement — the stopping side stays retryable behind its retention lease, and every resignal re-drives the handler. The hook also stopped gating on `job.createdAt === generationId`: during a replacement handoff the store holds the replacement while the abort targets the predecessor, and that gate silently skipped exactly the generation being acknowledged (the store job is owner identity, never a generation gate). A local owned abort acquires the same held lease before tripping its provider; post-CAS the trip cannot be withheld, so acquisition failure downgrades delivery and the retention handoff keeps the user fenced. - Committed-but-lost-reply disambiguation. A thrown abort transition released the contender lease as if nothing had happened, but a Lua CAS can commit and lose its reply — an aborted job invisible to active-set scans whose provider was never signalled, with no fence left. The throw path now re-reads the exact generation: only a job still live under the caller's identity proves no commit; committed or ambiguous outcomes hand the fence to the deterministic stop lease (kept on the contender lease if even that fails) before rethrowing. - Replacement handoff covered end to end. A LOCAL replacement abort acquires the predecessor's held owner lease before the trip (failure reports the receipt undelivered, engaging retention). Failed-handoff retention is no longer a swallowed one-shot: it heartbeats with the durable acknowledgement proof as its renewal predicate — acquisition failures keep retrying for as long as the fence is needed, and the retainer stands down (without clearing the shared field) once the remote owner ACKs and thereby holds its own lease. - A LOCAL resignal delivery hands off to the owner lease BEFORE clearing the retained stop lease, and keeps the stop lease when that handoff fails. Barrier tests: hook rejection suppressing the ACK, commit-then-lost-reply retention with its provably-uncommitted counterpart, local-resignal owner handoff ordering, pre-ACK owner lease held past the store TTL until release (and provably stopped after), and failed-handoff retention retrying on its heartbeat — verified fail-before/pass-after by stashing the fixes. * fix: finish scheduled chat lifecycle hardening * fix: close scheduled chat review follow-ups * fix: generation-fence abort recovery evidence * test: wait for settled approval tool output * test: preserve scheduled init reconciliation option * refactor: rebuild scheduled chats on durable agent triggers * test: reset MCP cache mock between cases * test: isolate scheduler startup in server specs * fix: harden scheduled run lifecycle * fix: normalize schedule capacity conflicts * test: type schedule collision fixture * fix: re-fence scheduled resume and expiry * fix: fence scheduled resume handoffs * fix: release superseded manual schedule leases * fix: release failed run-now claims * fix: release superseded engine claims * fix: repair schedule dialog interaction and rework its form The agent picker was unusable: ControlCombobox portals its popover to the body by default, which lands it outside the dialog's Radix focus trap. Clicks passed through it, it could not be tabbed into, and the trap fighting Ariakit for focus locked the page up on selection. The prop is documented for exactly this case — pass `portal={false}` and give the dialog `overflow-visible`, as ProjectButton already does. The time and day dropdowns defaulted the same way. Alongside that: - Extract the agent builder's instructions editor (special-variable menu plus expand-to-fullscreen) into a controlled `VariableEditor` and use it for the schedule prompt. Insertions now route through `onChange`, so react-hook-form sees them — the schedule PATCH is built from `dirtyFields`, and a `setValue` that skipped dirty tracking would have dropped an inserted variable silently. - Replace the hand-rolled frequency buttons with the shared `Radio`. They marked the selection with `bg-surface-hover` on an outline button whose hover is the same token, so the selected option was indistinguishable from a hovered one; `Radio` is also a real radiogroup rather than four `aria-pressed` toggles. - Wrap the fields in a real `<form>` and associate the footer button by id, so Enter submits. Group the frequency, day and time controls in fieldsets. - Add placeholders for name and prompt, match the textarea fill to the other fields, and label the hourly case as minutes past the hour. - Widen the dialog to `md:max-w-3xl` and pair name with agent so the form fits without scrolling on desktop. - Move scheduled chats below skills and above prompts in the side nav. The new dialog spec fails when the portal fix is reverted. * test: cover scheduled and subagent deletion drains * refactor: own the form-control appearance in the client primitives Addresses the codex finding on ScheduleDialog: a feature-local `FIELD_CLASS` restated the `Input` primitive's border, radius, height and background so it could be pasted onto the schedule dropdowns, leaving those controls with no connection to the primitive they were imitating. Move that appearance into `packages/client/src/components/Field.ts` as the single source `Input` and `Textarea` now compose, and give `Dropdown` and `ControlCombobox` a `variant="field"` that applies it. The schedule dialog passes the variant and carries no class strings of its own. This also repairs a break the dev merge would otherwise have introduced: the newer `Dropdown` splits `className` (wrapper) from `triggerClassName`, so the old pasted classes would have landed on the wrapper and left the triggers unstyled. The semantic-token guard now watches the shared module and asserts each primitive still composes it, which covers more than the two files it read before. * fix: keep an explicit schedules disable from becoming an opt-in `use` is two things at once for a dual-purpose runtime interface field: a permission bit, which DB overrides strip, and the runtime disable signal that `getLimits` reads. Stripping it from `{ use: false, maxPerUser: 2 }` leaves an object, and `getLimits` treats any object without `use: false` as enabled — so an admin override written to stop scheduled billing for a role or user started it instead. Collapse an explicit disable to the boolean form before the strip, on both paths that accept it: the `interface.schedules` field patch, which admitted the object wholesale because bare runtime paths deliberately bypass the permission gate, and the overrides merge, which reached the composite-field branch and kept `maxPerUser`. Objects that only narrow limits are untouched, so a principal can still be given a smaller cap. Both regressions fail without the normalizer. * fix(schedules): Wave A — null-balance CAS, atomic paused-card clear, clustered erasure sweep Slice 1 (thread r3804518381): route the existing-null balance initialization through a { user, tokenCredits: null } compare-and-set instead of a blind $set, so a concurrent initializer/charge landing between the preflight read and the write is never handed back its spent starting balance. On a CAS miss the preflight re-reads the winner. The absent-record $setOnInsert path and the credited-record refill-config sync are unchanged. Adds the initializeNullBalance adapter (no upsert) and regression coverage for winner/miss/sync cases. Slice 2 (thread r3804518388): updateScheduleById now drops a `requires_action` lastRun projection atomically with the configRevision bump. Any pause present at edit time was projected under the pre-edit revision and can never be replaced by its own revision-fenced terminal outcome — a disabling edit would strand the card on "Needs approval" forever. Implemented as classic-operator CAS branches (DocumentDB rules out a conditional pipeline $unset), fenced on the card STILL being the pause so a terminal outcome or newer occurrence that races in is preserved. Terminal history survives untouched. Slice 4 (thread r3803826204): expose initializeScheduleErasureSweep from the schedule runtime facade and start it in every clustered (experimental) worker after Mongo is up. It re-drives eraseScheduleIfDrained for soft-deleted rows so a hidden prompt cannot outlive its drain when the delete/erase-on-settle attempts miss. It arms nothing else and never infers owner death from a process-local missing job (isTopologySafeToArm gates that). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq * fix(schedules): Wave B.1 — reversible account-deletion schedule suspension Thread r3804518383. Account-deletion quiesce marked every schedule `deleting`, disabled it, and cleared nextRunAt destructively. When a later cascade step (or the drain itself) failed, the controller cancelled the user-deletion fence — restoring the user — but the schedules stayed `deleting` and were erased by the sweep, silently losing all of a live user's scheduled prompts. Replace the destructive marking with a REVERSIBLE, token-fenced suspension: - suspendUserSchedulesForDeletion(userId, token) snapshots each schedule's prior enabled/nextRunAt under a per-attempt token, then fences firing (disable, clear nextRunAt, rotate claimToken). It never sets `deleting`, so a suspended row is not erasure-eligible. Snapshotting reads then bulkWrites (a classic update cannot copy field values under DocumentDB), fenced per row so an already- suspended/soft-deleted/edited row is left alone; idempotent per token. - restoreUserSchedulesFromDeletion(userId, token) reverses it, re-enabling and re-arming only rows still carrying the exact attempt token and not independently deleted — so an owner-deleted or newer-attempt-suspended schedule is never resurrected. - deleteUserController generates the attempt token, passes it to quiesce, and on any failure that cancels the user-deletion fence restores the suspended rows. A successful deletion hard-deletes them (and their snapshots) via the existing cascade and never restores. Adds a `deletionSuspension` embedded field (excluded from the wire schedule), data-method regression tests (suspend/restore/fence/idempotency/no-resurrect), and controller tests (restore on drain-false and post-quiesce cascade failure, no restore on success). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq * fix(schedules): Wave B.2 — wire the interactive Stop persistence protocol Thread r3804255932. The durable Stop primitives (requestRunAbort 'stop', getScheduleRunAbortState, markRunAbortPersisted) already existed but production only used requestRunAbort(..., 'deletion'). An interactive Stop flipped the job to `aborted` and then persisted its partial message + checkpoint inside `beforePublish`, without ever stamping the schedule Stop or acknowledging it — so reconciliation, the generation owner, or a concurrent schedule/account deletion could terminalize the run and release its capacity (and erase data) mid-write. Wire the request -> persist -> acknowledge -> settle barrier through the schedule runtime (the route never touches raw Mongo): - Expose beginScheduledStop / acknowledgeScheduledStopPersistence on the service. - The abort route stamps the Stop BEFORE signalling abortJob (a serialized 'in_progress' loser returns 409 STOP_IN_PROGRESS without a second abort), acknowledges only after beforePublish persistence succeeds, and on a persistence failure leaves the barrier unresolved so the run stays preserved (client retries; stale-owner timeout is the bounded recovery). A failed abort releases the stamp it placed so a replacement/retry is never blocked through its predecessor. - recordScheduleOutcome (the owner settlement path) now waits, bounded, for the Stop acknowledgement before terminalizing; a resolved/non-stop/stale marker proceeds immediately. The paused Stop settles only after its own ack. Adds service-layer barrier tests and route-level ordering/persistence-failure/ in-progress tests; the data-layer serialization is already covered. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq * fix(schedules): Wave C.1 — reconcile durable trigger delivery with the reservation Threads r3803826192 (manual limiter) and r3804255924 (PII/moderation), plus the independently-found long-Retry-After race. fireSchedule reserves a `started` run and a global capacity slot BEFORE the durable trigger delivery reaches the chat route, where an interactive limiter (manual Run Now), PII, or moderation can reject it before any generation job exists — dead-lettering the delivery while the run sat `started` until the 30-minute orphan sweep mislabeled it interrupted. And a valid delivery deferred by Retry-After (up to 24h) could be orphan-settled and have its capacity released, then fire anyway. Translate durable delivery state into the schedule outcome: - Store the deterministic trigger deliveryKey on the ScheduleRun reservation (computed from the envelope BEFORE enqueue, so an ambiguous commit still has it). - Add a getTriggerDelivery engine dep (wired to the merged trigger service's getDelivery) that reads the durable delivery by key. - Schedule reconciliation, for a jobless `started` run: staging/pending/leased → admission is live, never orphan; dead → record `error` from the durable lastError and release capacity promptly (no 30-minute wait), through the ordinary outcome/auto-disable path; succeeded or no record → the existing legacy orphan policy (interrupted only past the cutoff); a delivery lookup failure defers rather than orphaning a possibly-live delivery. Limiter/PII/ moderation middleware writes no schedule state. Adds reconcile state-mapping tests (dead/pending/leased/staging/succeeded/none/ lookup-failure) and a fire test that the reservation's deliveryKey equals the enqueued delivery's idempotency key. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq * fix(stream): Wave C.2 — durable retry/ack for terminal host lifecycle actions Thread r3804518375. Approval expiry won the `requires_action → aborted` CAS and then invoked the host hook best-effort: `runApprovalExpiredHandler` swallowed a failure, and because later sweeps enumerate only `requires_action` jobs, the now- aborted job was never offered again. In the clustered entrypoint (no schedule reconciler) the ScheduleRun stayed `requires_action` and its retained job persisted indefinitely. Make the host lifecycle work durable rather than schedule-specific: - Add a generic `terminalHostActionPending` marker, set ATOMICALLY in the same terminal transition (ApprovalLifecycle.expireWithIdentity), only when a host adapter is installed. - Retain and index such jobs: both stores keep them out of terminal reaping and expose getTerminalHostActionJobs(); Redis adds a set + extended (24h-bounded) TTL, in-memory a bounded 24h retention so a permanently-failing hook cannot leak. - The manager clears the marker only after the adapter acknowledges success, fenced by generation identity (clearTerminalHostAction), so a replacement generation can neither clear nor execute its predecessor's action. - cleanup()/expireStaleApprovals() enumerates unacknowledged terminal host actions across restarts and replicas and retries the idempotent hook; the relay only re-invokes while the marker is unacknowledged, so a successful ack prevents duplicate work. Store-won expiry marks it too, so a loser-replica relay still crosses the hook. - Terminal SSE notification continues regardless of host-hook outcome. Covers in-memory behavior (retry after failure, restart/other-replica retry, ack prevents duplicates, identity fence, terminal notification on failure, no marker accumulation for non-scheduled jobs) and updates the Redis cluster-membership contract test for the new index. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq * style(schedules): satisfy import sorting in fire.ts and fire.spec.ts CI "Static checks" failed on IMPORT_SORT for the two files Wave C.1 added imports to (the AgentTriggerEnvelope type import and getAgentTriggerIdempotencyKey). Applied scripts/sort-imports.mts to exactly those files — imports-only reordering, no behavior change. Other files reported by a repo-wide check are pre-existing on dev and deliberately left untouched so this PR is not widened. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq * fix(schedules): repair the deletion CLI and order restore before the fence release Addresses three findings from the fresh Codex review. P1 — config/delete-user.js called methods.disableUserSchedulesForDeletion, which Wave B removed in favor of suspendUserSchedulesForDeletion. The file is @ts-nocheck and its spec mocked the removed name, so neither typecheck nor tests caught it; the real CLI would throw a TypeError before deleting anything and then only unwind the fence. The CLI now uses the tokenized protocol: it mints a suspension token, suspends with it, and restores that exact attempt's rows in its finally block when the deletion does not commit. Its spec mocks the real methods, so the breakage can no longer hide. P2 — both the HTTP controller and the CLI released the user-deletion fence BEFORE restoring schedules. That fence is what refuses new schedule writes/claims, so the gap let an owner PATCH edit a still-suspended row and have its enabled/next-run state overwritten by the older snapshot, and let a second deletion attempt re-suspend under a new token — making the first restore a no-op and stranding the disabled snapshot permanently. Restore now runs first, while writes are still fenced. Hardening for the same defect class across a crash: suspendUserSchedulesForDeletion now ADOPTS an existing suspension's snapshot when re-suspending a row abandoned by an earlier attempt, instead of re-capturing the row's current (already-suspended) state. Without this, an attempt that died before restoring would have its successor snapshot "disabled, no next run" and permanently strand the schedule. Tests: CLI restore-before-fence ordering and no-restore-on-success; the same ordering assertion on both controller post-quiesce failure paths; a data-method regression that a second attempt adopts the abandoned snapshot and restores the original enabled/next-run state. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq * fix(schedules): converge dead deliveries in topology-safe maintenance Codex finding: the `dead` delivery mapping added in Wave C.1 lives only inside startScheduleEngine's reconciler, but the clustered entrypoint arms no engine — it runs erasure-only maintenance. A delivery queued before a restart into clustered mode and then rejected before generation creation (interactive limiter, PII, moderation) dead-letters while its ScheduleRun stays `started`, holding a global capacity slot indefinitely for an ordinary non-deleting schedule. Add a dead-delivery convergence pass to the erasure sweep, so every topology that runs schedule maintenance settles it. The pass is POSITIVE-EVIDENCE-ONLY and is therefore safe where absence-based reconciliation is not: a `dead` delivery is durable shared state proving no generation owns the reservation. It settles only when the job is confirmed absent or identity-mismatched (an identity-matched job still owns the run), defers on an unknown job lookup, on an in-flight abort, and on an in-flight resume hand-off, ignores legacy reservations with no deliveryKey, and applies a short grace so an accepted delivery still creating its generation is never settled mid-handoff. Auto-disable policy is deliberately left to the armed engine; this path records the failure and frees the slot. Deliberately does NOT touch api/server/experimental.js — the clustered entrypoint already starts this sweep, so the convergence arrives through the existing initializer and the shared-file footprint stays as-is. Tests: settles a dead delivery as error under an explicitly UNSAFE topology, leaves live deliveries alone, never settles under an identity-matched running generation (delivery is not even consulted), defers an in-flight abort, and ignores a reservation with no deliveryKey. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq * fix(stream): refresh host-action retention on each retry attempt Codex finding: unacknowledged terminal host-action evidence was capped at the 24h pause TTL, so a host dependency (Mongo) unreachable for longer than that let the Redis key — and its pending marker — expire with no generation-fenced acknowledgement, stranding the ScheduleRun where no reconciler is armed. Measure retention from the LAST retry rather than from the terminal transition: enumerating a pending host action IS the retry attempt, so both stores refresh its retention as they hand it to the hook (Redis re-EXPIREs the job key; the in-memory store stamps terminalHostActionRefreshedAt and bounds from it). Evidence therefore survives as long as some replica is still actively retrying, while a deployment that stops sweeping entirely still lets it age out — so this does not reintroduce the unbounded leak the cap existed to prevent. Test: after a failed hook, a later cleanup pass keeps the marker pending and moves its retention basis forward. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq * fix(schedules): defer settlement when the Stop barrier times out Codex finding: waitForStopPersistence returned after its 5s poll budget even when the Stop was still fresh and unacknowledged, and recordScheduleOutcome then went straight on to terminalize the run — releasing its capacity, deletion, and erasure barriers while beforePublish may still have been writing. Slow checkpoint cleanup is indistinguishable from a dead route on that signal, so the timeout was being treated as if the barrier had been satisfied. The poll budget now means "undecided", not "clear". On timeout with a fresh, unacknowledged Stop the barrier DEFERS: recordScheduleOutcome returns false without recording, leaving the run active/preserved. Settlement then happens either when the route acknowledges, or once the existing stale-owner cutoff (ABORT_OWNER_PRESUMED_ALIVE_MS) authorizes a later attempt — which the loop already treats as clear-to-settle. Callers with durable retry (the approval-expiry host action, reconciliation) re-drive it, so a deferral converges rather than stranding the run. Test: a fresh Stop that never acknowledges within the budget reports not-settled and records no outcome. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq * fix(schedules): require definite delivery failure, extract its message, converge deferred Stops Third Codex round. Three findings in code from this closeout, plus one pre-existing P1 that is a one-line operator-safety fix. P1 — dead-delivery settlement demanded too little evidence. `dead` does not prove a request was rejected: the trigger host marks response timeouts and invalid success responses `certainty: 'ambiguous'`, and the engine dead-letters those once retries are exhausted. The erasure sweep treated every dead letter as positive evidence, so an ambiguous one sitting over a generation a peer had accepted could terminalize the run and release its capacity mid-flight. It now settles only on a DEFINITE rejection, unless job absence is deployment-authoritative (safe topology), where the confirmed-absent job is itself the evidence. P1 — `lastError` is an `AgentTriggerDeliveryFailure` object, not a string. A duplicated local interface declared it `string` (against CLAUDE.md's no-duplicate- types rule), so both the sweep and the engine reconciler passed the object into the String-typed run/schedule `error` fields; Mongoose would reject the cast, the per-row catch would swallow it, and the run would keep its global capacity slot. The dep type now reuses the canonical `AgentTriggerDeliveryFailure` and both call sites pass `.message`. Re-typing immediately surfaced a stale test that had asserted a string. P1 (pre-existing) — the base-config global stop is honored in `getLimits` via `isRuntimeDisabled` rather than a literal `=== false`. The stop has two shapes, and deepMerge turns base `{ use: false }` plus a principal override of `true` into `{ use: true }`, so the literal check reported the feature enabled and Run Now dispatched straight through fireSchedule, bypassing the operator's emergency stop. Now the same predicate the engine gate already uses. P2 — a Stop whose settlement DEFERRED past the poll budget had no convergence path where no reconciler is armed. `acknowledgeScheduledStopPersistence` now optionally re-drives the terminal outcome once the barrier clears; `recordRunOutcome` is match-guarded and idempotent, so an owner that already settled makes it a no-op. The abort route passes it for a running generation; a paused job still settles explicitly. Tests: ambiguous dead letters refused under unsafe topology but settled when absence is authoritative, definite rejections settled either way, the failure message carried through, and the abort route's re-drive present for running / absent for paused. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq * fix(schedules): converge terminal runs in clustered workers, retry suspension restore Fourth Codex round on 74f7af8. Both findings are in code from this closeout. P2 - a clustered worker never settled a run whose generation finished but whose outcome write failed. `recordScheduleOutcome` retries three times and then returns false; the owner honors that by PRESERVING the terminal job as the only surviving evidence, and the armed engine's reconciler replays exactly that. The clustered entrypoint arms no engine, and its sweep only covered deleting schedules (settleAbandonedRuns) and dead deliveries - an identity-matched job was skipped outright. So an ordinary live schedule's run stayed `started`, held its GLOBAL capacity slot, and kept a preserved job that carries no `completedAt` and is therefore invisible to the store's finished-job sweep: both leaked until store expiry. The live-schedule pass now also converges from a retained terminal job, mirroring the reconciler branch it stands in for: honor the owner's stamped outcome over the generic status (so a balance refusal still walks its streak rather than resetting it), clear the reserved conversationId when the generation never emitted its created event, and delete the retained job only AFTER the outcome write is durable. This stays positive-evidence-only and safe in every topology. Presence, not absence, is the evidence: an identity-matched job is authoritative wherever it is observed - a shared store shows the real generation, a process-local store can only be showing this process's own - which is why it needs no canInferOwnerDeathFromMissingJob fence, unlike the absence-based paths. The in-flight abort/resume fences still defer, so an `aborted` job cannot settle a run whose owner may still be persisting. Both cases now share one pass over one window rather than two, and `retainedOutcome` moved to types.ts so the sweep reuses the engine's mapping instead of duplicating it (and stays independent of the engine). P2 - a failed restore stranded a live account's schedules. Cancelling an account deletion restores the suspended rows while the deletion fence is still armed and then releases that fence; nothing re-drives the restore afterwards, so one transient write failure left the user with silently disabled, next-run-less schedules. The restore is now retried at the single choke point both the HTTP controller and the CLI share. Retrying is safe because each attempt re-reads only the rows STILL carrying the token: a partially-applied unordered write converges on exactly the stragglers, and a fully-applied one finds nothing. The fence is still released when every attempt fails, deliberately: retaining it would refuse the live account's schedule writes AND make beginAgentTriggerUserDeletion report `in_progress` forever, blocking the retry that is the convergence path (a later attempt adopts this snapshot, so its cancel restores these exact rows). Both callers now log the user id and suspension token so the state stays recoverable by hand if that never happens. Tests: retained terminal job settled and its evidence released, stamped outcome preferred over the generic status, stamped failure reason carried, reserved conversationId cleared for a never-created conversation, identity-mismatched terminal job ignored, abort-in-flight deferred; restore retried past a transient write failure and converging on a partially applied restore. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq * fix(schedules): converge unprojected pauses and release replayed bookkeeping jobs Two of the four findings I flagged as pre-existing to my closeout commits. Both are in code this PR introduces, so merging would have shipped them; both are the same capacity/evidence-leak class the last four rounds have been closing. The remaining two (slotless rows escaping the per-user cap, and the non-rotating `started` reconciliation bucket) are genuinely latent and deliberately left alone. An UNPROJECTED PAUSE held a global capacity slot forever. The pause projection is what moves a run row off `started`; `recordScheduleOutcome` already retries it, but the request controller discarded the result, so three failed attempts were dropped silently. The armed engine's reconciler replays that state, but the clustered sweep did not: a paused job is not terminal, so the retained-job path returned without settling, and the dead-delivery path never inspects an identity-matched job at all. The row stayed `started` with no cutoff that would ever clear it. The call site now surfaces the failure, and the sweep converges it, mirroring the reconciler's pause branch: project `requires_action` (which frees the slot) but do NOT release the job's evidence — unlike a terminal job it is still live, awaiting an approval. The resume hand-off fence still defers, so re-projecting cannot release a slot a continuation just claimed. The BOOKKEEPING REPLAY pass leaked its retained job. A run reaches that pass only because its owner crashed before bookkeeping — which is also before it could release the job it retained for exactly this recovery. The active-run pass clears its own; this one never did, and a preserved job is deliberately kept WITHOUT `completedAt` so the store's finished-job sweep cannot reap it early, so nothing else ever would. It now clears after `finalizeBookkeeping` succeeds — identity-guarded, a no-op when no job is retained, and skipped entirely when the replay itself failed, since the retained job is the only surviving evidence in that case. Tests: pause projected and the slot freed while the live job's evidence is kept, paused job deferred during a resume hand-off, retained job released once replayed bookkeeping is durable, and retained job kept when the replay fails. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq * fix(schedules): fence the clustered pause replay against a concurrent resume claim Codex round 5, on code I added in f735341. The finding is correct and the exposure is one I introduced. The pause replay lives in a sweep that runs in EVERY clustered replica, so several sweepers can observe the same unprojected pause. Its guards were all derived from an in-memory row SNAPSHOT: hasResumeHandoffInFlight reads the snapshot's resumeClaimedAt, and recordRunOutcome's pause branch matched any row currently in `started`/`requires_action` with no fence of its own. The race: sweeper A projects the pause and frees the slot; the owner's approval then claims a fresh one (markRunResumeClaimed takes the row to `started` WITH resumeClaimedAt in one write); sweeper B, still holding the pre-projection snapshot, passes its hand-off check and replays — `$unset: { capacitySlot, resumeClaimedAt }` under a continuation that is already running. The run reverts to `requires_action` while its generation proceeds outside global capacity. The engine's reconciler makes the same call and has the same snapshot-derived guard, but v1 arms exactly one engine, so it has no concurrent racer; the sweep is the first thing to run this transition in parallel. Left the engine alone rather than widening the change: its `requires_action` re-affirmation is deliberate and single-writer. Fixed where the race is, in the write itself. `recordRunOutcome` takes an optional `requireNoResumeClaim`, which adds `resumeClaimedAt: { $exists: false }` to the pause filter, and the sweep sets it. Because the stamp is written in the SAME update that moves the row to `started`, its absence is atomic proof no resume owns the row. The flag is deliberately NOT set by the generation owner: its own re-pause legitimately clears the stamp as the hand-off's completion signal. The fence cannot block the recovery it exists to enable: markRunResumeClaimed only matches `requires_action`, so a genuinely stuck `started` row can carry no resume claim — it is in fact blocking its own approval until this replay frees it. Tests: a replay from a stale snapshot leaves a resume-claimed row's status, slot, and claim stamp intact, while a stuck row with no claim still recovers; and the sweep is asserted to send the fence. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq * fix(schedules): fence stale resume claims by age, not existence Codex round 6, on the fence I added in ee00c60. Correct again, and the defect is the mirror image of the one it fixed. The fence was an EXISTENCE check (`resumeClaimedAt: { $exists: false }`) while its caller's guard is a FRESHNESS check (hasResumeHandoffInFlight, bounded by RESUME_HANDOFF_STALE_MS). They agree while a claim is fresh and disagree once it is abandoned: a worker that dies after markRunResumeClaimed takes the row to `started` and stamps resumeClaimedAt — but before the continuation resumes or releaseRunResumeClaim rolls it back — leaves the stamp set forever. Past the bound the sweep correctly stops deferring and tries to recover the row, but the write rejected it purely because the field still existed. The row stayed `started` holding its global capacity slot, and its approval was unresumable for good, since markRunResumeClaimed only matches `requires_action`. That is exactly the stuck state this replay exists to clear, so the fence had reintroduced it for the crashed-resume case. `requireNoResumeClaim: boolean` becomes `resumeClaimStaleBefore: Date`, and the filter matches a row with no claim OR a claim older than that cutoff. The sweep passes the SAME bound its in-flight check uses, so the two can no longer disagree. A genuinely racing claim is by construction fresh — it is created after the sweeper's snapshot — so the race from round 5 stays closed. Tests: a row whose resume claim was abandoned by a dead worker now recovers (status, slot and stamp all cleared), alongside the existing two — a fresh claim still repels a stale-snapshot replay, and an unclaimed stuck row still recovers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-20 11:51:30 -04:00
Danny Avila	5e3c680761	🪃 feat: Wake Parent Agents on Child Completion (#14975) ... * feat: wake parent agents on child completion * wip: harden child completion wakeup lifecycle * fix: close the completion-wakeup static failures Type the durable-claim store fixture, the continue-envelope test helper, and the terminal message's task metadata so the wakeup suites compile against the shapes they actually exercise. Replace `Array.prototype.at`, which the package target library does not provide. Capture the prepared child thread in a non-optional local before the provider callback closes over it, and narrow the trigger envelope itself on `mode === 'continue'` rather than a separately copied mode, so reading the continue target is sound. Lift the parent-message fallback out of a nested ternary into a named resolver. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H1cCMDrTWaRNkmtKjpWELZ * test: cover the active-predecessor admission fence The Redis job-creation call gained a thirteenth scalar argument, so the spec helper reconstructed the HSET pairs one slot early and rebuilt an invalid job hash; three creation tests failed on that alone. Give the fence itself direct coverage in both store adapters, which it had none of despite deciding whether an automatic continuation may replace a live parent turn. Each proves a running and a requires_action predecessor are refused with the state a controller needs for a finite 409, that an absent or settled predecessor is admitted, and that an ordinary user turn without the policy still replaces its predecessor. The Redis case also asserts a refused continuation leaves the parent's durable job and chunks untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H1cCMDrTWaRNkmtKjpWELZ * fix: harden completion wakeup rollout and claims * fix: close completion wakeup race windows * test: keep the child store fixture exact * fix: close final subagent wakeup gaps * fix: preserve ambiguous completion claims * fix: release pre-admission wakeup claims * fix: stabilize subagent completion recovery --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-19 12:18:31 -04:00
Danny Avila	e4d6bb71f9	📁 feat: Surface Stateful Workspace Downloads (#14984) ... * feat: surface stateful workspace downloads * fix: sort workspace change imports * fix: reuse workspace button primitives * fix: hide collapsed workspace actions	2026-08-18 22:05:13 -04:00
Dustin Healy	a33b128c47	🪪 fix: Preserve Stored Access Token Expiry Over ID Token Exp (#14982) ... * 🪪 fix: Preserve Stored Access Token Expiry Over ID Token Exp extractOpenIDTokenInfo let the ID token exp claim overwrite the token set's stored expires_at. The ID token is minted at login and never refreshed, so once a session outlives the ID token TTL, isOpenIDTokenValid reports the access token as expired even when expires_at is hours in the future, and OpenID placeholder substitution silently stops: MCP headers configured with {{LIBRECHAT_OPENID_ACCESS_TOKEN}} ship the literal placeholder string as the bearer credential and the receiving server rejects every connection with an unparseable JWT until the user fully logs out and back in. The ID token exp now only fills a missing expiresAt instead of overriding a stored one. Identity claim enrichment from the ID token is unchanged, and the exp fallback for token sets without expires_at is preserved. * 🪪 fix: Validate ID Token Expiry Before ID Token Placeholder Substitution The precedence fix made isOpenIDTokenValid track only the access token expiry, so an MCP header using {{LIBRECHAT_OPENID_ID_TOKEN}} could substitute an ID token that had already expired. The ID token exp is now preserved separately as idTokenExpiresAt and checked at the ID token substitution site, so an expired ID token substitutes empty rather than a stale credential while access token substitution is unaffected. * 🪪 fix: Address OpenID Expiry Review Round Fix expires_at at the source in the OpenID JWT strategy. The stored value described the incoming bearer's exp even when access_token came from the session or a cookie, so it could describe a different credential entirely. A new decodeJwtExpiry helper reads the exp of the token actually stored, and payload.exp is kept only when the raw bearer is the resolved access token. Opaque session or cookie tokens now store no expiry rather than a wrong one. Apply a 30 second clock skew buffer in isOpenIDTokenValid and isIdTokenCurrent via a new exported OPENID_EXPIRY_BUFFER_SECONDS, mirroring OPENID_REUSE_EXPIRY_BUFFER_SECONDS in AuthController. Tokens that would expire in transit are treated as already expired. Make isIdTokenCurrent fail closed when idTokenExpiresAt is absent. exp is REQUIRED in an ID token, so a missing value means the token is malformed or the claims parse threw. The check uses == null so an exp of 0 counts as present and therefore expired. Read the ID token exp with a numeric type check so an exp of 0 records idTokenExpiresAt and fails closed downstream while a non-numeric exp is ignored, and compare the stored expiry with != null so a gap filled expiry of 0 reads as expired instead of as no expiry at all. Raise an actionable re authentication error for the ID token placeholder instead of substituting an empty string. An empty substitution produced a malformed Authorization header and a 400 downstream rather than a clean signal that the user must re authenticate. Raise the same re authentication error from processSingleValue when a user has an OpenID identity, the stored token set is no longer valid, and the value still contains a credential bearing OpenID placeholder, so the expired access token case that motivated this PR signals re auth instead of silently shipping or stripping the placeholder. Only the access token, ID token, and generic token names raise: identity metadata resolves from the user document and an expiry hint never needed a token, so those keep their existing literal then strip behaviour. Unknown placeholder names also stay literal and diagnosable, matching the existing resolvable placeholder policy. Add the comments the review asked for on the exp fallback heuristic, the EXPIRES_AT placeholder semantics, why stale ID token claims stay usable for identity fields, and the advisory nature of the freshness check. * 🪪 fix: Honour Opaque Access Tokens And Type The OpenID Re-Auth Error Drop the ID token exp fallback in extractOpenIDTokenInfo. Storing the access token expiry honestly means an opaque access token now records no expiry, and the fallback then handed the ID token exp authority over a credential it does not describe. A deployment issuing opaque access tokens alongside a short lived ID token saw isOpenIDTokenValid go false and the credential guard reject a perfectly good access token, which worked before this branch. An unknown access token expiry is now treated as no expiry, and the ID token exp only ever gates ID token substitution through idTokenExpiresAt. Give the re-authentication signal a type. OpenIDReauthRequiredError is raised at both the ID token placeholder and the credential placeholder guard, ErrorController maps it to a 401 carrying the actionable message, and the class exposes statusCode so the agent generation path answers 401 instead of a bare 500 for the same condition. Omit rather than blank a header whose credential placeholder is still unresolved on a final resolution pass, since an empty bearer credential is malformed under RFC 6750 while an absent header lets the upstream answer its own challenge. Identity placeholders keep stripping to an empty string. Move the resolvable placeholder docblock onto the pattern it describes, resolve an EXPIRES_AT of 0 as the string 0 for consistency with the neighbouring null checks, and let AuthController consume the exported OPENID_EXPIRY_BUFFER_SECONDS so the 30 second skew allowance has a single definition.	2026-08-18 22:02:33 -04:00
Ravi Kumar L	da0491d5db	💻 fix(agents): require Code Interpreter for programmatic MCP tools (#14977) ... * fix(agents): require code interpreter for programmatic MCP tools * test(data-provider): fix tool options fixture type * fix(agents): address programmatic tool review feedback * fix(agents): avoid no-op update on version revert	2026-08-18 17:16:58 +02:00
Danny Avila	547bd8c4bf	🧵 feat: Persist View-Only Subagent Threads (#14957)	2026-08-18 07:41:42 -04:00
Danny Avila	7d62be2ad3	🕸️ feat: Run Saved Agent Teams as Subagents (#14944) ... * feat: Add graph subagent integration * style: Sort response usage test imports * fix: Preserve lazy graph runtime context * fix: Use isolated graph input helper * test: Align graph integration fixtures * fix: Preserve lazy graph runtime capabilities * fix: Bound lazy graph metadata preload * fix: Harden lazy graph resolution lifecycle * fix: Coalesce lazy graph member resolution * fix: Snapshot initialized graph members only * fix: Preserve lazy agent runtime context * fix: Preserve batched lazy context preparation * fix: Preserve graph member capability bounds * fix: reconcile graph subagents with execution profiles * style: align graph subagent types with formatter	2026-08-17 18:02:52 -04:00
Danny Avila	aa35cd42b1	📬 feat: Add Durable Agent Trigger Delivery (#14925) ... * feat: wire trusted agent trigger dispatch * feat: add durable agent trigger delivery * fix: annotate trigger envelope byte limit * test: isolate trigger startup in server specs * fix: fence trigger delivery during account deletion * test: isolate trigger service in user controller specs * fix: close trigger deletion admission race * fix: harden account deletion fences * fix: close durable trigger review gaps * fix: require offline stale-fence recovery * fix: type trigger lane sequence ids * fix: fence admin user deletion triggers * fix: make trigger deletion recovery durable * fix: harden offline user deletion * fix: serialize trigger lane publication * style: sort trigger delivery imports * fix: recover orphaned trigger publications * fix: preserve trigger recovery ordering * fix: fence trigger publication during purge * fix: defer remote trigger deletion fences * fix: close durable delivery cleanup races * fix: drain CLI generation owners before deletion	2026-08-17 09:25:08 -04:00
Danny Avila	f8f118ef29	🛰️ feat: Execute Generic Agent Trigger Deliveries (#14921) ... * feat: add generic agent trigger dispatch seam * refactor: harden trigger dispatch contract * fix: annotate envelope depth alias * style: sort trigger envelope imports * fix: reject unknown trigger dispatch modes * fix: reject unknown trigger envelope versions * refactor: validate complete trigger envelopes * feat: add agent trigger execution host * fix: enforce trigger delivery contracts * fix: harden trigger admission path * fix: finish trigger cancellation handling * fix: retry strict steer rollout gaps * fix: retry paused trigger steers * fix: parallelize trigger admission setup	2026-08-17 08:45:49 -04:00
Danny Avila	57ea1137f6	🛡️ feat: Let Admins Restrict Stateful Workspace Scopes (#14910) ... * feat: let admins restrict stateful workspace scopes * fix: enforce stateful scope policy across agent paths * fix: close stateful scope policy activation gaps	2026-08-17 01:29:19 -04:00
Danny Avila	7d850c308a	🧠 feat: Add Live Reasoning Labels (#14893) ... * feat: add live reasoning labels * fix: Stabilize reasoning label checks * fix: Address reasoning label review findings * chore: Bump Agents SDK for reasoning labels * fix: Reset reused reasoning step evidence * fix: Reconcile cleared reasoning labels * fix: Fence reasoning label resets * fix: Reset reasoning ownership before gap labels * fix: Preserve THINK type through label reset * test: Expect run-global reasoning revision	2026-08-16 18:11:55 -04:00
Danny Avila	fb8ae881cf	⏱️ feat: Show Run-Step Durations On Tool Cards (#14892) ... * ⏱️ feat: Show Run-Step Durations On Tool Cards Surfaces how long each tool call took, derived from the `closed_at` / `created_at` pair already carried by `on_run_step_closed` — the same event #14871 and #14873 use for the terminal status. No new event, no new SDK surface. The duration is stamped onto the content part at the same three sites as `runStepStatus`, so it survives a reload and a resumable reconnect rather than living only on the live React message: - `callbacks.js`, on the aggregated part before the event is forwarded - `RedisJobStore`, in the host-authored replay reconstruction branch - `useStepHandler`, on the live message Rendering lands in the shared `ProgressText`, which nine tool cards already use, rather than in each card: one place decides whether a duration is shown and how it reads, and the cards only forward the number. That keeps this from adding a tenth independent state derivation to a component family whose label/announcement/progress split is already the subject of AI-1810. The value is deliberately absent rather than zero whenever it would be a guess — no `created_at`, non-finite input, or a negative elapsed time from two clocks that disagree, which is now reachable because a step can be opened in one process and closed in another after a checkpoint resume. Sub-second durations are suppressed as noise, and it renders only on a settled, non-error card, where the slot is not already carrying the cancelled icon or the error suffix. For assistive technology the compact form (`3.5s`) is hidden and paired with a spoken equivalent ("took 3.5 seconds"), both inside the button, so the accessible name carries the duration without an `aria-live` region re-announcing it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🎨 style: Sort Imports In Touched Files The import-sort gate runs against the files a PR changes, so pre-existing drift in `ProgressText.tsx` and `RedisJobStore.ts` surfaced on this branch. Both were already unsorted on `dev`; this is the sorter's output, with no semantic change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🐛 fix: Accept Partial Timestamps In Run-Step Duration Helper `getReportableRunStepDurationMs` declared its parameter as `Pick<RunStepClosedEvent, 'created_at' | 'closed_at'>`, where `closed_at` is required. That contradicted the function's own purpose: every guard inside it exists precisely to handle stamps that may be missing. The Redis replay branch reconstructs closures from persisted JSON and holds nothing stronger than "might be a number", so it failed to typecheck against the narrower signature. Widened to an exported `RunStepTimestamps` shape with both stamps optional, rather than asserting at the call site — an assertion would move the decision about what is trustworthy somewhere it cannot be enforced, which is the thing the helper exists to centralize. Callers holding a fully-typed event still pass, since a required field satisfies an optional one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🐛 fix: Suppress Duration When Failure Arrives As errorSuffix Alone At every call site `error` carries cancellation while failure travels through `errorSuffix` with `error` false, so gating the duration on `!error` alone rendered "· 3.5s" beside "· failed" — and announced it. The gate now checks both terminal-failure channels. The original test pinned only the `error: true` path, which is why this survived; the failed-via-suffix path is now pinned separately, both the visible and the announced half. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🧩 refactor: Persist Raw Run-Step Durations, Threshold At Render Only The three stamp sites filtered through the 1-second reportability threshold before persisting, baking a presentation rule into stored data: a 900ms step stored nothing, making "fast" indistinguishable from "not derivable" and unrecoverable if the display rule ever changes. Stamp sites now persist the raw `getRunStepDurationMs` value — absent only when genuinely not derivable — and the renderer alone decides what is worth showing, which `ProgressText` already did. Rendering is unchanged. `getReportableRunStepDurationMs` is removed; it existed only to serve the write-time filter, and a test now pins that sub-threshold durations survive to storage. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🐛 fix: Suppress Duration On Backgrounded Bash And Code Cards A backgrounded call's run step closes when dispatch returns the handle, so the stamped duration is the dispatch time. Rendering it beside "Running/Finished in background" misstated a detached task's runtime as seconds — and violated the "settled card only" rule, since the card is still tracking the detached run. Scope is exactly the two cards that parse background handles. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🌍 fix: Format The Sub-10s Decimal For The Active Locale The fractional seconds value was interpolated as a raw JS number, which hardcodes the en-US decimal point into every language — "1.4s" where the locale writes "1,4 s" — and translators cannot fix a number formatted in code. The value is now formatted via Intl.NumberFormat with i18n.language, following MessageTimestamp's pattern of threading the language into the util; plural-key selection stays on the numeric value. A malformed language tag falls back to the plain number. Also documents the two accepted limits of the derivation, so they read as decisions rather than oversights: positive clock skew is undetectable from a single stamp pair, and the value is wall-clock elapsed, so a step held open across a suspension (checkpoint resume, HITL approval wait) includes that time. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🐛 fix: Persist A Durable `backgrounded` Marker Through Harvest; Localize Minute Digits Codex round 3, both findings confirmed. **Background origin survived only as transient state.** The dispatch handle in `tool_call.output` and the live status-marker attachment are both gone once the harvester patches the settled task's stdout over the handle — so the round-2 suppression (`backgroundHandle == null`) came back on after harvest or reload, showing dispatch time as the task's runtime. Following the same rule as e4bd15d (persist facts, decide at render): the harvest patch now stamps `backgrounded: true` onto the tool call in the same atomic write that erases the handle — on the heal path too, which re-applies over full-row saves that reverted the part. The cards gate on handle-or-marker; the dispatch duration itself stays stored. **Minute-branch digits bypassed locale formatting.** The seconds branch went through Intl.NumberFormat while minutes interpolated raw numbers, so Arabic/Persian locales flipped to ASCII digits above one minute. All interpolated values now flow through the (renamed) formatDurationValue; an ar-EG test pins the localized digits. data-schemas cannot be installed in this environment (same npm ci 403 as packages/api), so message.ts/harvest.ts are syntax-checked with resolution off and otherwise verified by review; CI runs their real typecheck and suites. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🧪 test: Assert The `markBackgrounded` Stamp In Harvest Expectations The successful-harvest test's exact `toHaveBeenCalledWith` object did not include the newly forwarded `markBackgrounded`, so the API suite would fail on it. All three harvest-call expectations now assert `markBackgrounded: true` — the exact-object one of necessity, the two `objectContaining` ones deliberately, since the durable stamp (on the best-effort file-failure path and the reapply heal alike) is now part of the behavior under test. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🎨 style: Wrap Harvest Spec Expectation Per Prettier Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-16 16:17:02 -04:00
Danny Avila	06bf324cf0	🛤️ feat: Per-Agent Code Execution Routing With Stateful Session Scopes (#14848) ... * feat: route code execution per agent profile * chore: sort execution profile imports * test: preserve stateful environment literal types * fix: isolate stateful code environments by user * fix: preserve per-agent code routing end to end * fix: route code priming by execution profile * fix: isolate code profile lifecycle state * fix: preserve mixed-profile code resources * fix: complete stateful skill routing	2026-08-16 09:42:15 -04:00
Danny Avila	eb3b353712	📡 fix: Publish App-Level MCP Tool Catalogs Without a Reserved Revision (#14858) ... * 📡 fix: Publish App-Level MCP Tool Catalogs Without a Reserved Revision Shared MCP servers advertised no tools to agents, so every turn failed with "configured to use MCP tools, but none are available" (#14857). `replaceAppServerTools` returned false whenever a publication carried no `publicationRevision`, but only `refreshChangedTools` reserves one. Every other app-level publisher — the first-connect snapshot, reinitialization, on-demand catalog reads, the retained-catalog restore — was silently dropped. The agent path fails closed on that drop: the skipped write returns null, so reinitialize yields no tools and the turn 503s. Startup hid it. `connectAppServers()` defers the initial refresh and calls `refreshToolList()` itself, which does reserve, so a boot that reaches its MCP servers looks healthy. Only a lazily created app connection — the server not yet up when LibreChat boots, a dropped connection, a cold cache — takes the unreserved path. `ConnectionsRepository` now reserves before its own `tools/list`, matching the list_changed path; a failed reservation publishes unordered rather than failing the connection. Publishers with no pre-fetch reservation point have already fetched by the time they reach the cache, so they take the next revision at write time instead of being discarded. `mergeAppTools` still publishes at revision 0 and stays deferential to a live catalog. * 📡 fix: Bind App Catalog Ordering to the Fetch That Produced It Addresses review feedback on the previous commit: allocating a revision at publish time lets a slow `tools/list` of an old catalog outrank a newer one that reserved after it started, and it would let the retained-catalog restore — which republishes deliberately pre-mutation data — outrank a live catalog. Ordering now travels with the data. `fetchToolsSnapshot` reserves before its first page and returns the ticket on the snapshot, so every app-level publisher reads the revision belonging to the read it is publishing rather than one allocated at an unrelated moment. `fetchOrderedToolsSnapshot` carries the refresh's revision when it defers to one, since that is whose catalog it returns. With the reservation at the single point where app-level tools are read, no publisher can forget it, so `replaceAppServerTools` goes back to refusing an unordered write: a publication that lost its ticket fetched at an unknown time and cannot be ordered. A failed reservation is reported as `orderingUnavailable` rather than swallowed, which keeps the list_changed path retrying instead of publishing a catalog that would be silently dropped, and leaves inspection unaffected by a transient cache outage. `MCPServerInspector.getToolFunctions` becomes `getToolCatalog` and returns the revision with the tools, so there is no variant that quietly discards ordering. * 📡 fix: Retry an Empty App Catalog That Could Not Reserve Ordering Review follow-up. The no-tools-capability branch destructured the reservation result and dropped `orderingUnavailable`, publishing without a revision when the revision store was transiently unavailable. That write is rejected in silence, and unlike the snapshot branch this one returned without reaching `refreshToolList()`, so whatever the server last advertised stayed in place until the connection was recreated or the cache expired. Both branches now route an unreservable catalog through the same retry path. * 📡 fix: Serve Tools Whose Shared Catalog Write Could Not Be Ordered Review follow-up. Only the shared catalog write needs ordering; the tools themselves were just read from the server and are correct to serve. Discarding them because the write could not be ordered is what turns a cache failure into a server that appears to have no tools at all, which is the reported symptom. `updateMCPServerTools` now returns the tools it built when the publication has no reserved revision, instead of null. A superseded write still discards — there another replica holds something newer. Reinitialization also asks the connection to republish under backoff when its snapshot could not reserve ordering, so the shared catalog does not stay cold until something else triggers a refresh. * 📡 fix: Surface a Discarded App Catalog Instead of Debug-Logging It #14857 went a release without a diagnostic because the only trace of a dropped app-level catalog was a debug line no deployment runs. Operators saw agents fail every turn with nothing in the logs to explain it, and the reporter had to read the source to find the cause. A publication discarded because it cannot be addressed or ordered means this server's tools are unavailable to every agent that selected them, and serving an unpublished catalog means every request re-fetches it. Both are warnings now. A superseded write stays at debug: concurrent replicas produce it routinely and the winner already holds newer tools. Tests pin the level, so a later refactor cannot quietly make the failure silent again. * 🧪 test: Pin the Reinitialize Path's Catalog Ordering Reinitialization is the path an agent falls back to when the shared catalog is cold, so it is where #14857 surfaced as "configured to use MCP tools, but none are available". Nothing pinned that it forwards the ordering its snapshot was fetched with, nor that it asks the connection to republish a catalog it could not order. Both assertions fail against the pre-fix source.	2026-08-15 12:48:23 -04:00
Anubhav Anand	a2ad0aa0c8	🤐 feat: Allow Promptless Sends When Files Are Attached (#13717) ... * ✨ feat: Allow sending file attachments without a text message When an agent asks the user to upload a document, the user could attach the file but still had to type a placeholder message ("OK", "Here is the file") before the send button enabled and the submit guard let the message through. Attachments now count as submittable content: - New isSubmittableMessage(text, fileCount) util: non-whitespace text OR at least one attached file. - ask() in useChatFunctions uses it instead of bailing on empty text, so an empty draft with attached files submits. - SendButton receives the attached file count and enables accordingly. - ChatForm only marks the text field as required when no files are attached, so react-hook-form validation no longer blocks handleSubmit. Submitting an empty draft with no attachments is still rejected at all three layers. Fixes #13646 * Address review: support replayed file-only turns + drop empty vision text - ask(): count replayed attachments (overrideFiles) in the submittable check and skip it entirely for regenerate, so a file-only message can be regenerated or saved-and-resubmitted instead of being rejected as empty. - formatVisionMessage(): omit the text content part when the message text is empty. Anthropic rejects empty text content blocks with HTTP 400, and an empty block adds nothing for other providers; image-only sends now format cleanly. Added formatMessages tests for with-text and image-only (Anthropic + other) cases. * Address review: keep attachment-only turns valid for providers, answer mode, and titles - formatMessage: substitute minimal text when a user turn carries files but no inline content, so Anthropic does not reject an empty user message for RAG or code-environment attachments. - assistants chatV1: send the same stand-in for attachment-only Threads messages, which reject an empty body. The persisted message keeps empty text. - ChatForm: attachments no longer make an empty draft submittable in answer mode, where submitText consumes the click without answering or sending. - agents request: seed title generation from attachment filenames when the turn has no text, so immediate-mode titles are not invented from an empty string. - useChatFunctions.regenerate.spec: mock the utils barrel over the real module so new exports resolve. * Cover the agents path for attachment-only turns AgentClient formats its payload with the SDK's formatMessage, not the local one, so the earlier guard missed the endpoint the feature actually targets: an attachment-only turn still reached Anthropic as an empty user message. Apply the same stand-in after the file-context and quote merges, so a turn that already gained inline content is untouched. * Carry filenames on freshly attached files The fresh-file submission mapping copied only file_id, filepath, type, and dimensions, so the attachment-only title fallback read an undefined filename and produced nothing. Include filename, and cover it with a test that submits an empty draft with one attachment. * Address review: cover assistants v2, fresh agent attachments, editor, and title fallback - agents client: the current turn has no files during buildMessages, so read the resolved attachments from message_file_map instead. The previous guard only ever fired for persisted historical turns. - assistants chatV2: the default assistants endpoint routes here, so it needs the same stand-in body chatV1 got. - assistants title: fall back to filenames, then the response, and keep the default title rather than saving an empty one. - EditMessage: retained attachments make an empty edit submittable, matching the composer, so the overrideFiles replay path is reachable from the UI. --------- Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com>	2026-08-15 12:47:31 -04:00
Danny Avila	88747f0ad8	🩺 fix: Render Stopped Run Steps From Explicit Status (#14871) ... * 🩺 fix: Render Stopped Run Steps From Explicit Status Tool calls decided "still running" vs "stopped" with a whole-message heuristic: const cancelled = !isSubmitting && progress < 1 && !hasError; That inference cannot tell which step actually stopped. An aborted step keeps spinning while `isSubmitting` is still true, and when submitting ends, every unfinished part flips to "Cancelled" at once regardless of which one died. `@librechat/agents` v3.4.6+ emits `on_run_step_closed`, a terminal per-step signal carrying `status` and timestamps — including for steps swept at end-of-run because the caller aborted. The pinned 3.5.1 already ships it; nothing consumed it. - `StepEvents.ON_RUN_STEP_CLOSED` plus `RunStepClosedEvent` / `RunStepStatus` types mirroring the SDK payload. - `PartMetadata.runStepStatus` — a dedicated field, since `status` is already claimed by activity-label and question-form parts. - Server handler forwards the event without the visibility gating the other step handlers apply: a step whose open reached the client must get its close, or the client is left inferring again. - `useStepHandler` writes the terminal status onto the tool call part. - Both decision points (`ToolCall`, the shared `useToolCallState`) prefer explicit status, keeping the heuristic as fallback for messages saved before this and endpoints that do not emit the event. Threaded through the five cards sharing `useToolCallState`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🩹 fix: Address Codex Review On Run Step Closure Rendering - Persist the terminal status server-side. The handler emitted the closure without folding it into `contentParts`, so the status existed only on the live React message: a reload or resumable reconnect dropped it and fell back to the very heuristic this fixes. Now stamped onto the aggregated tool-call part (via `stepMap`, falling back to the event's own index) before forwarding. - Honor terminal status independently of output parsing. Gating on `hasError` meant a `failed` step with unparseable output rendered as "cancelled", while a `failed`/`cancelled` step whose output did parse as an error was not terminal at all and shimmered indefinitely when no completion event arrived. A closed step now forces progress complete and reports `failed` as an error state on its own authority. - Pass the status to the second `BashCall` branch, which rendered the same updated component without it. - Reuse `Agents.RunStepClosedStatus` in `PartMetadata` instead of redeclaring the union, so a future SDK status cannot diverge between the event and the persisted part. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * ♿ fix: Replay Closed Status On Redis Resume, Announce Failures - Apply closure events during Redis reconstruction. The stamp added in the previous commit mutates only the originating process's in-memory `contentParts`; a resumable reconnect landing on another replica rebuilds from `RedisJobStore.getContentParts`, whose allowlist omits `on_run_step_closed`. The status was therefore absent from the sync snapshot and, being snapshot-covered, never redelivered as pending — so multi-replica resume fell back to the whole-message heuristic. Handled as a host-authored event alongside `on_steer_applied` and `on_activity_label`, since the SDK aggregator has no notion of it. - Announce terminal failures in the live region. Forcing terminal progress for a closed step meant a `failed` tool reached the `aria-live` region through `getFinishedText()`, which only special- cased cancellation and otherwise announced "completed function" — telling screen-reader users the opposite of what the card showed. A regression introduced by the previous commit; error states now announce failure before any completion string. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🎯 fix: Resolve Closed Steps By ID, Never By Index The steer and HITL offset wrappers clone and shift only `ON_RUN_STEP` and `ON_AGENT_UPDATE`; every other event passes through untouched. A stored `on_run_step_closed` therefore carries the SDK's unshifted index, while the part it belongs to was rebuilt at the shifted one. Any run containing a steer insertion or HITL resume would stamp the status onto an earlier tool card, or none — leaving the real card on the fallback heuristic while mislabeling a different one. - Redis reconstruction builds a step ID -> index map from the replayed `on_run_step` payloads (which carry the shifted index) and resolves closures against it, mirroring what the live callback does via `stepMap`. - The live handler drops its `?? data.index` fallback for the same reason. Skipping is the safe failure: a missing status degrades to the old heuristic, whereas a misplaced one actively mislabels the wrong card. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-08-15 10:14:16 -04:00