mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-20 15:02:02 +00:00
166 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
14d8bb8d3c
|
⚡ perf: Throttle Code Highlighting While a Message Streams (#16073)
* ⚡ perf: Throttle Code Highlighting While a Message Streams * fix: harden streamed code highlight throttling * test: cover wall clock changes during highlighting * style: tidy highlight regression test * fix: restart highlight work across lifecycle changes * style: avoid nested highlight timing ternary * fix: propagate highlight throttle to startup config * fix: share and validate highlight throttle * fix: keep streamed code visible during highlighting * test: stream highlight tool arguments * fix: preserve highlights when cadence config resolves * fix: reschedule highlights when cadence changes * test: preserve default fake tool chunking * fix: prevent stale streamed highlights * test: open the tool pane before asserting streamed highlights Every scenario in this file waited for highlight tokens without opening the card. Dev's #16143 stopped passing a closed pane any code at all, so the four runs polled a card that renders raw text and had nothing to find: the failed traces show the disclosure at aria-expanded="false" for the whole run, with no click against it and no tokens in the code element. Highlighting is now asserted through the card's disclosure, which ProgressText owns and which carries an aria-expanded state only once the card has input to show, so waiting on it also waits for the first streamed argument chunk. The open happens inside expectHighlightedCode, so a scenario cannot assert tokens without it, and it lands while arguments are still streaming, which is the only window where the throttle is under test at all. * test: reopen the rebuilt card for the settled highlight Opening the pane got the streaming assertions passing and the traces show 240 highlight spans mid-stream, so the throttle itself lands. The settled check still failed because persisting the streamed message rebuilds the card, and a rebuilt card starts closed while autoExpandTools is off: the disclosure goes back to aria-expanded="false" and a closed pane renders raw text. The settled and cancelled states now assert through the same open-then-wait helper, so each one measures the card the reader would actually be looking at instead of the tokens the streaming card happened to leave behind. * test: leave a window to cancel a streaming highlight With the settled assertions fixed, the cancellation scenario timed out on the Stop button instead: it waited for a highlight before stopping, and the ~40 argument chunks it streams at 35 ms are gone in under two seconds, so the run had already finished and there was nothing left to cancel. It failed on all three attempts. The card is now opened and the run stopped before any token is awaited, since tokens after cancellation are what the scenario is about, and the cancel variant streams its arguments at a wider cadence so the window does not depend on how loaded the runner is. Only that label changes cadence; every other scenario keeps the timing it had. * fix: preserve initial highlights and honor shared-page cadence --------- Co-authored-by: Lia <lia@librechat.ai> |
||
|
|
29ed0993a8
|
✂️ refactor: Name Tool Rounds From the Trace, and Match a Cut Response From Its End (#16113)
* ✂️ fix: Name Tool Rounds From the Trace, and Match a Cut Response From Its End * 🧵 fix: Join a Cut Model Call's Round Only When One Is Waiting * 🏷️ fix: Read Labels From the End Only for the Response the Limit Cut * 🪪 fix: Say a Tool Round Came From the Conversation Only When It Did * 🧾 fix: Carry Where a Tool Round's Calls Came From |
||
|
|
9acf9f0ac8
|
🫗 fix: Roll Back Incomplete Skill Archive Imports (#16052)
* 🫗 fix: Roll Back Incomplete Skill Archive Imports
A .zip or .skill import created the skill first and persisted its bundled
files one by one, keeping the skill and answering 201 when a file failed
validation, a size limit, storage, or its database row. The client never
read the internal `_importSummary`, so it showed "Skill created" and
navigated to a skill that silently lacked the resources SKILL.md names.
Import is now atomic: any failed bundled file rolls back the skill, its
file rows, the ACL grant, and the blobs already written to storage, and
the request answers 422 `skill_import_incomplete` with the failed paths
and reasons. The upload dialog stays open and lists them.
* fix: Localize Import Failure Reasons and Report Every Lost File
Addresses the first review round on the archive import rollback.
Failure reasons are now stable codes (`invalid_path`, `file_too_large`,
`archive_too_large`, `archive_entry_changed`, `persistence_failed`) that
the client localizes, so a non-English locale no longer sees English text
and storage or driver messages stay in the server log.
The decompression budget stops the scan mid-archive, and the entries it
never reached were missing from the response, telling the user they had
seen the whole failure set. Those paths are now reported too.
Rollback deletes stored blobs only once `deleteSkill` confirms the skill
is gone; otherwise the blobs stay and the request answers 500
`skill_import_rollback_failed`, because a surviving skill pointing at
deleted files is worse than unreferenced storage.
The failure list scrolls instead of overflowing the dialog, and a
response that resolves after the dialog was dismissed no longer
repopulates it for the next session.
* fix: Ignore Late Skill Import Successes
* fix: Make Skill Import Rollback Retryable
* fix: Propagate Incomplete Skill Cleanup
* fix: Settle Rollback Blob Cleanup
* test: Type Partial Cleanup Fixture
* fix: Preserve Settled Skill Cleanup Steps
* fix: Export Skill Cleanup Step Type
* fix: Reconcile Partially Deleted Skills
* fix: Reconcile Skill Rollback Caches
* style: format cleanup reconciliation changes
* fix: complete skill rollback cleanup
* chore: leave generated locale catalogs unchanged
* fix: preserve synced skill blob cleanup
* test: type synced blob cleanup mock
* fix: Validate Archive Paths and Bound Rollback Cleanup
---------
Co-authored-by: Lia <lia@librechat.ai>
Co-authored-by: Danny Avila <danny@librechat.ai>
|
||
|
|
2452f499a8
|
🗼 fix: Serialize MCP OAuth Token Refresh Across Replicas (#15993)
Some checks failed
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Docker Dev Images Build / publish (push) Waiting to run
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Has been cancelled
Frontend Unit Tests / Codegraph select (push) Has been cancelled
Frontend Unit Tests / Build packages (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Has been cancelled
Frontend Unit Tests / TypeScript type checks (client) (push) Has been cancelled
Frontend Unit Tests / Tests: @librechat/client (push) Has been cancelled
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Has been cancelled
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Has been cancelled
Frontend Unit Tests / Vite build verification (push) Has been cancelled
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Has been cancelled
* fix: Serialize MCP OAuth token refresh across replicas `MCPTokenStorage.inflightRefreshes` coalesces refresh-token redemptions inside one Node process. Behind a load balancer without session affinity, one user's concurrent requests land on different replicas, each reads the same not-yet-rotated refresh token and redeems it. RFC 9700 servers treat the second redemption as replay and revoke the whole grant family, so the user is asked to authorize the MCP server again. `forceRefreshTokens` now takes a cross-replica refresh flight before redeeming, using the same `FlowStateManager.acquireLease` primitive the OAuth teardown fence uses under a distinct key. A replica that waited for the flight adopts the tokens the holder rotated instead of redeeming again; when the flight is still held after the wait window it falls back to the unfenced redemption every earlier release performed. * fix: Redeem unfenced when the refresh flight lease store fails * fix: Fence MCP refresh waiters instead of redeeming unfenced A waiter that could not take the cross-replica refresh flight fell through to an unfenced redemption after 10s, while the holder's own stale abort does not fire until 60s. A refresh taking 11 to 60 seconds therefore still let two replicas redeem one refresh token, the replay this fence exists to prevent. The waiter now polls until it either adopts the tokens the holder rotated or acquires the flight, and fails as MCPTokenRefreshUnavailableError rather than redeeming beside a live holder. `getTokens` callers defer connection recovery on that error, and the stored credential is left intact for a later attempt. The flight is keyed by the stored credential (tenant, user, server name) rather than the caller's OAuth binding digest, so a rolling config change cannot hand two replicas different locks over one stored token. The wait is an operator lever, `oauthRefreshWaitTimeout`, clamped to half the stale window. An adoption read that fails no longer gives up the held flight. * fix: Annotate derived refresh-flight constants for isolatedDeclarations * fix: Recapture the credential generation a peer published on adoption * fix: Hold the MCP refresh flight until redemption settles Four corrections to the cross-replica refresh flight. The flight lease equalled the window that aborts a stalled redemption, on the claim that an expired flight could never belong to a redemption still able to reach the token endpoint. Aborting proves no such thing: the request may have been processed with its response lost, and a stalled event loop can delay the abort past its own deadline. The lease now outlives the abort, so a peer cannot redeem a credential the provider has already rotated. A live replica still releases on settle, so the margin is paid only by one that died holding the flight. The credential snapshot was taken after the first failed acquisition, so a holder that stored and released in that gap was snapshotted post-rotation: the next attempt saw an unchanged record and redeemed the credential it should have adopted. It is taken before the first attempt now. `oauthRefreshWaitTimeout` accepted zero while the runtime mapped every non-positive value to the default, so the config validated and then behaved contrary to its value. Zero is rejected at load. Adoption announced a second credential change through `handleOAuthRefreshSuccess`, whose `onOAuthCredentialsChanged` advances authorization state after persistence this replica did not perform. That retired the generation recaptured beside it and fenced the build against its own tool publication. Adoption now updates the local token-flow cache and recaptures, without announcing. * fix: Check every held MCP refresh flight for a peer's rotation Four corrections, two of them consequences of moving the credential observation ahead of the first lease attempt. That move created a storage read inside `beginRefreshFlight`, and its failure reached a handler written for a lease-store outage, so a transient read error became an unfenced redemption beside a live peer. The flight now handles its own reads: losing the observation costs adoption and nothing else, and only the lease store failing may redeem unfenced. The observation was also never compared when the first acquisition succeeded. A peer that rotated and released before this replica contended left an acquisition that looked uncontended, and its fresh credential was redeemed a second time. Every acquisition now runs one rotation check, which also collapses two code paths into one. `getTokens` already loads the refresh record to decide a refresh is needed, so it is passed on as the observation baseline instead of read again, and the read taken under the flight is reused as the credential redeemed. Two reads on a latency-counted path where there were three. `runSilentRefresh` collapsed contention into null, sending the 401 path to interactive OAuth, so a slow peer prompted the user to authorize a server whose credential was about to be valid. The retryable outcome now travels through the silent-refresh layers and the connection defers, matched by name as well as identity because these errors cross the package boundary. * chore: Regenerate the Agents OpenAPI spec for repositoryInstructions The drift check added by #15928 fails on dev: #16008 added `repositoryInstructions` to the agent schema without regenerating the committed spec, so every branch merging dev inherits the failure. This is the remedy the check itself prescribes and carries no hand-written change. Drop this commit if dev regenerates first. * fix: preserve MCP refresh outcomes across replica boundaries * fix: keep MCP adoption fenced through publication * fix: anchor MCP refresh adoption to rejected credentials * fix: Complete OAuth Adoption Adapters and Isolate Legacy Flow Readers * fix: Preserve OAuth Request Identity Through Recovery and Discovery * fix: Fence OAuth Adoption Against All Credential Writers * fix: Gate Coordinated OAuth Rollout and Preserve Unauthenticated Identity * fix: Preserve OAuth Coalescing Across the Staged Rollout * style: Sort OAuth Timeout Constant Import * fix: Invalidate Both OAuth Token Flow Protocols on Rotation * test: Complete the Rollback Token Flow Fixture --------- Co-authored-by: Lia <lia@librechat.ai> Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
|
9633edcb88
|
🗣️ fix: Keep Provider Error Text on Unclassified Agent Failures (#16034)
* 🗣️ fix: Keep Provider Error Text on Unclassified Agent Failures
An upstream failure LangChain does not classify discarded the provider's own
message and answered with the generic upstream sentence plus a status, so a
gateway or privacy-proxy rejection lost the only account of what happened.
The provider text now rides along in the persisted payload as `message`,
withheld only where a content policy inspects the traffic — the same condition
`getUserFacingProviderError` and `getUserFacingRequestError` already decide by.
* fix: Harden Provider Error Detail Retention
---------
Co-authored-by: Danny Avila <danny@librechat.ai>
|
||
|
|
a26a6709ff
|
📪 feat: Send Per-Request MCP Headers Without Hiding the Tool Catalog (#15988)
* 📪 feat: Send Per-Request MCP Headers Without Hiding the Tool Catalog
A `{{LIBRECHAT_BODY_*}}` placeholder in an MCP server's `headers` marks the
server request-scoped, and catalog discovery has no conversation to resolve it
against, so LibreChat skipped tool discovery and the server appeared with no
individual tools.
Operators can now declare those headers separately in `requestHeaders`, which
discovery omits and chat turns resolve and merge over `headers`.
* fix: normalize requestHeaders at each resolution entry point
Registering the new field in every consumer that enumerates header-bearing
config fields left five gaps: startup inspection, case-insensitive override,
the admin-configurable comparison surface, Graph token preprocessing, and
direct-bearer recovery.
Fold `requestHeaders` into `headers` at the entry of each resolution pipeline
instead, so Graph preprocessing, direct-bearer detection, `processMCPEnv` and
the transports keep reading one header map and never learn a second exists.
Catalog paths strip it at the same boundary, startup inspection included.
* fix: keep servers with chat-only headers off the shared app connection
A static `requestHeaders` map left a server eligible for app-level sharing, and
`ConnectionsRepository.loadConnection` handed the raw config to the factory, so
the shared session's own `initialize` and `tools/list` carried headers
documented as chat-only — and every later catalog read reused that session.
One session cannot serve both sides of the field. Exclude such configs from
`canUseAppConnection`, the single predicate both `connectAppServers` and the
repository's own app-level gate already consult, so the chat turn keeps its
headers on a user-scoped connection while catalog work never sees them.
* fix: normalize requestHeaders before the manager's bearer decisions
`createUserConnectionInternal` reads `usesDirectOpenIDBearerRecovery` and
`resolveDirectOpenIDBearerConfig` on the config it receives, while the merge sat
in `resolveRuntimeConfig`, a leaf helper below those calls. An `Authorization`
template declared in `requestHeaders` therefore never entered direct-bearer
mode: a `tools/list` 401 skipped the force-refresh and the connection was
returned with a stale bearer.
Normalize at the birth of that pipeline's config instead, ahead of all four
decisions, and make `getMCPAppToolsPublicationGeneration` normalize before
hashing. Connection paths hold a merged config while cache paths hold the
declared one, so without a canonical hash the two would address different
catalog slices.
* fix: Preserve MCP request header ownership across lifecycle boundaries
* fix: Narrow remote header identity before rebuilding config
* fix: Preserve request header precedence over catalog API keys
* docs: Synchronize generated agent API schemas
* fix: Ignore shadowed generated MCP user key requirements
---------
Co-authored-by: Lia <lia@librechat.ai>
Co-authored-by: Danny Avila <danny@librechat.ai>
|
||
|
|
dea7346d72
|
🧷 feat: Carry Ask User Answers Past Compaction (#15944)
* 🧷 feat: Carry Ask User Answers Past Compaction The user's answers to `ask_user_question` lived only in the tool result that received them. A checkpoint summary, the tool-output pruner or a full context window could remove that message, and the agent then asked again or acted on a summary's paraphrase. Every answered ask on the conversation branch is now rendered verbatim into the run's dynamic instructions, on the first request and on every resume, where the answer just given is read from the seeded content. Older answers drop first past a token budget; the newest set is always kept. New lever: `endpoints.agents.askUserQuestion.retainedAnswers` (`enabled`, `maxTokens`), on by default with a 4096-token budget. * 🧷 feat: Carry Ask User Answers Past Compaction The user's answers to `ask_user_question` lived only in the tool result that received them. A checkpoint summary, the tool-output pruner or a full context window could remove that message, and the agent then asked again or acted on a summary's paraphrase. Every answered ask on the conversation branch is now quoted verbatim at the top of the current user turn's prompt copy, in the user's own role: rebuilt from the stored rows every turn, never persisted, excluded from the memory copy, and counted in the prompt total with the active encoding. A warm event-actor turn reads the branch on demand. Older answers drop first past a token budget that includes separators; the newest set is always kept. New lever: `endpoints.agents.askUserQuestion.retainedAnswers` (`enabled`, `maxTokens`), on by default with a 4096-token budget. * 🧷 refactor: Read The Warm Branch Inside The Module The warm event-actor read lived as a conditional and a `getMessages` call in the CJS client. The typed module now takes `historyLoaded`, the identity and the query as dependencies, decides when to read, and owns the failure policy for both the read and the block itself; the client passes plumbing only. * 🧷 fix: Complete The Branch Behind A Summary Cut `loadHistory` already stops its walk at a checkpoint summary, so the rows `buildMessages` receives never held the answers this feature carries. The module now detects a branch that stops short of its root, whatever cut it, and completes it through the injected stored-row query, stored rows winning. The block goes on the latest user-authored row so a continued generation keeps it, and the render measures the whole block before reserving the omission note, so a set is only dropped when something truly has to go. * 🧷 fix: Reuse Loaded History And Read Every Ask Stamp A post-compaction turn read the conversation twice: once in `loadHistory`, which cuts at the checkpoint summary, and again in the module to complete the branch. The loader now keeps the rows it read and the module prefers them, querying only for a turn that made no history read. Two stored shapes were missed: a pause record whose `args` is the bare question string, now read as the legacy question, and a call whose input failed validation, whose persisted error output is no longer quoted as an answer. * 🧷 fix: Keep The Block Off The Stored Row And Out Of Memory Copies Array-form content is replaced instead of edited through the array the stored row shares, the block's cost is measured as two fresh counts of the same copy rather than against a possibly calibrated stored count, the memory copy is only built when memory extraction will read it, the loaded rows reach the agents client through a hook and are released once the block exists, and the tool-call-id merge is gone: rows are unique by message id and providers may reuse ids across turns. * 🧷 refactor: Apply The Answers Block Inside The Module The placement lookup, the prepend, the count delta and the memory-copy decision lived in the client after the context kickoff. They now live in `applyRetainedAnswers`, which receives the rows, the prompt copies, the count map, the counter and the memory dependency; the client keeps one call and passes plumbing. * fix: Apply Retained Answers After Summary Replay 🧷 * perf: Count Retained Context Exactly Within Its Budget 🧮 * fix: Preserve Retained Counts Across Checkpoint Recounts 🧾 * fix: Isolate Retained Context Counting and Provenance 🧵 * fix: Replace Warm Retained Context and Verify Answer Provenance 🪪 * fix: Use Workspace-Compatible Array Traversal 🧭 * fix: Refresh Retained Context at the Warm Turn Boundary 🧲 |
||
|
|
2ffee7e143
|
🪂 feat: Opt-In Text Fallback for Files No Tool Can Read (#15940)
* 🪂 feat: Opt-In Text Fallback for Files No Tool Can Read An upload routed to tools (`llmDeliveryPath: none`) reaches nothing when the agent handling the turn runs neither Run Code nor File Search, or runs one that cannot read the type. With `fileConfig.endpoints.<endpoint>.textFallbackWithoutTools` (also accepted at the top level and inherited from `default`), such a turn delivers the text extracted at upload instead. The setting is off by default, which keeps today's behavior. - Upload: when the setting is on and an inferred `none` route finds no reading tool, the built-in document parser or native text reader stores the text on the file, best effort, under the extracted-text size cap and content policy. - Turn: `resolveTurnLLMDeliveryPath` re-resolves each inferred route against the tools the agent runs, and `applyTurnTextFallback` marks copies of the records a turn loads, so endpoint filtering, limits, inspection, history replay and steer media all see the text the turn delivers. The stored route is never rewritten. - Explicit destinations, records predating routing, and turns whose tools are unknown are left as they are. * 🧪 test: Resolve Custom Endpoint Fallback Under the Endpoint Name Uploads route a custom endpoint agent by its saved provider, the endpoint name. initializeAgent marks turn copies before getProviderConfig swaps the provider for the backing client, and the endpoint keeps the name afterward, so the client and child encoders resolve the opt-in under the same endpoint. Cover both the initialization path and the post-initialization agent shape. * ♻️ refactor: Share One Extracted-Text Storage Cap Across Uploads The fallback defined its own copy of the 15 MiB cap context uploads already apply to extracted text. Export MAX_STORED_EXTRACTED_TEXT_BYTES from files/extract and use it in both places, documenting that it bounds what a MongoDB file document can hold rather than operator policy. * 🧭 fix: Keep Fallback Text Wherever a Later Turn Can Deliver It Extraction skipped uploads filed under a reading tool, so a handoff agent without that tool, or the same agent after its tools or grants changed, had no text to fall back to. Store fallback text for every inferred route on a message attachment; files kept on an agent's tool resources never reach a prompt and still skip it. Turn marking only ran while the fallback was enabled, so a stored tool-routed record whose type the endpoint now routes to text stayed marked `none` and `extractFileContext` skipped it. applyTurnTextDelivery (renamed from applyTurnTextFallback) now marks any stored `none` record with text that the turn resolves to text, and returns early when no such record is loaded. * 🚦 fix: Admit Every Attachment by the Route Its Turn Delivers A record upload routed to tools that the current endpoint sends to the provider stayed marked `none` on the turn copy, so admission skipped it while BaseClient encoded its bytes, bypassing attachment count and size limits. applyTurnDelivery (renamed from applyTurnTextDelivery) now gives every record whose route upload inferred the route the turn resolves, before endpoint filtering, limits and inspection, matching the run-file encoder. hasInferredLLMDeliveryPath shares the inferred-route rule with the resolver. Fallback extraction selected the document parser inside the CJS upload path. resolveUploadFallbackText now takes the upload and picks parseDocument or parseTextNative itself, with the extractors injectable, so process.js passes request data only. Its tests run the real parser against the xlsx fixture and the real native reader against temporary files. * 🎚️ fix: Read Custom Endpoint Dialect From Config on Both Sides of Init initializeAgent materializes turn routes before getProviderConfig swaps the saved custom endpoint name in agent.provider for its backing client, so routing read that name as a non-OpenAI dialect. Media a custom endpoint opted into could then be materialized off the provider path, skipping admission, while the finalized client still sent the bytes. resolveAgentDeliveryRouting now reads the dialect with getCustomEndpointProvider, as upload does, which holds before and after the swap. The upload fallback plan now keys on the chosen-destination marker the turn resolver reads, so an upload recorded as chosen, including every legacy chooser upload, never pays for text a turn cannot deliver. * 🛡️ fix: Apply Turn Routes Only While the Record Stays Admitted Resolution while initializing and at delivery can disagree: delivery reads a Responses API choice only the finished client config holds. Applying a route that takes a record out of admission, off the model path or to text it never stored, would then skip the limits for a file the client still sends. applyTurnDelivery now applies a route only while the record stays model-bound; admitting a record the turn leaves out cannot slip past a limit. * 🕹️ feat: Run Declared Project Actions on Attached Workers (#15943) * feat: use declared attached project actions * Complete named action definitions and approval previews * Sort named environment imports * Sort native environment fixture imports * 🔓 fix: Keep Code Approval Mode Selectable Mid-Run (#15938) * 🔓 fix: Keep Code Approval Mode Selectable Mid-Run The composer's code approval mode selector was disabled while a run streamed, and a mode picked during a run was reverted when the final or abort event merged the server conversation back. The selector now stays usable in flight and both merges retain a mid-run selection, which the next send carries. * 🔓 fix: Retain Mid-Run Approval Mode Across Cache and Recovery Stamp the live approval mode onto the submission conversation at send so the retention baseline matches what was read, keep a retained mode in the conversation query cache on final and cancel, and rebuild failed or aborted conversations from a preset that carries the mid-run pick. * 🔓 fix: Scope Retained Approval Mode to the Submitted Conversation Retention now requires the live conversation to be the one the run submitted, or the id the server assigned to a new chat, so navigating elsewhere mid-run cannot write that conversation's mode back. A single recovery helper rebuilds failed or aborted conversations from the retained preset and patches the detail cache for every recovery site. * 🔓 refactor: Keep the Local Approval Mode on Every Server Merge Replace per-path retention with one rule: a locally picked code approval mode is newer than any server copy of the same conversation, so the final and abort merges and the error recovery preset keep it. The selector writes the pick into the conversation's detail cache, which navigation rebuilds from, so the pick is conversation-scoped without reading the index-global atom from the SSE handlers. * 🔓 fix: Prefer the Live Conversation When Caching the Final Merge The final handler's detail-cache merge now takes the local approval mode from the open conversation when it is the one that finished, falling back to the cached record only for a conversation that is no longer on screen. * 🔓 fix: Keep the Local Approval Mode Through Settled-Start Reconciliation The resumable transport's settled-start reconciliation and replacement handoff wrote a fetched conversation straight into conversation state and the detail cache. Both now apply the same local-mode rule as the event handlers, treating a new chat's pending id as the settled conversation's own. * 🔓 fix: Seed the Detail Record When a Mode Is Picked A conversation that has its id but no detail record yet now gets one from the live conversation when a mode is picked, the same key the resumable transport seeds optimistically, so recovery and navigation find the pick. A chat with no id yet still keeps the pick in conversation state alone. * ⬆️ chore: bump agents to 3.8.7 (#15947) * 🧭 refactor: Settle Turn Delivery Routing Once in Agent Initialization `initializeAgent` resolved the provider and its client options only after the turn's files were loaded and admitted, and the two delivery readers each rebuilt the attachment routing from the agent object at their own moment: `BaseClient` took the custom-endpoint dialect from the already-swapped `agent.provider`, the run-file encoder from whatever child config it was handed. Move `getProviderConfig`/`getOptions` ahead of file discovery, where nothing in between fed them, and settle one `deliveryRouting` value with every input final: the file policy under the endpoint's own name, the dialect its config declares, the Responses API decision the model call uses, and the transcription setting. `InitializedAgent`, the child encoder and `BaseClient` consume that value; `resolveTurnLLMDeliveryPath` is the one place a stored route is resolved again. * fix: Preserve attachment reachability across handoffs and late steers * test: Type the untrusted attachment reference fixture --------- Co-authored-by: Ravi Kumar L <upman@users.noreply.github.com> |
||
|
|
854645ffc0
|
🔁 feat: Retry OpenID Discovery After Startup Failures (#15899)
* fix: retry OpenID discovery after startup failures * fix: keep OpenID background retries resilient * refactor: move OpenID retry policy into API package * 🔁 feat: Expose OpenID Discovery Retries Through Registration Config * 🧭 refactor: Own OpenID Discovery Registration and Defaults in TypeScript --------- Co-authored-by: insoln <insoln@ya.ru> |
||
|
|
696bcb6c1e
|
🏦 fix: Reserve Credits for In-Flight Requests (#15879)
* 💳 fix: Reserve Balance for In-Flight Requests Admit each balance-checked request against the credits that other in-flight requests have not already reserved, and hold its prompt cost on the balance record until the turn settles. Apply a due auto-refill in the same fenced write so one refill window credits exactly once. - data-schemas: reserveBalance / releaseBalanceReservation replace createAutoRefillTransaction; the write is a compare-and-swap on tokenCredits, reservations and lastRefill (no pipeline updates) - balance.reservations is select:false so existing reads never see it - checkBalance returns a reservation handle; BaseClient and the assistants controllers release it once the turn settles - balance.reservationTtlMs bounds how long an unreleased reservation counts (default 30 minutes) * 🧮 refactor: Admit Reservations Without Contention and Record Refills Durably - admission is a conditional $push/$inc against a reservedCredits total, so concurrent admissions of a funded balance commit on their first write - expired reservations are pruned per element with their credits - auto-refill is fenced on every setting its decision reads and leaves a pendingRefill ledger marker in the same write; any later read records the transaction under the fixed id and clears the marker - a missing record is created with $setOnInsert inside reserveBalance - reservation lifecycle moves to packages/api (createBalanceReservations, withBalanceReservations); BaseClient releases once usage is recorded * 🪪 fix: Resolve Each User to One Balance Record - balance records a creator inserts are keyed by the user id, so creators racing on a missing record converge on one document; every balance read and write resolves a user to their oldest record, so existing duplicates stay inert - upsertBalanceFields takes insert-only fields; the balance-config middleware, schedules and lazy initialization pass the starting credit through them so it never overwrites a record another writer created - the auto-refill write is also fenced on reservedCredits - the scheduled-chat balance pre-skip counts reserved credits as unavailable * ⏳ fix: Count Only Unexpired Reservations in Scheduled Balance Checks findBalanceByUser with includeReservedCredits now totals the reservations that have not expired, so a reservation left by a crashed request stops pre-skipping scheduled chats at its expiry instead of when a later reservation write prunes it. * 🔁 fix: Renew Balance Reservations While Their Request Runs A reservation is renewed every half TTL until it is released, so only a reservation whose process stopped expires. Pruning removes a reservation only while it is still expired at the write, so one renewed after the pruning read survives. * 🛟 fix: Keep Reservations Alive Through Renewal Failures and Lost Acks - renewal reschedules itself: every half TTL, capped to the timer range, and a failed renewal retries within a tenth of that window - an admission write that errors removes its possibly committed reservation before rethrowing * 🛑 fix: Hold a Stopped Turn's Reservation Until Its Charge Lands - an aborted turn lets its reservations lapse after a minute (or the TTL) instead of releasing them, so the Stop request's charge lands before the credits are admitted again - balance.reservationTtlMs has a 10 second minimum, enforced in the schema and at runtime, so renewal stays bounded * ↩️ revert: Release a Stopped Turn's Reservation When the Turn Settles An aborted turn releases its reservations when it settles, the same as every other exit, instead of holding them for a fixed lapse window that refused funded requests after every Stop. The minimum reservation TTL stays. * ⏸️ fix: Hold Assistants Reservations Through Background Run Retries An in-progress Assistants run continues in the background after the handler responds; the turn's reservations now stay held until that run settles (createBalanceReservations().holdUntil). * 🔁 fix: Apply an Auto-Refill Once per Admission A refill interval that is due again the moment it is applied, such as a zero interval, let one admission refill repeatedly until the retry bound. Each admission now applies at most one refill, as the previous check did. * 🧹 fix: Prune Expired Reservations in One Write An admission that found expired reservations issued one write per hold, so a backlog left by a stopped process fanned out into as many concurrent writes. The prune is now a single write fenced on every hold it removes; a hold renewed after the read makes the write miss and the admission re-read. * ⏱️ fix: Renew Reservations From Their Stored Expiry The next renewal was scheduled half a TTL after the previous write settled, and the first one half a TTL after the admission returned, so a slow write could let a live hold expire before its renewal. Renewals are now scheduled from the expiry the last write stored. Expired holds are pruned as one bulk write of per-hold fenced updates, so a hold that changed after the read is skipped while the rest are removed, and the admission no longer spends a retry on the whole prune. |
||
|
|
9cf33cc8be
|
🧿 fix: Bind Login Flows to the Browser That Started Them (#15880)
* 🧿 fix: Bind Login Flows to the Browser That Started Them Social OAuth logins (Google, Facebook, GitHub, Discord, Apple) now store a random state in an HttpOnly cookie scoped to the callback path and only exchange the returned code when the callback carries that state. Apple's form_post callback uses a SameSite=None cookie, and passport-apple's own fixed state no longer bypasses the store. Local login, temp-token 2FA verification and admin local login reject requests a browser sent from another origin (Sec-Fetch-Site, falling back to Origin vs Host), trusting DOMAIN_CLIENT and DOMAIN_SERVER. Requests without browser fetch metadata, such as server-side callers, pass unchanged. The sessionless OAuth callbacks drop failureMessage, which wrote to req.session and turned any rejected callback into a 500. * 🧿 fix: Configure Login State Cookies and Localize Origin Rejections - OAuth state cookies take secureCookie and maxAgeMs from the caller; registration.oauthStateTtlMs in librechat.yaml sets the lifetime (default 10 minutes). - Secure deployments use a __Host- cookie at Path=/ so a sibling subdomain cannot plant a state. - The cookie keeps up to three pending states, so logins started in separate tabs all complete, and an unmatched callback no longer discards them. - passport-apple's preset state is dropped by deferStateToStore in packages/api. - The Origin fallback compares scheme and host against the request itself. - The same-origin rejection carries ErrorTypes.AUTH_CROSS_ORIGIN, which the login and 2FA screens show as a localized message. * 🧿 refactor: Sign Login State Against a Per-Browser Binding - The OAuth state store keeps a random per-browser binding in a __Host- cookie (reused across flows) and signs each state as issuedAt.nonce.HMAC(JWT_SECRET, provider, binding, issuedAt, nonce). Nothing is stored per flow, so any number of tabs complete, each state expires on its own clock, and concurrent starts no longer overwrite each other. - The experimental server entrypoint passes its app config to configureSocialLogins. - The same-origin guard also trusts an explicitly configured ADMIN_PANEL_URL. * 🧿 fix: Keep Separate Bindings for Racing First Logins A browser without a login-state binding now gets one under a cookie name with a random suffix, so two tabs that start their first login at once each keep their own binding. Later starts reuse an existing binding, and a callback is accepted when its state was signed for any binding the browser presents. * 🧿 fix: Sanitize Request-Derived Values in the Cross-Site Rejection Log The same-origin guard logged the request Origin and path verbatim. A non-browser client fully controls the Origin header, and the console transport uses a printf format that does not escape control characters, so a crafted value could forge log lines. Strip ASCII control characters (CR, LF, NUL and the rest) before logging, matching the pattern apiNotFound already uses. |
||
|
|
e18606e5ce
|
🚀 v0.8.8-rc3 (#15859)
Some checks failed
Docker Dev Images Build / publish (push) Waiting to run
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
Publish `@librechat/client` to NPM / pack (push) Has been cancelled
Publish `librechat-data-provider` to NPM / pack (push) Has been cancelled
Publish `@librechat/data-schemas` to NPM / pack (push) Has been cancelled
Publish `@librechat/client` to NPM / publish-npm (push) Has been cancelled
Publish `librechat-data-provider` to NPM / publish-npm (push) Has been cancelled
Publish `@librechat/data-schemas` to NPM / publish-npm (push) Has been cancelled
* 🚀 chore: prepare v0.8.8 * 🚀 chore: prepare v0.8.8-rc3 * docs: mention Agent workspace defaults |
||
|
|
f43301f818
|
🔭 feat: Conversation Trace Viewer (#15869)
* 🔭 feat: Conversation Trace Viewer Adds a trace view for a user's own conversations: a summary, a pinned waterfall overview with interval focus, a hierarchical record ledger and a record inspector, opened from the chat header (or the mobile overflow menu) and read through a provider-neutral seam with Langfuse as the first reader. - interface.traceViewer (enabled, showInputOutput, maxRecords, maxContentLength, requestsPerMinute), off by default - /api/traces/:conversationId availability, records and record routes, re-proving conversation ownership and narrowing the Langfuse session to traces the user's own sampled responses produced - Langfuse reader over the v2 Observations API with cursor paging, time bounds, typed failures and server-side input/output withholding - forks no longer copy the source messages' trace sampling record * 🛂 fix: Address Trace Viewer Review Round 1 - interface.traceViewer.requestTimeoutMs bounds each Langfuse round trip - availability is one existence query across every sampled response instead of the oldest one, run alongside the ownership check - tree items report their position among visible siblings - point-in-time Langfuse events read as completed, not running - reads prefer the project holding the most responses when a tenant connection arrived mid-conversation * 🎯 fix: Leave Focus With a Navigation That Closes the Trace * 🧭 fix: Keep Trace Reads on One Project and Cancel Abandoned Reads - page cursors and record details stay on the project that served the first page; pages report an opaque sourceId - Open in Langfuse only appears when the linked project served every loaded page (the session link now reports its destinationId) - client and server abort Langfuse reads when the viewer closes or the client disconnects - reads never wait on the central project lookup, which the request timeout does not govern - the cost total is withheld when any model call with usage is unpriced * 🕰️ fix: Refresh Trace Pages After a Turn, Honor Clock Settings, Show Refresh Failures - a settled run invalidates the cached trace pages, so reopening shows the new turn instead of pages fresh for another 30 seconds - trace durations and clock times format in the app language and the user's 12/24-hour setting, as message timestamps do - a failed refresh with every page loaded surfaces an alert and a retry * 🔐 fix: Only Server-Authored Responses Grant Trace Access - the message-create route and imports strip langfuseSampled and langfuseDestinationIds, and trace reference queries ignore client-authored rows (isUserSubmitted) and user turns, so a forged row cannot claim a trace - the Langfuse reader receives its destination resolver and HTTP client from the route instead of defaulting to process-global ones - availability asks the client to check again while central's project id is still resolving, instead of caching a false answer - listRecords tries the next readable project when legacy responses left the first one empty - a settled run also invalidates cached record details - running is read from a record's status, not a missing end time * 🏢 fix: Scope Trace Reads to the Tenant and Reach Every Project Holding a Turn - ownership and both trace reference queries carry the requester's tenant (tenantless is its own scope), so a duplicate user and conversation id in another tenant cannot grant or supply trace references - a read visits, page by page, every project that could hold a turn no earlier project provably holds, so legacy turns in another project stay reachable after a newer destination ranks first - the trace read limiter takes its store from the route - transient availability failures retry instead of hiding the control * 🛟 fix: Fail Over Between Projects, Refresh One Page, Search Visible Labels - a fresh read fails over to another project holding the same turns when the preferred one fails; the cursor carries the failed sources so later pages keep the rebuilt plan, and detail reads fail over the same way - destinations for one project collapse into one source - the trace read limiter keys each user within their tenant - refresh and a settled run keep only the newest page before refetching, so older pages are not replayed through the limiter; retry repeats the read that failed - search matches the localized kind and status labels a row shows * 🧭 fix: Read the Newest Turn First, Report Failures Behind Empty Projects, Fit Trimmed Views Rank trace sources by the newest sampled response they could hold before coverage and preference, so a conversation that moved projects shows its latest turn first. Throw a list failure when a failed project could hold a turn that no answering project provably holds, instead of rendering an empty trace. Detail reads keep probing projects after an empty answer, skip ones whose turns an answering project provably holds, and surface a failure that could hide the record. Refresh and settled runs trim the cache to the newest page; the viewer now fits a focused interval to the remaining records or clears it, and drops a selection whose record left with the older pages. * 🧵 fix: Page Traces by Turn, Include Failed Runs, Keep the Active Row Mounted List reads now page by turn segments instead of whole projects. A segment is the newest unread turn and up to 49 older consecutive turns one project could hold. The request filters Langfuse to those turns' run and title traces, so pages stay newest first when a conversation's responses alternate between projects. Legacy turns without destination ids are asked of the other projects in the segment. A project that failed for newer turns is retried when it alone holds older turns. A page whose rows all fail the observation schema is an upstream failure, not an empty trace. A failed turn's error row now stores the sampling fields of the run that failed, plus langfuseRunId when that run was created, so the trace viewer and feedback scores follow the run's trace. langfuseRunId is stripped from client message writes and imports, dropped by forks, and hidden from client message projections. The ledger keeps its active tree item mounted while virtualization scrolls it out of the window, so aria-activedescendant always names a rendered row. * 🕰️ fix: Merge Multi-Project Trace Pages by Start Time, Keep Literal Content A segment that reads more than one project now runs every read before returning. A read that fills the record budget cuts the page at the oldest start time all reads loaded, and the continuation carries that start time as an inclusive bound instead of a Langfuse cursor. Every project continues from the same point, so a newer turn held by another project is never left behind an older one, and failover works on any page. A page drawn from several projects no longer names a single source. A continuation that would repeat its own bound fails instead of looping. Input and output that are literally the strings {} or null are kept; only an empty object or JSON null value is dropped. * 🔎 fix: Read Each Trace Segment From a Project Shown to Hold It Recorded destination ids say where a trace was eligible to go, not where it landed. A list read now looks up the turns more than one readable project could hold before choosing where to read them. Each project, preferred first, is asked which of those traces it holds a root observation for, and each turn is read from the preferred project that holds it, or from the preferred eligible one while a run's root is not exported yet. Turns only one project could hold need no lookup. Segments are consecutive turns from one project, paged by that project's own Langfuse cursor. Records that share a start time page correctly, and the page budget bounds a single read. A failed project is dropped for the request. A turn only it could hold raises its failure at the turn's own place in page order, instead of leaving an empty or partial trace. Cursors carry the segment's trace-set hash and start over when the segment changed or Langfuse rejects them. Detail reads ask every project until the record turns up. The trace summary withholds its cost total when any generation has no cost, with or without usage. * 🧷 fix: Evidence per Trace, Continuations Without Replays, Stable Turn Order A turn's run and title run are exported separately, so each trace id is looked up and read from a project shown to hold it. A trace no project shows (a title run the turn never had, or a run whose root is not exported yet) joins an adjacent segment whose project could hold it. Only a run that a failed project alone could hold raises that failure. A continuation now stays on its project. Its failure is returned as is, and a cursor the segment no longer matches, or one Langfuse rejects, returns invalid_request instead of replaying the segment's first page. The viewer answers invalid_request on an older page by reloading from the newest page, with a message saying the trace changed. Sampled responses sort by createdAt then _id, so responses saved in the same millisecond keep one order across page requests. * 🪪 fix: Scope Trace Reads to the Exporting User, Order Split Title Runs Trace ids derive from message ids, which a request can influence, so a colliding id must not return someone else's trace. Every Langfuse list, probe and detail read now also filters by userId: the requester's internal id, plus the configured langfuse.trace.userIdField value when set. The server stamps that id on the trace at export. The handler passes the requesting user into the trace query. When a turn's run and title run sit in different projects, the probes' root start times decide which is read first, so a title generated after the response is not left behind Load older. Cursors name the trace a segment starts at within its turn. The failed-turn trace lookup moves into packages/api as getFailedTurnTraceFields. The agent controller only passes the error row id, the run id and whether the run was created. * 🔒 fix: Authorize Trace Reads by the Internal User Id Only A configured langfuse.trace.userIdField can export a non-unique value (name, username) or one that repeats across tenants (email, provider ids) as the trace userId, so it cannot prove who a trace belongs to. Reads now require the internal user id alone. A deployment that exports another field reports no trace and logs why once, instead of trusting that value. A failed read of a title run that a probe found in that project now raises its failure instead of dropping the title. The page contract now states that pages are ordered by turn and a turn's records may continue on the next page. * 📚 fix: Page Sampled Responses With the Trace, Share Ingress Stripping List reads load sampled response references a segment at a time: up to 50 ending at the cursor's turn, plus one older response that names the next page. Pages of a long conversation no longer rematerialize every sampled response. getConversationTraceRefs gains through and limit, anchored by createdAt and _id. The message-create route and conversation imports strip trace sampling fields through withoutTraceRefs instead of their own field lists. A userIdField the exporter ignores counts as the internal id. A root lookup whose cursor repeats fails that project instead of paging forever. In the viewer, Refresh also rereads the open record's detail, and closing the inspector after a filter removed every row returns focus to the search field. * ⏱️ fix: One Round Trip per Trace Reference Read, Detail Reads Scoped to Their Turn Cursors carry the anchor response's order key (creation time, then _id). The bounded sampled-response read then runs alongside the first-message lookup, with no anchor lookup ahead of it. A key that no longer names its response returns nothing, and the reader answers the cursor as changed. Detail reads take the turn the list attributed the record to (?message=). They load only that sampled response instead of the conversation's whole history, and ask Langfuse within that turn's traces. The route rejects a detail read without it, and the client sends it from the listed record. A list read whose Langfuse cursor repeats now fails instead of handing the next page the same records. * 🧭 fix: Reject Out-of-range Trace Cursors, Open Any Listed Turn, Mark Copies Unsampled A page cursor whose base-36 time parses to a number outside the Date range reached Mongoose as an Invalid Date and surfaced as a 500 the viewer retried; it now resumes nothing, which the reader reports as invalid_request so the viewer reloads its newest page. The detail read capped the message id at the Langfuse record-id bound while the app stores response ids at any length, so a listed turn could refuse to open; message ids only name a stored row and are accepted at any length. Ids that reach a Langfuse filter keep their caps. Copied and client-authored rows now carry langfuseSampled: false instead of no record, so a feedback score on a fork, import or posted message never recomputes sampling for a trace that was never made. * 🧪 test: Type a Recordless Row for the Unsampled-copy Spec An object literal with none of the trace fields fails TypeScript's weak-type check against the helper's all-optional constraint; the spec now passes a typed row that declares the optional field, as every stored message type does. * 🚰 fix: Release Failed Langfuse Response Bodies A non-success Langfuse response was turned into a TraceReadError without consuming or cancelling its body, and Node's fetch returns a connection to its pool only once the body is released, so a sustained upstream failure left sockets checked out until garbage collection. The body is now cancelled before the status becomes an error. |
||
|
|
f25242df98
|
🦘 feat: Share Run-Scoped Files With Subagents (#15848)
* feat: add run-scoped file sharing for subagents Delegate current-message files through authorized execution catalogs and publish selected immutable code-output versions through the existing file pipeline. Requires the companion agents SDK context adapter before the dependency can be released and pinned. * fix: preserve child tool artifacts and verify file-sharing lifecycle Keep citations and other non-sandbox artifacts on their existing delivery path while retaining private code outputs and inspected images until publication. Cover native and text delivery, nested and sibling authorization, concurrent runs, cancellation cleanup, and checkpoint recovery through the browser harness. * test: align file-sharing fixtures and integration cleanup * test: repair LibreOffice conversion fixtures * test: verify deployment skill bytes across checkout formats * fix: align subagent file budgets and refresh boundaries * chore: update agents SDK to 3.8.6 * refactor: move code output processing into API workspace * fix: enforce shared file budgets for parents and teams * fix: declare the shared file encoder interface |
||
|
|
442d477acf
|
🛎️ fix: Preflight MCP Readiness for Scheduled Chats (#15782)
* fix: Validate Scheduled MCP Readiness 🕰️
* fix: Keep Schedule Permission Store Failures Retryable
* perf: Parallelize Scheduled MCP Preflight
* fix: Preserve Scheduled MCP Invariants
* fix: Close scheduled MCP recovery gaps
* fix: harden scheduled MCP recovery
* fix: bound scheduled MCP validation
* fix: harden scheduled MCP admission bounds
* fix: align scheduled MCP graph admission
* fix: bound scheduled MCP admission
* fix: align scheduled MCP preflight with runtime
* fix: persist scheduled MCP failure outcomes
* fix: match scheduled MCP runtime semantics
* fix: parallelize scheduled MCP admissions
* fix: mirror scheduled subagent traversal
* fix: enforce scheduled subagent config limits
* fix: bound scheduled MCP initialization
* fix: mirror lazy subagent metadata budgets
* fix: complete scheduled MCP recovery
* fix: attribute missing MCP tools
* fix: harden scheduled MCP admission
* fix: preserve scheduled admission invariants
* fix: retain inaccessible MCP requirements
* fix: make MCP admission recovery durable
|
||
|
|
65bb64bc68
|
🛟 fix: Recover CodeAPI File Uploads After Throttling (#15784)
* fix: recover CodeAPI uploads after throttling * fix: cover eager CodeAPI uploads * fix: scope CodeAPI upload recovery * test: clarify upload recovery deadlines * fix: isolate CodeAPI recovery state * fix: cancel CodeAPI recovery waits * test: provide CodeAPI registry in handler fixture * fix: propagate CodeAPI recovery controls * fix: bound CodeAPI recovery by run * fix: preserve skill upload cancellation * test: update deferred loader signal contract * fix: preserve recovery cancellation * fix: cancel recovery source reads * fix: classify Axios upload cancellation * fix: preserve lazy provisioning cancellation * fix: close resource recovery boundaries * fix: preserve provisioning failures compatibly |
||
|
|
9f311b810d
|
🪔 feat: Expose Bounded Attached Bash Timeouts (#15776)
* 🕰️ feat: Add BYOM Bash Execution Timeouts * 🔧 fix: Type Bash Timeout Bounds * 🔒 fix: Bound BYOM Bash timeouts by policy * 🧪 test: Propagate BYOM timeout policy fixture |
||
|
|
559fd7d607
|
🪓 feat: Cancel Background Tool Invocations (#15773)
* ✨ feat: Cancel ordinary background tools * 🛡️ fix: Keep cancellation receipts rollout-safe * 🛡️ fix: Harden Background Tool Cancellation * 🧪 test: Cover cancellation config wiring * 🎨 style: Sort cancellation imports * 🧪 test: Align Cancellation Claim Result * 🛑 fix: Complete Background Cancellation Lifecycle * 🎨 style: Align Cancellation Test Formatting |
||
|
|
af6e97f0ae
|
🏠 fix: Keep MCP Caches Fresh on Demand (#15765)
* fix: stop refreshing MCP caches on every chat message * fix: reconcile MCP caches independently of chat turns * fix: scope MCP polling to visible controls * fix: refresh status in the unpinned MCP submenu |
||
|
|
ab6cde0c79
|
⏳ fix: Separate Temporary and General Chat Retention (#15734)
* fix: separate temporary and general chat retention * fix: inherit retention when branching messages * fix: harden retention lifecycle boundaries * fix: preserve subagent retention context * fix: trust stored chat retention for uploads * perf: overlap generated file retention lookup * fix: derive retention from trusted chat state * fix: preserve retention across the full turn lifecycle * test: accept the conversation retention read adapter * fix: carry retention deadlines through durable job saves * fix: resolve retention at cached and new chat admission * fix: resolve direct tool retention before execution * fix: preserve retention across assistant writes and direct tools |
||
|
|
5ca1093ed9
|
🛫 feat: Enable Full Access for BYOM Coding (#15718)
* feat: Enable Full Access for BYOM Coding 🛫
* test: Preserve endpoint policy under Full Access
* fix: Gate Full Access on expected root agent metadata
|
||
|
|
f1bd85498e
|
🪫 fix: Recharge Direct OpenID MCP Bearer Sessions (#15704)
* fix: recover direct OpenID MCP bearer sessions * fix: harden direct bearer recovery lifecycle * fix: close direct bearer lifecycle races * test: align direct bearer checks with current dev * fix: unify direct bearer recovery budget * fix: infer direct bearer recovery from placeholder * fix: preserve bearer precedence and recovery budget * fix: propagate bearer recovery across joiners * fix: preserve direct bearer fallback without session provider * fix: close direct bearer recovery race windows * chore: sort bearer recovery test imports * fix: align bearer recovery with OAuth cleanup lifecycle * fix: fence request-scoped MCP creation during config mutations * fix: retain suspended MCP catalog authentication outcomes * fix: cancel MCP bearer recovery and reuse refreshed credentials * fix: close MCP bearer recovery lifecycle boundaries * fix: preserve ordered catalog authentication rejection * fix: isolate bearer recovery across request and transport boundaries * fix: preserve shared recovery ownership and credential precedence * fix: wait for recovery borrowers and validated OpenID publication * fix: observe the validated browser publication flight * fix: cancel abandoned OpenID publication followers * fix: fence refresh grant admission after lease acquisition |
||
|
|
e370ce4b21
|
🎒 fix: Bound Agent Attachment Context (#15694)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
|
||
|
|
b181ef0232
|
🪬 feat: Configure Langfuse Trace User Identity and Metadata (#15667)
* 🪪 feat: Configurable Langfuse Trace User Identity and Metadata * 🪪 fix: Prefer createRun's conversationId and warn once on an unset userIdField * 🪪 fix: Validate Langfuse trace field names at the point of use * 🧹 chore: Sort identity imports * 🪪 fix: Source the trace model label from the initializer, not a prompt-affecting option * 🧪 fix: Build the trace context without a prototype so partial client contexts resume * 📦 chore: Bump `@librechat/agents` to v3.8.0 for the Langfuse trace userId override |
||
|
|
8f6efaf6ed
|
🛂 feat: Add Configurable BYOM Enrollment Policy (#15673) | ||
|
|
81154adcea
|
🍋 feat: Add AMD Lemonade Provider Recognition and Branding (#15664)
* feat: add AMD Lemonade provider recognition and branding * test: enforce packaged provider asset paths |
||
|
|
aec2121923
|
🔐 feat: Add M2M Authentication for Agent Management (#15555)
* feat: add M2M authentication for Agent Management * fix: Harden Agent Management authentication * fix: Preserve OIDC authentication safeguards * fix: Normalize management principal IDs * fix: Bound OIDC signing key lookups * fix: Allow non-browser Agent Management clients |
||
|
|
f9f1b2fb95
|
✨ v0.8.8-rc2 (#14845)
Some checks failed
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Publish `@librechat/client` to NPM / pack (push) Has been cancelled
Publish `librechat-data-provider` to NPM / pack (push) Has been cancelled
Publish `@librechat/data-schemas` to NPM / pack (push) Has been cancelled
Docker Dev Images Build / publish (push) Has been cancelled
Sync Helm Chart Tags / Ignore non-main push (push) Has been cancelled
Sync Helm Chart Tags / Sync chart tags (push) Has been cancelled
Publish `@librechat/client` to NPM / publish-npm (push) Has been cancelled
Publish `librechat-data-provider` to NPM / publish-npm (push) Has been cancelled
Publish `@librechat/data-schemas` to NPM / publish-npm (push) Has been cancelled
* chore: prepare v0.8.8-rc2 release * test: satisfy mongo meili static checks * test: avoid unsafe logger spy assertion * chore: refresh v0.8.8-rc2 release * test: restore mongo meili test isolation * chore: normalize v0.8.8-rc2 versions * docs: refresh v0.8.8-rc2 release surfaces |
||
|
|
d5b2a8580e
|
📖 feat: Add Claude Fable 5.1 Support (#15512)
Claude Fable 5.1 (`claude-fable-5-1`, GA 2026-09-01) succeeds Claude Fable 5
at the same input/output prices, with a 1M context window and 128K max output.
The Mythos-class helpers added for Fable 5 already match it — `isMythosClassModel`
tests `claude-(?:fable|mythos)[-.]?\d`, so adaptive thinking, summarized-display
opt-in, sampling omission, 1M context, the 128K output clamp, prompt caching,
the Vertex multi-region gate, and the Bedrock PDF exemption all cover 5.1
without a code change. What was missing was registration and pricing.
Cache reads are the one rate that differs, and the substring matcher would have
silently gotten it wrong: `claude-fable-5-1` falls through to the `claude-fable-5`
key, which prices cache hits at the usual 0.1x base input ($1/MTok). Fable 5.1
and Mythos 5.1 price them at 0.025x ($0.25/MTok), so every cache read would have
billed 4x — on exactly the long agentic sessions this model is built for.
- Register `claude-fable-5-1` in the shared Anthropic list and
`global.anthropic.claude-fable-5-1` in the Bedrock list
- Add 1M context / 128K output entries for `claude-fable-5-1` and
`claude-mythos-5-1`
- Add $10/$50 rates and `{ write: 12.5, read: 0.25 }` cache rates for both
- Update the `.env.example` and Vertex `librechat.example.yaml` examples
- Test that 5.1 resolves to its own key rather than collapsing onto 5.0, across
token maps, pricing, cache rates, and Mythos-class detection
The three documented breaking changes from Fable 5 need no handling here:
LibreChat never sends a forced `tool_choice` on the Anthropic path (the agents
SDK omits the param when unset, so the API default `auto` applies); thinking
blocks an earlier model can't read are dropped server-side, unbilled; and
history is rebuilt from stored content parts rather than replayed signed
thinking blocks, so the prefix-binding check has nothing to reject.
Refs #15506
|
||
|
|
057039570b
|
🔭 feat: Audit PII Filter Findings (#15508)
* feat: add audit-only content filter action * refactor: align filter action typing and tests * fix: preserve audit-only filter behavior * fix: enforce audit mode across content paths * fix: honor audit mode for canonical files |
||
|
|
51edb2a660
|
🛫 feat: Enable Self-Service Code Worker Pairing (#15367)
* feat: add self-service code worker pairing * fix: enforce code worker principal boundaries * fix: separate pairing control planes from execution * fix: preserve principal worker routing * fix: pin artifact traffic to code workers * fix: authenticate code worker selection * test: mock worker-aware artifact headers * fix: harden self-service code worker routing * fix: fail closed for principal code pairing * style: format code execution headers * fix: close self-service pairing gaps * fix: satisfy code environment type checks |
||
|
|
a9ccac8656
|
🧲 feat: Enable Secure Attached Environment Pairing (#15355)
* feat: add secure code environment pairing * fix: satisfy code environment type checks * fix: secure code environment administration * fix: isolate code pairing control plane * fix: validate code pairing control responses * fix: secure code pairing transport * fix: validate code pairing wire format * fix: harden pairing secret lookup |
||
|
|
c1cb591d49
|
🗿 feat: Add Attached Stateful Code Environments (#15352)
* feat: add attached stateful code environments * fix: include code environment in lazy agent type * fix: harden stateful environment routing * fix: complete code environment route isolation * fix: declare code environment map type * fix: preserve configured code execution routes * fix: harden stateful environment updates * test: assert route-scoped sandbox readiness * fix: harden stateful environment lifecycle * fix: isolate migrated code sessions * test: assert route-qualified code sessions |
||
|
|
16c2bb4149
|
📬 feat: Establish Background Continuation Admission (#15348)
* feat: establish background continuation admission * fix: preserve routed subagent wakeup controls * fix: probe configured subagent task store |
||
|
|
a5fd00cd7a
|
🎼 feat: Unify Agent Turn Execution Behind One Immutable Plan (#15296)
* feat: unify agent turn execution planning * fix: preserve automatic subagent activity handles * fix: gate subagent wakeups for ephemeral parents * fix: scope subagent wakeup guidance by agent |
||
|
|
03fa98eb37
|
🕸️ feat: Complete Keenable Web Search Provider (#15288)
* feat: wire Keenable web-search provider into config, schema, and UI Keenable landed as a search provider in @librechat/agents (#285, shipped in 3.2.58+), but LibreChat did not yet expose it. This adds the config/schema/UI glue so it can be selected, mirroring the existing Tavily provider. - data-provider: add `keenable` to SearchProvider type + SearchProviders enum, keenableApiKey/keenableApiUrl schema fields, and a keenableSearchOptions block (maxResults, site, attributionTitle, timeout). - data-schemas: register keenable in webSearchAuth.providers and default the key/URL placeholders in loadWebSearchConfig. - api/web: pass keenableSearchOptions through to the provider and handle Keenable's keyless model. Unlike other providers it authenticates with no key (the public endpoint), picking up an optional key/URL when set; the URL override is SSRF-preflighted like other user-provided URLs. - client: add Keenable to the provider dropdown with an optional API-key input. - docs: document KEENABLE_API_KEY/KEENABLE_API_URL in .env.example and a webSearch example in librechat.example.yaml. - tests: keyless + keyed auth resolution, config defaults, and schema parsing. * fix: ESLint no-unused-vars and clarify Keenable yaml example - Remove the now-unused RerankerTypes import in data-schemas web.ts (the lint job runs with --max-warnings 0 on changed files, so this latent warning failed CI once the file was touched). - Note in the librechat.example.yaml Keenable stanza that a scraper (and reranker) is still required for web search to load, and include a Firecrawl scraper in the example. * chore: fix import order drift (sort-imports) * feat: add Keenable as a keyless scraper and select it without a pinned provider The Keenable scraper landed in @librechat/agents#337, so wire the scraper category the same way the search provider already is: `scraperProvider: keenable` reads pages through Keenable's public fetch endpoint with no key (a key only lifts rate limits, and the endpoint is overridden with KEENABLE_FETCH_URL). Paired with `rerankerType: none` this makes a fully keyless web-search stack possible for the first time. Also closes the Codex finding on this PR: because none of Keenable's auth fields are required, the generic auth loop skips it whenever it isn't pinned, so a key submitted through the API-key dialog (which cannot pin a provider) left the providers category unauthenticated. Keenable is now selected in that case, gated on one of its values actually being present so installs that configured nothing keep their current behavior. The scraper gets the same fallback, additionally gated on Keenable being the resolved search provider, so it never silently scrapes for another provider. * fix: select the Keenable scraper from a supplied key, not only for Keenable search The API-key dialog submits credentials and cannot pin a provider, so choosing Keenable as the scraper while search stays on Serper/SearXNG/Tavily had no effect: the unpinned-scraper fallback required Keenable to also be the resolved search provider. A supplied Keenable value now triggers it as well, which is the only signal the dialog can send. The fallback still runs only when no keyed scraper authenticated, and with neither trigger the category stays unauthenticated, so a deployment that never configured Keenable is unaffected. Note the fully keyless choice still cannot be expressed through the dialog: Keenable's key is optional, so picking it with no key submits nothing at all. librechat.example.yaml now documents pinning scraperProvider: keenable for that case. * style: Sort Keenable imports * fix: Harden Keenable auth resolution * fix: Preserve Keenable selection intent * fix: Fail closed on invalid web search auth * fix: close keenable auth gaps * style: sort web auth imports * fix: preserve web search selection integrity * fix: isolate web search auth ordering * fix: silence expected credential misses * chore: bump agents sdk * fix: preserve web search preference ownership * fix: forward cleared Keenable endpoint * style: sort web search hook imports * fix: require intent for credential clears --------- Co-authored-by: Ilya Bogin <ilya.bogin@keenable.ai> |
||
|
|
8ba76507ae
|
🧷 fix: Verify Scheduled HITL State Before Pausing (#15284)
Some checks failed
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
Publish `librechat-data-provider` to NPM / pack (push) Has been cancelled
Publish `librechat-data-provider` to NPM / publish-npm (push) Has been cancelled
* fix: require durable storage for scheduled HITL * fix: verify scheduled HITL checkpoints before pause * fix: narrow scheduled HITL capability checks * fix: honor ask tool filters in schedule preflight * fix: enforce durability across scheduled HITL resumes * fix: scope scheduled HITL admission to active tools * refactor: move scheduled HITL admission to TypeScript * fix: close scheduled HITL admission gaps * fix: bind scheduled pauses to current checkpoint * fix: cap pending actions at binding deadline * fix: reject expired pending actions * fix: guard approval deadlines atomically |
||
|
|
52fa88fc54
|
📬 feat: Serialize Bound Actor Event Mailboxes (#15260)
* feat: serialize bound actor event mailboxes * test: harden actor mailbox batch ordering * fix: treat batch roots as mailbox authority * test: pin mailbox replay semantics * fix: preserve mailbox authority through settlement |
||
|
|
68fc46a055
|
🪢 feat: Resume Bound Event Actors from Checkpoint Forks (#15227)
* feat: resume event actors from checkpoint forks * fix: fence event actor checkpoint uncertainty * fix: satisfy event actor type contracts * fix: make actor reconciliation recoverable * fix: preserve event actor lifecycle transitions * fix: fence event actor lifecycle outcomes * fix: enforce event actor lifecycle ownership * fix: retain event actor settlement proof * 🔒 fix: Retain Event Actor Receipts Through Repair and Bound Their Journal Repair and compensation deleted the reconciliation row they resolved, which was the only durable proof that the invocation had already applied an external action. A delayed duplicate owner could then reacquire the same invocation id and repeat that action. Both resolutions now retire their receipt to `settled` and record how it settled, so the same-id tombstone survives; `history_repaired` and `action_compensated` still force a cold rebuild. Compensation undoes the effect without re-authorizing the delivery, so a legitimate retry must arrive under a new invocation id. A retried repair converges on its own receipt. Bound the journal so a long-lived actor cannot grow its conversation document without limit: a new fence is admitted only when no active lifecycle row exists, so a capped push can evict nothing but the oldest settled receipts. Stop shipping the unbounded source payload on every bound-child continuation. It rode the delivery body regardless of the feature flag while the sibling `fire` body deliberately sends event identity alone, so a large webhook payload could push a previously working delivery past the chat route's body limit. The actor binds an invocation from identity and never builds the prompt from it. Blank the positional token map on warm continuations. It is derived from the full DB history, while a warm run executes on checkpoint-restored state, so its indices address different messages and the pruner never recounts them — misattributing cached counts to the wrong messages in both directions. Keep the replaced-claim exit on its cleanup path when preserving reconciliation fails: the committing CAS already left a blocking row, so the failed status upgrade costs provenance, not safety. * 🧪 test: Pin the Warm Continuation's Map/Summary Asymmetry Give the warm-continuation client test a populated token map and a real cross-run summary so its assertions bite: the positional map must arrive blank (checkpoint-restored state no longer matches DB-derived indices, and the pruner never recounts a populated entry) while `initialSummary` must pass through unchanged — it rides the system tail and summarizes pre-boundary turns that were excluded from the very history the committed checkpoint was built from, so blanking it would silently drop context no warm run can recover. * ⚖️ fix: Honor Compensation in Settlement and Age-Bound the Receipt Journal A compensated receipt still tombstones its invocation id, but its external effect was explicitly undone — the terminal handler nonetheless replayed every settled lifecycle's stored action as authoritative and settled the public outcome as applied, telling an action-aware source the operation stands and suppressing the new-invocation retry compensation requires. The handler now settles a compensated invocation as failed with an explicit compensation error, overriding even fresh applied run evidence from a replayed generation; verified and repaired receipts continue to replay applied. Receipt eviction is now primarily age-based: a stale same-id owner is bounded by time, not by how many newer invocations settle, so the previous count-only slice let a high-rate actor evict a tombstone while its delayed duplicate owner could still wake and repeat the action. Admission prunes only settled receipts older than a retention window that dwarfs every generation, job, and delivery-retry lifetime, and the count cap is demoted to a raised document-size backstop. * 🔀 fix: Serialize Compensation Against Settlement and Never Evict Live Receipts The terminal handler read its lifecycle snapshot, verified history, and then settled the public outcome — so a compensation resolving the same receipt during that window lost: the handler settled applied from its stale snapshot and no retry could ever change the replay-identity-locked outcome. The receipt's status CAS is now the serialization point: verification resolves the receipt BEFORE settling, whichever transition wins determines the public outcome, and a crash between resolve and settle converges through the retained receipt's replay. The verified-replay probe requires the receipt's own resolution, so a compensated receipt can never satisfy a verification retry. This inverts the settle-before-receipt ordering deliberately: that ordering guarded proof that resolution used to delete, and the receipt now retains its full action proof through resolution. The document-size cap is no longer an eviction quota. A receipt inside its retention window is never discarded: when the journal holds a full cap of unexpired receipts, new invocations are refused fail-closed until receipts age out, making duplicate protection and document integrity simultaneous invariants instead of a rate-dependent trade. * 🎓 fix: Keep Skill-Bearing Event Actors on the Legacy Path Skill primes are spliced into the message list directly ahead of the newest message, and a warm continuation forwards only that newest message — so a checkpoint-restored actor would keep serving the prime bodies baked in at its last cold start and never observe an edited or newly attached skill. Until the actor head carries a context fingerprint that forces a cold rebuild when the agent's skill context changes, agents with always-apply or manual skill primes stay on the legacy path, which re-primes fresh bodies every turn: correct on every event, just never warm. * 📜 fix: Gate Fork Mode on the Skills Capability, Not Just Request-Time Primes History-derived re-priming was a third path into the same staleness class: an actor that previously invoked a skill carries no request-time prime arrays, yet primeInvokedSkills re-resolves that skill's current body from history each turn and the warm slice drops the reconstruction — leaving the checkpoint's old body active after edits. The fork gate now keys on the priming hook itself (present exactly when the skills capability is enabled) alongside the request-time arrays, so every skill-body path routes to the legacy rebuild until #15235's context fingerprint restores warm continuation for skill-bearing actors. * 🧾 fix: Capture Applied-Action Proof at Tool Execution, Not After sendMessage The executor read applied-action evidence from the run-step collection the instant sendMessage resolved, but that collection is populated asynchronously — an applied invocation could classify as actionless (runSteps still empty while the tool result already streamed), discarding its fork and stranding the actor cold while the terminal handler later settled the same delivery as applied from the persisted evidence. Authoritative proof is now recorded in graph context the moment the expected tool executes: the request-owned recorder observes the tool-end chain (which ToolNode dispatches synchronously with both input and output) and applies the same fences as run-step evidence — exact tool name with the MCP-suffixed form, the declared argument subset against the execution input, an error-free result, and the background non-execution receipt exclusion. readAppliedAction consults the receipt first; run-step inspection remains the fallback for paths that bypass the tool-end chain. Regression coverage reproduces the observed ordering: the real executor commits the head from the receipt while run steps are empty, warm-continues the next event, and never re-executes the action; recorder fences and the receipt-first controller wiring are covered separately. * 🎯 fix: Supply Execution Arguments to the Tool End Callback The live Vertex + MCP canary exposed a contract mismatch the synthetic fixtures hid: the ON_TOOL_EXECUTE execution path invoked its tool end callback with output only, while the action recorder must verify the declared argument subset against the execution input. The receipt never qualified, every turn fell back to cold history rebuilds, and the tournament advanced with zero actor heads and zero retained checkpoints while looking successful. The execution handler owns both halves at the same moment, so the fix is at the source rather than a correlation store: ToolEndCallbackData gains the executed call's input and every callback site passes tc.args. A handler-level regression drives the real createToolExecuteHandler and asserts the callback receives both fields; recorder regressions pin the production shapes — an output-only tool end must starve an argument-fenced receipt rather than trust an unfenced match, and still qualifies a name-only expected action. * 🕵️ fix: Mark Background Deliveries So They Cannot Impersonate Applied Actions The background-claim callback reports the ORIGINAL tool's name for artifact attribution on the poll turn that harvests a completed task. A name-only expected action could therefore be impersonated by work some earlier turn dispatched: the recorder would attribute that delivery to the current invocation and commit a head whose state never contained the invocation's own action. The run-step evidence path never had this hole — it sees the poll tool's name — so the recorder must match its provenance discipline. Delivery callbacks now carry an explicit backgroundDelivery marker set at the one site that rewrites the name, and the recorder ignores marked deliveries outright. Regressions pin both halves of the contract: the delivery callback must carry the marker with the poll call's arguments, and a marked delivery can never qualify even a name-only expected action. * 🧿 fix: Version Invalidations, Keep Evidence Ahead of Output Policy, Gate Detachable Actions Three closeout-round findings, each converted into an invariant. Every legacy-path invalidation now advances a durable epoch — including for headless and already cold-marked actors, where the marker alone leaves no CAS-visible trace — and the actor-head CAS requires the epoch observed at preparation. A concurrently prepared fork whose history predates an intervening legacy turn can no longer commit past it; the commit reports an ordinary conflict and journals. Execution identity is now emitted before post-execution output policy: when a side-effecting tool succeeds but its returned content is withheld by the output filter, the callback delivers an outputFiltered receipt with blank content — the recorder accepts it as proof (rejecting model-detached calls it cannot distinguish through the blank shape), the artifact path never sees it, and an applied action is no longer reclassified actionless and re-executed on retry. Background-capable expected actions stay off the fork path: dispatch returns a launch handle every evidence fence correctly rejects, and the completion is provenance-marked as another turn's work, so a fork would settle actionless before the external effect lands with nothing to stop a retry from dispatching it again. The gate mirrors the MCP-suffix name matching of the evidence path. * 🚧 fix: Seal the Whole Legacy Turn Behind a Second Epoch Advance The epoch fenced only the legacy turn's start: a fork preparing after the begin invalidation but before the turn's message persistence observed the new epoch and cold marker, rebuilt from history that did not yet contain the turn, and committed cleanly because nothing advanced the epoch again — making the incomplete rebuild authoritative and clearing the marker. Every legacy event turn now seals its invalidation at terminal persistence with a second epoch advance, on the success, replaced-claim, and error exits alike. Sealing deliberately carries no quiescence requirement — it must succeed while a fork fence is active, because that is exactly the mid-turn race it defeats — and a fork that already committed against the begin epoch is healed the same way: the seal re-marks the head cold, so the next event rebuilds with complete history. Seal failure never diverts the turn's own exit; the begin bump still fences everything prepared before the turn. * 🔗 fix: Replace the Best-Effort Epoch Bump With a Durable Legacy-Turn Fence The second epoch advance could not make a legacy turn atomic, and three findings shared that root cause: two conditional updates left a headless gap a fork could create the head inside; the error exit sealed before saveErrorTurn made the error history durable; and any crash or failure between persistence and sealing left an incomplete fork authoritative, because the seal was best-effort and its failure was swallowed. A legacy turn now carries one durable fence. A token is written before execution by a single update-pipeline write — no two-write gap, and the cold marker is applied only where a head exists via $cond/$$REMOVE. While the token is present no fork may prepare (the adapter refuses) or commit (the CAS requires its absence), because the turn's messages are not yet durable. One atomic write clears the exact token and advances the epoch once history is persisted — after saveErrorTurn on the error exit — and success, replacement, and error exits all route through it. Failure is now fail-closed rather than silent: a failed seal leaves the token set, which keeps blocking forks and is logged as such, and a fence abandoned by a crashed turn is reclaimed only once stale, advancing the epoch and marking any head cold so the next event rebuilds from whatever history actually survived. * fix: serialize legacy event actor turns * fix: close legacy actor fence ownership gaps * fix: preserve legacy actor persistence fences |
||
|
|
8969ee4b18
|
🎚️ feat: Configure Agent Event Runtime in YAML (#15128) | ||
|
|
67b7b441b2
|
🛂 feat: Filter Model-Bound Content by Source (#14425)
* feat: introduce optional content protection seam * feat: enforce source-aware content filters * feat: complete source-aware content enforcement * test: activate skill file-text fail-close fixtures * fix: harden source-aware content filters * fix: harden model-bound content filtering * fix: preserve legacy filters and generated files * fix: inspect shared scalar metadata * test: align mocks with current dev dependencies * feat: add persisted content filter safeguards * feat: complete source-aware content filter enforcement * fix: move resume content preflight into TypeScript * fix: close content inspection edge cases * fix: harden content protection boundaries * fix: complete content protection safeguards * test: align persisted memory filter coverage * fix: reconcile content protection with current dev * fix: reconcile content protection with latest dev * fix: close content protection review gaps * fix: enforce source-aware provider boundaries * fix: preserve legacy PII preflight semantics * test: stabilize stored branch preflight fixture * fix: defer agent writes until protected model admission * perf: harden source-aware model-bound filtering * fix: canonicalize provider lineage before validation * fix: satisfy model-bound callback type checks * perf: Bound content protection filtering work * fix: Bound submission array traversal * fix: Stabilize bounded content snapshots * fix: Scope model-bound traversal overflows * fix: Preserve scoped content inspection * fix: Accumulate aggregate traversal scopes * fix: centralize content policy boundaries * test: align deferred tool policy context * test: align controller policy mocks * style: normalize content protection imports * fix: close content policy review gaps * fix: narrow active skill policy config * fix: address content protection review boundaries * fix: retain exact provenance overflow sentinel * fix: preserve literal and scoped provenance updates * fix: narrow persisted edit provenance * fix: isolate exact overflow attribution * fix: centralize stored prompt protection * fix: fail closed on incomplete transcript evidence * fix: align canonical transcript routing * refactor: centralize content policy preflights * fix: isolate upload policy error typing * style: sort policy preflight imports * refactor: centralize content policy boundaries |
||
|
|
33e42e6d5d
|
🎛️ feat: Configurable SearXNG Search Options (#14987)
* feat: configurable SearXNG search options SearXNG queries were hardcoded to google,bing,duckduckgo with no way to change the engine list, the result language, or the request timeout. Most self-hosted instances get served CAPTCHAs by DuckDuckGo, so a third of every query silently returns nothing and operators have no lever to pull. Add a searxngSearchOptions block to the webSearch config that accepts engines (as a comma-separated string or a list), language, timeRange, and timeout, and thread it through to the search tool. Engines are normalized to the comma-separated form SearXNG expects, with blank entries dropped so a stray comma cannot produce an empty engines parameter. Refs #14117 * fix: normalize SearXNG engines on the runtime config path The engines transform lived only on the zod schema, but loadCustomConfig returns the raw YAML object rather than result.data, so nothing downstream ever saw the transformed value. A YAML list reached the SDK as an array and threw "options?.engines?.trim is not a function" when the search tool was built, taking web search down entirely for the exact block the example yaml documents. An untrimmed string reached SearXNG with spaces still in it. Extract the normalization into normalizeSearxngEngines and apply it in loadWebSearchConfig as well as the schema, so both the parsed and the raw path produce the same comma-separated value. Widen the loader's parameter to TWebSearchConfigInput, which models engines as the list or string an operator actually writes, and cover the raw path with tests that call the loader rather than the schema. * chore: drop unused RerankerTypes import in web config loader |
||
|
|
29f6ec6eae
|
🙈 feat: Config Option to Hide Response Feedback Buttons (#15085)
* feat: add interface option to hide response feedback buttons Adds `interface.feedback` to librechat.yaml. When set to false, the thumbs up/thumbs down buttons are removed from the message action row and the feedback endpoint rejects writes with 403, so deployments that do not consume the data can stop collecting it. Defaults to true. * refactor: hoist the feedback gate out of the message row and into typed middleware Reading startup config inside HoverButtons put a query observer and two Recoil subscriptions on every message row, and rows never unmount, so the cost grew with the conversation. Resolve the flag once per chat in useChatHelpers and carry it on TMessageChatContext; useMessageActions withholds handleFeedback when it is off, which the action row already treats as "no feedback controls". The flag now stays false until the config resolves, so a disabled deployment never flashes controls whose writes are rejected. Move the server-side policy into requireFeedbackEnabled under packages/api so the route keeps no policy of its own. * test: stub the feedback gate in specs that replace the api package The messages router now imports requireFeedbackEnabled, and express rejects an undefined handler at require time, so every spec that mocks @librechat/api wholesale has to carry the export. |
||
|
|
8c14f03432
|
🗂️ feat: Scope Scheduled Chats to Chat Projects (#15056)
* 🗂️ feat: Scope Scheduled Chats to Chat Projects Adds an optional chat-project destination to a schedule, plus the operator config to require one — or to pin every scheduled run to a specific project. Feature: - `chatProjectId` on the schedule row, accepted on create/update, projected on the wire, and carried into the run's conversation through the durable trigger envelope's `run` context. - `interface.schedules.requireProject` refuses schedules that are not filed under a project; `interface.schedules.projectId` pins every run to one project and implies the requirement. - Dialog gains a project picker (required when configured, a read-only row when pinned); the card shows the destination and the new disabled reasons. Invariants: - ONE resolver (`resolveScheduleProjectId`) decides the destination for the write handler, the fire path, and the wire projection alike, and an operator pin OUTRANKS the stored id in all three. Tightening the config therefore redirects — or stops — existing schedules instead of grandfathering where their runs land. A pin implies `requireProject` for the same reason: without it, a row created before the pin would keep firing with no project at all. - Create/fire precheck symmetry, mirroring `resolveAgentFireAccess`: a write this handler accepts is one the next fire also accepts. Any edit leaving a schedule ENABLED re-validates its EFFECTIVE (possibly stored) project, like the existing stored-agent and cadence-floor rechecks. A DISABLING edit skips the requirement, or a schedule auto-disabled for `project_required` could never be turned off. - Fire-time enforcement auto-disables rather than filing runs loose, matching agent_deleted: new `project_required` (requirement raised after creation) and `project_deleted` (gone, or pinned to a project this owner does not have) reasons, both refused BEFORE a billed generation is dispatched, and both advancing so a schedule can never wedge on the occurrence. - `computeCreateDigest` appends the field only when present, so a payload without a project digests byte-identically to one from before this change — a create retried across the upgrade still matches its own row instead of reading as key reuse. - Project reads are scoped to the owner, so ownership and existence are the same lookup; a read ERROR propagates instead of failing closed, so a Mongo blip retries the fire rather than auto-disabling the schedule. The trigger idempotency key hashes principal/event/target and never `envelope.run`, so the added run field cannot destabilize delivery identity. * 🗜️ fix: Keep the Schedule Dialog Inside Its Height Budget The project picker landed as a new ROW in the schedule dialog, which broke the e2e edit spec: `md:overflow-visible` turns off the template's scrolling from `md` up, so the dialog's content must fit the viewport. The extra row pushed the footer's Save button below a 720x1280 window, where Playwright reported a visible, enabled button it could never click — 226 scroll-into-view retries and a 2-minute timeout, on all three attempts. Measured against `dev` at 1280x720 (Save button's bottom edge, viewport 720): dev 688 (32px slack) project row ~790 (off-screen, CI failure) 3-column row 704 (16px slack — half the budget spent) this commit 686 (34px slack, 2px better than dev) The identity row is now three columns — name, agent, project — and its caption moved out of the agent cell to sit full width beneath the row: at a third of the dialog that sentence wraps an extra line, and the row is the tallest thing competing for the budget. The caption is grouped with the row rather than left to the form's own 4-unit rhythm, which spent more height on the gap than the caption occupies. The e2e spec now asserts the button is in the viewport before clicking it, so the next field that overflows this dialog says so in one line instead of a two-minute timeout on a visible element. FOLLOWUPS.md records what the planned dialog controls (multi-day weekly, timezone, attachments) need first: give `ControlCombobox` the `portalElement` prop `Dropdown` already has, portal the popovers into the dialog content, and let the form scroll again. * 🧹 fix: Address Codex Review on Scheduled Chat Project Scope Four P2 findings, all real. Unreachable clearing path (ScheduleDialog). The picker only held live projects, so `com_ui_schedule_project_none` was a PLACEHOLDER — nothing selectable. Once a schedule had a project the owner could never take it away, leaving the server's `chatProjectId: null` path reachable only by API. The picker now carries a real "No project" option whenever a project is optional, and omits it when one is required, where there is nothing valid to select. Placeholder shown for a real project (ScheduleDialog). A stored or pinned project outside the first loaded page had no name in the paged map, and the combobox renders its placeholder for an empty display value — telling the owner a scoped schedule had no project. That one project is now read by id, with the raw id as a last resort: a poor label, but an honest one. Project policy skipped at the resume boundary (service.ts). `claimScheduleResume` re-applied the schedules gate, the revision fence, the kill switch and SCHEDULES:USE, but not the project policy this PR added. Approving a paused run whose project was deleted — or whose owner now sits under a requirement or a pin it no longer satisfies — billed a continuation the very next scheduled fire would refuse and auto-disable the schedule for. The effective-project resolution now runs there too, refused before the lease and the capacity slot so a policy refusal costs nothing and leaves no state to unwind. NOTE: agent access and balance are still not rechecked on resume; that gap predates this PR and is left alone. Per-card project derivation (ScheduleCard). Every card ran the projects hook and rebuilt the full option array, name map, and one icon element per project, to use a single name — O(schedules x projects) per render and per project-list refresh. The hook is split: `useChatProjectNames` (map only, for the panel, which resolves every card's name once and passes it down) and `useChatProjectPicker` (options and pagination, for the dialog's one combobox). The panel skips the query entirely until some schedule actually has a scope. Tests: four at the resume boundary (verified failing without the gate) and four in the dialog spec. The picker selection in one existing test now goes through the search field — the popover's VIRTUALIZED renderer sizes its window from a scroll height jsdom always reports as 0, so with three options it materialized only two. Full schedules e2e re-run green against the rebuilt client. * 🎯 fix: Settle Project-Policy Refusals and Keep Create Retries Idempotent Second Codex round, four P2s. Three were consequences of the resume gate added in the previous commit, which was half-built: it admitted where it should not and stranded the run where it refused. Project policy moves from `claimScheduleResume` into `isScheduleLive`'s `policy` branch. Both entry points consult that branch FIRST, and both already route its refusal through abort-and-settle — so a policy stop now settles the occurrence instead of answering a bare 409 while the job stays `requires_action`, the card keeps reading "Needs approval", and every retry repeats the same 409 until expiry. No change to resume.js: the existing branch does the work. The rule is deliberately NARROW. It refuses only where no valid destination is left — the requirement is on with nothing satisfying it, or the schedule's own project is gone (which also unset it on the conversation). It does NOT refuse because an operator's pin moved: the paused conversation cannot be rebound (`chatProjectId` is excluded from the resume context and the continuation reuses the same conversationId), so refusing would strand a pending approval over a destination it can never reach, for a pin that governs only where the NEXT run lands — which the fire path already redirects. Create retries are idempotent again. Project policy had been applied BEFORE the `clientRequestId` replay lookup, so a raised requirement, a deleted project, or a moved pin could answer 400 for a create that already committed — pushing the client to rotate its key and create a DUPLICATE schedule, the exact failure the key exists to prevent. Policy now applies only to a genuinely new insert, and the digest is computed from the CLIENT's payload rather than the resolved destination, so today's policy can no longer re-digest a genuine retry into a mismatch. An explicit `chatProjectId: null` under a pin is refused rather than silently resolved to the pin. The payload contract defines `null` as clearing the scope; answering 201 while filing under the pin reported success for the opposite of what was asked. Only an OMITTED field takes the pin silently. Tests: five on the policy branch (both refusals verified failing without it, plus guards that a moved pin and a live project still admit) and two on the handlers (the pinned explicit clear, and a committed create recovered by retry after the policy tightened — verified answering 400 without the reordering). * 🧭 fix: Converge the Stored Project on the Destination a Fire Resolved Third Codex round, four P2s. The root confusion behind the resume findings: an operator pin outranks the stored id at fire time, `fireSchedule` sends the pin in the trigger envelope, and the row keeps its old value. The row therefore LIED about where that occurrence's conversation went, and every later re-validation — the resume boundary above all — checked a project the conversation was never filed under. A schedule storing A, pinned to B, with B later deleted and A still live, was admitted for resume into a conversation that had just been unscoped. Fixed at the source rather than at each reader: a fire that resolves a destination different from the stored one writes it back, claim-token fenced like every other worker-side write and deliberately WITHOUT a configRevision bump — this is the server reconciling itself to policy, not an owner edit, and a bump would fence an in-flight occurrence off its own run. Written only AFTER the destination validates, so an unusable pin never lands in the row, and best-effort: the envelope already carries the right destination, so a failed write costs accuracy on a later recheck, never the run itself. The wire projection already reported the pin, so this also stops the row and the UI disagreeing. An explicit `chatProjectId: null` under a pin is now refused on the DISABLING edit path too. The pin check and the requirement are independent rules, and folding them together let `{enabled: false, chatProjectId: null}` skip the pin check entirely, unset the row, and answer with a wire projection still naming the pin. Only the requirement is waived for a disabling edit. The dialog no longer requires a project for an edit that leaves a schedule DISABLED. The server waives the requirement there precisely so a row auto-disabled for `project_required` can still be renamed or tidied up; requiring it in the form made that unreachable, and an owner with no projects could not edit the stopped schedule at all. FOLLOWUPS.md records the two residual gaps with their exact triggers: the sub-second deletion race inside the resume claim window (which needs the effective project persisted per OCCURRENCE plus a distinct policy conflict routed through abort-and-settle), and the fact that convergence happens only when a schedule fires. Tests: four on convergence (pin written, no write when unchanged, no write for a destination that failed validation, fire survives a failed write), two on the disabling-edit pin rules, two in the dialog. Schedules e2e re-run green. * 🔑 fix: Keep an Explicit Project Clear Out of an Omitted Field's Digest `computeCreateDigest` appended `chatProjectId` on `!= null`, so an OMITTED field and an explicit `null` produced the same digest. Because the replay lookup and `matchesCreateIntent` deliberately run before project policy, a request could reuse a pinned create's `clientRequestId` while explicitly sending `chatProjectId: null` and receive 201 describing the pinned row — success reported for the opposite of what it asked, and the pinned-clear refusal the normal create path applies never reached. Now `!== undefined`: an omitted field still digests byte-identically to a payload from before project scope existed, so a create in flight across the upgrade still matches its own row, while an explicit clear is a distinct intent and digests differently. A pre-scope client never sent the field at all, so nothing legacy can carry an explicit null. * 📍 fix: Validate a Paused Run Against the Project Its Own Occurrence Used The schedule-wide convergence from the previous commit was not enough, and the reason is the single-active run index: it covers `status: 'started'` only, so a PAUSED run does not block the next occurrence. While run 1 sat paused in project A, a pin move plus one later fire rewrote the schedule row to B — and the resume policy then validated B while run 1's conversation was still filed under A. Delete A and the continuation was admitted into a conversation that had just been unscoped. That window lasts as long as the pause, not the sub-second race the previous commit documented. The reservation now records the destination THIS occurrence used, and `isScheduleLive` validates that record when given the occurrence's `scheduledFor` — which `resume.js` already reads two lines above the call. No new conflict type and no settlement-path surgery: the refusal rides the abort-and-settle branch that check already has. An ABSENT record falls back to the schedule-level resolution rather than reading as unscoped. A pre-scope occurrence, or one whose row is gone, must never be treated as evidence to stop a run — the fallback keeps legacy paused runs behaving exactly as they do today. Schedule-wide convergence stays: it keeps the row honest for the UI and for every check that has no occurrence in hand. FOLLOWUPS.md now describes the one remaining gap accurately — a deletion inside the claim window, which needs a distinct policy conflict routed through abort-and-settle rather than the bare 409 an `inactive` conflict produces. Tests: three on occurrence-vs-row precedence (the decisive one verified failing without the lookup), two on what the reservation records, and the resume controller spec now pins `scheduledFor` in the policy call. Schedules e2e re-run green. * 🏷️ fix: Tell a Deliberately Unscoped Occurrence From an Unrecorded One The occurrence fallback added in the previous commit conflated two different absences. A post-upgrade run that deliberately went unscoped omitted the field exactly like a row written before the field existed, so both took the fallback — and a paused unscoped run was then validated against the schedule's CURRENT project. Under a requirement or a pin added while it sat paused, that admitted a billed continuation into a conversation satisfying no present policy. The reservation now ALWAYS records its decision, `null` for unscoped, and the read reports `recorded` from key PRESENCE rather than truthiness. Only an unknown record — a pre-scope row, or no row at all — falls back to the schedule-level resolution; a recorded null is the genuinely unscoped occurrence it says it is, and is refused once a project becomes required. The distinction rests entirely on a stored `null` surviving as a present key while a never-written field stays absent, so that is asserted against real Mongo rather than assumed: if it ever stopped holding, unscoped runs would silently start being validated against the schedule's current project again. Tests: two against mongodb-memory-server (recorded null vs never-written vs missing row), plus refusal of a recorded-unscoped occurrence under a new requirement and the preserved fallback for a pre-scope one. Schedules e2e green. * 🔁 fix: Validate an Initial Scheduled Start Against Its Own Occurrence The initial-start policy check in `request.js` called `isScheduleLive(..., { policy: true })` without `scheduledFor`, so it fell back to the schedule-level resolution even though the run row already carries the occurrence's recorded scope. An occurrence reserved unscoped, with a pin introduced while its loopback request sat queued, was therefore admitted against the new pin — producing a billed unscoped conversation under a requirement it does not satisfy, from an envelope already built without a project. `scheduledFor` was already in scope there. Passing it makes the initial start and the resume validate the same way: against the destination the occurrence itself recorded. |
||
|
|
c5276fc63d
|
⏱️ feat: Run Scheduled Chats Through Durable Agent Triggers (#14939)
* feat: Scheduled Chats — agent-centric scheduled runs creating real conversations
Squash of the full review-hardened branch (PR 14540, supersedes 14373) onto
latest dev, preserving the exact verified tree. History prior to this commit
lived on the pre-squash branch; every invariant below survived 25 Codex review
rounds plus two external audit rounds (R26) with regression tests that fail
without their fixes.
Feature:
- Schedules CRUD + side-panel UI (cadence dialog, run cards, Run Now), roles/
permissions (SCHEDULES:USE), interface.schedules availability, per-user limits
and capacity slots, timezone-aware cadence with DST-conservative floors and
misfire grace.
- Engine: single-process claim/fire loop with leases, loopback POST dispatch
(signed schedule-fire JWT claims, per-occurrence idempotency key inside the
route's clientRequestId charset), overlap/balance/capacity/duplicate skip
policies with auto-disable streaks (too_many_failures, insufficient_balance),
reconciliation from retained terminal-job evidence, erasure sweep.
- Scheduled runs create real conversations through the resumable agents chat
path: HITL pauses surface on the card (requires_action), resumes re-apply the
fire boundary's admission policy (revision fence, enabled, global kill switch,
SCHEDULES:USE, availability) before continuing a billed generation.
Correctness invariants (the audit surface):
- Settlement discipline: every persistence-producing write happens-before a
run's terminal outcome write; Stop/complete/pause race through single-winner
terminal CAS claims (dev's TerminalJobClaim substrate) with retained,
completedAt-less evidence for scheduled fires plus an owner-intended outcome
stamp (scheduleOutcome) the reconciler prefers over re-derived success —
round-tripped through the Redis hash mapper.
- Swallowed generation failures (client error content parts) classify to
error/skipped_balance instead of success on both initial and resumed paths;
stale stamps are refreshed evidence-first when persistence plus the Mongo
outcome write both fail.
- Abort honesty: delivery judged by generation ownership and the CAS's actual
from-status; republication escalates to the transport's acknowledged variant;
the Stop route settles only genuinely paused runs.
- Account deletion: one-way barrier (deletionRequestedAt) with auth-cache
tombstone-before-stamp, boundary rechecks across all auth strategies, quiesce
of scheduled + interactive work with durable per-stream abort fences
(positive-evidence acknowledgement only), owner-side finalization markers for
post-terminal billed writes, deferred-deletion sweep (explicitly ensured
partial index) that completes cascades autonomously. Remote OpenAI-compatible/
Responses requests are documented as outside the quiesce and tracked in issue
14594.
- Store compatibility: finalization markers optional on the legacy IJobStore
contract with coherent degradation (registration fails -> synchronous title
fallback; count reads 0).
* fix: fence terminal response persistence from deletion; retain stale-pause evidence
Two blockers from the third external review round.
Terminal persistence visible to account deletion:
- The finalization marker was registered only when post-terminal TITLE work was
possible, but every persistence-owning terminal CAS opens the same window: the
claim drops the job out of the active set BEFORE the response save (and the
background user-message/convo saves), so a deletion quiesce landing there saw
neither an active job nor a marker and could cascade while the admitted request
could still recreate messages. Both controllers now register the marker before
every persistence-owning claim — the fresh-turn path and the HITL resume — and
release it only after their pending saves (and any post-terminal title) have
landed, on success and failure paths alike. The TTL bounds a crash.
- settleAbortFence no longer clears a complete/error fence while
`terminalPersistencePending` is set: terminal at the CAS is not settled while
the owner is still persisting. The job facade now surfaces the flag.
- The marker trio is REQUIRED by the runtime store contract (assertJobStoreV2
refuses a store without it at configure time, keeping the failure loud and
deterministic) while remaining optional on the legacy public IJobStore type for
source compatibility. The silent degrade path from the previous round is gone —
it was not deletion-safe.
Stale-pause recovery retains scheduled evidence:
- Three crash/timeout recovery paths — ApprovalLifecycle.failStalePausePersistence
and the InMemory/Redis stale-pause cleanups — unconditionally stamped
`completedAt`, putting a scheduled fire's failed-pause error terminal on the
short completed TTL. A Mongo outage longer than that TTL erased the evidence and
the reconciler recovered the run as `interrupted` instead of `error`. All three
now follow the controller-observed path from the previous round: scheduled jobs
omit `completedAt` (retained-evidence TTL) and stamp the error outcome.
The PR description now explicitly narrows the deletion guarantee for the remote
OpenAI-compatible/Responses paths (tracked in issue 14594).
* fix: deletion-fence marker protocol — generation-scoped, atomic, fail-closed, all terminal paths
One consolidated pass over the finalization-marker mechanism, per the fourth
external review round. The invariant it establishes: NO persistence-owning
terminal CAS runs without a durable, generation-qualified marker covering the
window it opens, and every consumer treats a pending terminal as unsettled.
- Generation-scoped markers. Entries were keyed (userId, streamId), so a
Stop-superseded generation finishing late could clear the marker its
replacement registered on the same conversation. Marker fields are now
qualified by the generation's createdAt; clears must present the same
identity, and an unqualified legacy clear cannot drop a qualified entry.
- Atomic Redis registration. HSET-then-EXPIRE loses the fresh marker when the
user's existing hash expires between the two commands (or the process dies
there); registration is now a single Lua script carrying both.
- Fail closed everywhere. Registration failure (after one retry) now REFUSES
the terminal CAS instead of proceeding uncovered: the completion claim throws
into the error path, the error path skips completeJob and leaves the job
ACTIVE — deletion-visible by itself, recovered by the stale-running reaper —
and abortJob returns a new retryable `fence_unavailable` failure the Stop
route answers with 503 and the deletion quiesce treats as an unacknowledged
stop (fence kept). The previous round's log-and-proceed is gone.
- Every terminal path enrolled. abortJob now owns its window (register before
the abort CAS, clear in its finally — the Stop route's checkpoint prune and
partial save run inside beforePublish, between CAS and publication); the
interactive and background generation-error paths register before their
completeJob; the resume controller's error finalization registers before its
completeJob. A lost or thrown claim releases the marker after pending saves
flush instead of holding the user's deletion behind the TTL.
- Every pending terminal unsettled. settleAbortFence defers on
terminalPersistencePending for ALL statuses — including `aborted`, whose
route-side persistence the previous guard missed.
Also: a direct Redis regression for scheduled stale-pause retention (the P2
test gap), and the PR description no longer claims to carry every commit.
* fix: lease-token lifecycle fences — same-generation isolation, admission fence, undelivered-Stop retention, legacy abort enrollment
Fifth external review round; four P1s, handled as the requested consolidated
lifecycle-fence pass.
- Lease tokens. Marker fields were (streamId, createdAt), shared by every
contender on the same generation — completion and Stop, or two racing Stops —
so a losing contender's cleanup erased the winner's still-live marker. Every
registrant now carries a unique lease token in the field and may only ever
clear its own lease; unqualified legacy clears cannot touch qualified entries.
- Admission fence. Authentication can pass before the deletion barrier goes up,
and the durable createJob is several async steps later — a deletion quiesce in
that window saw neither an active job nor a marker and could cascade before
the admitted request created its job. The controller now registers an
admission lease and THEN rereads the deletion barrier: the ordering guarantees
either this request observes the barrier (403, lease released, slot/claim
cleanup) or the quiesce observes the lease and defers. Held until createJob is
durable; released on every refusal and initialization-error path. Fail closed
when the lease itself cannot be registered (503 retryable).
- Undelivered-Stop retention. abortJob released its lease in a finally even when
delivery AND publication had provably failed — the job reads terminal
(invisible to active-set scans), a user Stop writes no durable abort fence,
and the remote owner keeps generating and will persist its abort-catch writes
whenever the signal finally lands. The lease is now retained in exactly that
case, and each resignal attempt heartbeats a fresh lease so the fence outlives
the TTL for as long as delivery is still being driven. The abort-winning
turn's own loser-side pending saves are additionally fenced in the controller
catch (best-effort — those writes are already in flight).
- Legacy abort enrollment. abortMiddleware (assistants abort route fallback for
non-assistants endpoints) awaited abortJob and then spent usage and saved the
stopped response AFTER the abort's lease was released. Both writes now run
inside `beforePublish`, between the abort CAS and publication, covered by the
same lease as every other abort.
Barrier tests, each verified to fail without its fix: same-generation lease
isolation (store), racing two-Stop loser cleanup (manager, stale-read forced
CAS race), undelivered-Stop lease retention, resignal heartbeat, admission
refusal with lease-before-reread ordering plus release-on-durable-create, and
legacy-abort persistence inside beforePublish.
* fix: heartbeat-backed owner-lifecycle leases close the settlement handoff races
Sixth external review round: the remaining P1 interleavings were one structural
problem — lease handoffs that were not atomic — resolved as the requested
consolidated lifecycle-lease pass.
- Quiesce reads leases BEFORE the active-job scan. The admission-lease -> durable
-job handoff is only atomic against a reader in the OPPOSITE order of the
writer: writers hold the lease strictly until the job is active-set visible,
so leases-first shows every interleaving either the lease or the job.
Jobs-first allowed a request to create its job after the scan and release its
lease before the count — hiding both, cascading, and letting the new
generation persist into a deleted account.
- The abort acknowledgement is fenced by the owner-lifecycle lease. Redis ACKed
the moment the owner's AbortController tripped; the stopping side released its
lease on that ACK while the owner's asynchronous abort-catch persistence was
still ahead. The transport now awaits a manager-installed pre-ACK hook that
registers a DETERMINISTIC owner lease (exactly one owner exists per
generation, and determinism is what lets the signal-time registrant and the
owner's catch-side release agree across processes) before the acknowledgement
is persisted or published; an owned same-replica abort bridges to the same
lease before tripping its local controller. Both generation-owner catches
(fresh turn, resume) release it once their writes land.
- A failed replacement handoff no longer orphans the predecessor. The atomic
replacement removes it from active storage, and an unconfirmed handoff
terminalizes the replacement too — leaving nothing a quiesce could discover
while the predecessor's provider may still be generating. Its owner lease is
now retained at the point the receipt fails delivery; the owner replica renews
it through the pre-ACK fence when the signal finally lands.
- Leases HEARTBEAT while held. The five-minute store TTL only bounds a crashed
holder; live persistence — a stalled save, a long deferred title — must never
outlive its own fence. holdUserFinalization registers and renews every minute
until released; the controllers' completion/error/admission leases all hold.
The undelivered-Stop retention moved to a deterministic `stop` lease that
every resignal attempt renews and the first successful one clears (no more
opaque leases accumulating to TTL), and a THROWN abort transition releases the
contender lease instead of leaking it.
- The user-document abort-fence mutations now invalidate the auth user-doc
cache, matching every other user-doc write.
Barrier tests, each verified to fail without its fix: quiesce lease-scan
ordering (plus the observed-lease defer), pre-ACK fence ordering at the
transport, owned-abort owner-lease bridging, replacement-handoff predecessor
retention, held-lease heartbeat past the TTL, and failed-then-successful
resignal reaping the retained stop lease.
* fix: one manager-owned owner-lease span across every abort delivery path
Seventh external review round; four lifecycle-fence gaps, closed by making the
owner-lifecycle lease a single manager-owned, heartbeat-held span.
- Fail-closed acknowledgements. The pre-ACK hook registered a one-shot lease and
the transport ACKed even when it failed; a same-replica owned abort likewise
swallowed registration failure. The hook now acquires a HELD owner lease
(heartbeat until the owner's catch releases it via releaseOwnerLease) and a
rejection SUPPRESSES the acknowledgement — the stopping side stays retryable
behind its retention lease, and every resignal re-drives the handler. The hook
also stopped gating on `job.createdAt === generationId`: during a replacement
handoff the store holds the replacement while the abort targets the
predecessor, and that gate silently skipped exactly the generation being
acknowledged (the store job is owner identity, never a generation gate). A
local owned abort acquires the same held lease before tripping its provider;
post-CAS the trip cannot be withheld, so acquisition failure downgrades
delivery and the retention handoff keeps the user fenced.
- Committed-but-lost-reply disambiguation. A thrown abort transition released
the contender lease as if nothing had happened, but a Lua CAS can commit and
lose its reply — an aborted job invisible to active-set scans whose provider
was never signalled, with no fence left. The throw path now re-reads the exact
generation: only a job still live under the caller's identity proves no
commit; committed or ambiguous outcomes hand the fence to the deterministic
stop lease (kept on the contender lease if even that fails) before rethrowing.
- Replacement handoff covered end to end. A LOCAL replacement abort acquires the
predecessor's held owner lease before the trip (failure reports the receipt
undelivered, engaging retention). Failed-handoff retention is no longer a
swallowed one-shot: it heartbeats with the durable acknowledgement proof as
its renewal predicate — acquisition failures keep retrying for as long as the
fence is needed, and the retainer stands down (without clearing the shared
field) once the remote owner ACKs and thereby holds its own lease.
- A LOCAL resignal delivery hands off to the owner lease BEFORE clearing the
retained stop lease, and keeps the stop lease when that handoff fails.
Barrier tests: hook rejection suppressing the ACK, commit-then-lost-reply
retention with its provably-uncommitted counterpart, local-resignal owner
handoff ordering, pre-ACK owner lease held past the store TTL until release
(and provably stopped after), and failed-handoff retention retrying on its
heartbeat — verified fail-before/pass-after by stashing the fixes.
* fix: finish scheduled chat lifecycle hardening
* fix: close scheduled chat review follow-ups
* fix: generation-fence abort recovery evidence
* test: wait for settled approval tool output
* test: preserve scheduled init reconciliation option
* refactor: rebuild scheduled chats on durable agent triggers
* test: reset MCP cache mock between cases
* test: isolate scheduler startup in server specs
* fix: harden scheduled run lifecycle
* fix: normalize schedule capacity conflicts
* test: type schedule collision fixture
* fix: re-fence scheduled resume and expiry
* fix: fence scheduled resume handoffs
* fix: release superseded manual schedule leases
* fix: release failed run-now claims
* fix: release superseded engine claims
* fix: repair schedule dialog interaction and rework its form
The agent picker was unusable: ControlCombobox portals its popover to the
body by default, which lands it outside the dialog's Radix focus trap. Clicks
passed through it, it could not be tabbed into, and the trap fighting Ariakit
for focus locked the page up on selection. The prop is documented for exactly
this case — pass `portal={false}` and give the dialog `overflow-visible`, as
ProjectButton already does. The time and day dropdowns defaulted the same way.
Alongside that:
- Extract the agent builder's instructions editor (special-variable menu plus
expand-to-fullscreen) into a controlled `VariableEditor` and use it for the
schedule prompt. Insertions now route through `onChange`, so react-hook-form
sees them — the schedule PATCH is built from `dirtyFields`, and a `setValue`
that skipped dirty tracking would have dropped an inserted variable silently.
- Replace the hand-rolled frequency buttons with the shared `Radio`. They marked
the selection with `bg-surface-hover` on an outline button whose hover is the
same token, so the selected option was indistinguishable from a hovered one;
`Radio` is also a real radiogroup rather than four `aria-pressed` toggles.
- Wrap the fields in a real `<form>` and associate the footer button by id, so
Enter submits. Group the frequency, day and time controls in fieldsets.
- Add placeholders for name and prompt, match the textarea fill to the other
fields, and label the hourly case as minutes past the hour.
- Widen the dialog to `md:max-w-3xl` and pair name with agent so the form fits
without scrolling on desktop.
- Move scheduled chats below skills and above prompts in the side nav.
The new dialog spec fails when the portal fix is reverted.
* test: cover scheduled and subagent deletion drains
* refactor: own the form-control appearance in the client primitives
Addresses the codex finding on ScheduleDialog: a feature-local `FIELD_CLASS`
restated the `Input` primitive's border, radius, height and background so it
could be pasted onto the schedule dropdowns, leaving those controls with no
connection to the primitive they were imitating.
Move that appearance into `packages/client/src/components/Field.ts` as the
single source `Input` and `Textarea` now compose, and give `Dropdown` and
`ControlCombobox` a `variant="field"` that applies it. The schedule dialog
passes the variant and carries no class strings of its own.
This also repairs a break the dev merge would otherwise have introduced: the
newer `Dropdown` splits `className` (wrapper) from `triggerClassName`, so the
old pasted classes would have landed on the wrapper and left the triggers
unstyled.
The semantic-token guard now watches the shared module and asserts each
primitive still composes it, which covers more than the two files it read
before.
* fix: keep an explicit schedules disable from becoming an opt-in
`use` is two things at once for a dual-purpose runtime interface field: a
permission bit, which DB overrides strip, and the runtime disable signal that
`getLimits` reads. Stripping it from `{ use: false, maxPerUser: 2 }` leaves an
object, and `getLimits` treats any object without `use: false` as enabled — so
an admin override written to stop scheduled billing for a role or user started
it instead.
Collapse an explicit disable to the boolean form before the strip, on both
paths that accept it: the `interface.schedules` field patch, which admitted the
object wholesale because bare runtime paths deliberately bypass the permission
gate, and the overrides merge, which reached the composite-field branch and
kept `maxPerUser`. Objects that only narrow limits are untouched, so a
principal can still be given a smaller cap.
Both regressions fail without the normalizer.
* fix(schedules): Wave A — null-balance CAS, atomic paused-card clear, clustered erasure sweep
Slice 1 (thread r3804518381): route the existing-null balance initialization
through a { user, tokenCredits: null } compare-and-set instead of a blind $set,
so a concurrent initializer/charge landing between the preflight read and the
write is never handed back its spent starting balance. On a CAS miss the
preflight re-reads the winner. The absent-record $setOnInsert path and the
credited-record refill-config sync are unchanged. Adds the initializeNullBalance
adapter (no upsert) and regression coverage for winner/miss/sync cases.
Slice 2 (thread r3804518388): updateScheduleById now drops a `requires_action`
lastRun projection atomically with the configRevision bump. Any pause present at
edit time was projected under the pre-edit revision and can never be replaced by
its own revision-fenced terminal outcome — a disabling edit would strand the
card on "Needs approval" forever. Implemented as classic-operator CAS branches
(DocumentDB rules out a conditional pipeline $unset), fenced on the card STILL
being the pause so a terminal outcome or newer occurrence that races in is
preserved. Terminal history survives untouched.
Slice 4 (thread r3803826204): expose initializeScheduleErasureSweep from the
schedule runtime facade and start it in every clustered (experimental) worker
after Mongo is up. It re-drives eraseScheduleIfDrained for soft-deleted rows so
a hidden prompt cannot outlive its drain when the delete/erase-on-settle
attempts miss. It arms nothing else and never infers owner death from a
process-local missing job (isTopologySafeToArm gates that).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): Wave B.1 — reversible account-deletion schedule suspension
Thread r3804518383. Account-deletion quiesce marked every schedule `deleting`,
disabled it, and cleared nextRunAt destructively. When a later cascade step (or
the drain itself) failed, the controller cancelled the user-deletion fence —
restoring the user — but the schedules stayed `deleting` and were erased by the
sweep, silently losing all of a live user's scheduled prompts.
Replace the destructive marking with a REVERSIBLE, token-fenced suspension:
- suspendUserSchedulesForDeletion(userId, token) snapshots each schedule's prior
enabled/nextRunAt under a per-attempt token, then fences firing (disable, clear
nextRunAt, rotate claimToken). It never sets `deleting`, so a suspended row is
not erasure-eligible. Snapshotting reads then bulkWrites (a classic update
cannot copy field values under DocumentDB), fenced per row so an already-
suspended/soft-deleted/edited row is left alone; idempotent per token.
- restoreUserSchedulesFromDeletion(userId, token) reverses it, re-enabling and
re-arming only rows still carrying the exact attempt token and not independently
deleted — so an owner-deleted or newer-attempt-suspended schedule is never
resurrected.
- deleteUserController generates the attempt token, passes it to quiesce, and on
any failure that cancels the user-deletion fence restores the suspended rows. A
successful deletion hard-deletes them (and their snapshots) via the existing
cascade and never restores.
Adds a `deletionSuspension` embedded field (excluded from the wire schedule),
data-method regression tests (suspend/restore/fence/idempotency/no-resurrect),
and controller tests (restore on drain-false and post-quiesce cascade failure,
no restore on success).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): Wave B.2 — wire the interactive Stop persistence protocol
Thread r3804255932. The durable Stop primitives (requestRunAbort 'stop',
getScheduleRunAbortState, markRunAbortPersisted) already existed but production
only used requestRunAbort(..., 'deletion'). An interactive Stop flipped the job
to `aborted` and then persisted its partial message + checkpoint inside
`beforePublish`, without ever stamping the schedule Stop or acknowledging it —
so reconciliation, the generation owner, or a concurrent schedule/account
deletion could terminalize the run and release its capacity (and erase data)
mid-write.
Wire the request -> persist -> acknowledge -> settle barrier through the
schedule runtime (the route never touches raw Mongo):
- Expose beginScheduledStop / acknowledgeScheduledStopPersistence on the service.
- The abort route stamps the Stop BEFORE signalling abortJob (a serialized
'in_progress' loser returns 409 STOP_IN_PROGRESS without a second abort),
acknowledges only after beforePublish persistence succeeds, and on a
persistence failure leaves the barrier unresolved so the run stays preserved
(client retries; stale-owner timeout is the bounded recovery). A failed abort
releases the stamp it placed so a replacement/retry is never blocked through
its predecessor.
- recordScheduleOutcome (the owner settlement path) now waits, bounded, for the
Stop acknowledgement before terminalizing; a resolved/non-stop/stale marker
proceeds immediately. The paused Stop settles only after its own ack.
Adds service-layer barrier tests and route-level ordering/persistence-failure/
in-progress tests; the data-layer serialization is already covered.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): Wave C.1 — reconcile durable trigger delivery with the reservation
Threads r3803826192 (manual limiter) and r3804255924 (PII/moderation), plus the
independently-found long-Retry-After race. fireSchedule reserves a `started` run
and a global capacity slot BEFORE the durable trigger delivery reaches the chat
route, where an interactive limiter (manual Run Now), PII, or moderation can
reject it before any generation job exists — dead-lettering the delivery while
the run sat `started` until the 30-minute orphan sweep mislabeled it interrupted.
And a valid delivery deferred by Retry-After (up to 24h) could be orphan-settled
and have its capacity released, then fire anyway.
Translate durable delivery state into the schedule outcome:
- Store the deterministic trigger deliveryKey on the ScheduleRun reservation
(computed from the envelope BEFORE enqueue, so an ambiguous commit still has it).
- Add a getTriggerDelivery engine dep (wired to the merged trigger service's
getDelivery) that reads the durable delivery by key.
- Schedule reconciliation, for a jobless `started` run: staging/pending/leased →
admission is live, never orphan; dead → record `error` from the durable
lastError and release capacity promptly (no 30-minute wait), through the
ordinary outcome/auto-disable path; succeeded or no record → the existing
legacy orphan policy (interrupted only past the cutoff); a delivery lookup
failure defers rather than orphaning a possibly-live delivery. Limiter/PII/
moderation middleware writes no schedule state.
Adds reconcile state-mapping tests (dead/pending/leased/staging/succeeded/none/
lookup-failure) and a fire test that the reservation's deliveryKey equals the
enqueued delivery's idempotency key.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(stream): Wave C.2 — durable retry/ack for terminal host lifecycle actions
Thread r3804518375. Approval expiry won the `requires_action → aborted` CAS and
then invoked the host hook best-effort: `runApprovalExpiredHandler` swallowed a
failure, and because later sweeps enumerate only `requires_action` jobs, the now-
aborted job was never offered again. In the clustered entrypoint (no schedule
reconciler) the ScheduleRun stayed `requires_action` and its retained job
persisted indefinitely.
Make the host lifecycle work durable rather than schedule-specific:
- Add a generic `terminalHostActionPending` marker, set ATOMICALLY in the same
terminal transition (ApprovalLifecycle.expireWithIdentity), only when a host
adapter is installed.
- Retain and index such jobs: both stores keep them out of terminal reaping and
expose getTerminalHostActionJobs(); Redis adds a set + extended (24h-bounded)
TTL, in-memory a bounded 24h retention so a permanently-failing hook cannot leak.
- The manager clears the marker only after the adapter acknowledges success,
fenced by generation identity (clearTerminalHostAction), so a replacement
generation can neither clear nor execute its predecessor's action.
- cleanup()/expireStaleApprovals() enumerates unacknowledged terminal host actions
across restarts and replicas and retries the idempotent hook; the relay only
re-invokes while the marker is unacknowledged, so a successful ack prevents
duplicate work. Store-won expiry marks it too, so a loser-replica relay still
crosses the hook.
- Terminal SSE notification continues regardless of host-hook outcome.
Covers in-memory behavior (retry after failure, restart/other-replica retry, ack
prevents duplicates, identity fence, terminal notification on failure, no marker
accumulation for non-scheduled jobs) and updates the Redis cluster-membership
contract test for the new index.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* style(schedules): satisfy import sorting in fire.ts and fire.spec.ts
CI "Static checks" failed on IMPORT_SORT for the two files Wave C.1 added imports
to (the AgentTriggerEnvelope type import and getAgentTriggerIdempotencyKey).
Applied scripts/sort-imports.mts to exactly those files — imports-only reordering,
no behavior change. Other files reported by a repo-wide check are pre-existing on
dev and deliberately left untouched so this PR is not widened.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): repair the deletion CLI and order restore before the fence release
Addresses three findings from the fresh Codex review.
P1 — config/delete-user.js called methods.disableUserSchedulesForDeletion, which
Wave B removed in favor of suspendUserSchedulesForDeletion. The file is
@ts-nocheck and its spec mocked the removed name, so neither typecheck nor tests
caught it; the real CLI would throw a TypeError before deleting anything and then
only unwind the fence. The CLI now uses the tokenized protocol: it mints a
suspension token, suspends with it, and restores that exact attempt's rows in its
finally block when the deletion does not commit. Its spec mocks the real methods,
so the breakage can no longer hide.
P2 — both the HTTP controller and the CLI released the user-deletion fence BEFORE
restoring schedules. That fence is what refuses new schedule writes/claims, so the
gap let an owner PATCH edit a still-suspended row and have its enabled/next-run
state overwritten by the older snapshot, and let a second deletion attempt
re-suspend under a new token — making the first restore a no-op and stranding the
disabled snapshot permanently. Restore now runs first, while writes are still
fenced.
Hardening for the same defect class across a crash: suspendUserSchedulesForDeletion
now ADOPTS an existing suspension's snapshot when re-suspending a row abandoned by
an earlier attempt, instead of re-capturing the row's current (already-suspended)
state. Without this, an attempt that died before restoring would have its
successor snapshot "disabled, no next run" and permanently strand the schedule.
Tests: CLI restore-before-fence ordering and no-restore-on-success; the same
ordering assertion on both controller post-quiesce failure paths; a data-method
regression that a second attempt adopts the abandoned snapshot and restores the
original enabled/next-run state.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): converge dead deliveries in topology-safe maintenance
Codex finding: the `dead` delivery mapping added in Wave C.1 lives only inside
startScheduleEngine's reconciler, but the clustered entrypoint arms no engine — it
runs erasure-only maintenance. A delivery queued before a restart into clustered
mode and then rejected before generation creation (interactive limiter, PII,
moderation) dead-letters while its ScheduleRun stays `started`, holding a global
capacity slot indefinitely for an ordinary non-deleting schedule.
Add a dead-delivery convergence pass to the erasure sweep, so every topology that
runs schedule maintenance settles it. The pass is POSITIVE-EVIDENCE-ONLY and is
therefore safe where absence-based reconciliation is not: a `dead` delivery is
durable shared state proving no generation owns the reservation. It settles only
when the job is confirmed absent or identity-mismatched (an identity-matched job
still owns the run), defers on an unknown job lookup, on an in-flight abort, and
on an in-flight resume hand-off, ignores legacy reservations with no deliveryKey,
and applies a short grace so an accepted delivery still creating its generation is
never settled mid-handoff. Auto-disable policy is deliberately left to the armed
engine; this path records the failure and frees the slot.
Deliberately does NOT touch api/server/experimental.js — the clustered entrypoint
already starts this sweep, so the convergence arrives through the existing
initializer and the shared-file footprint stays as-is.
Tests: settles a dead delivery as error under an explicitly UNSAFE topology,
leaves live deliveries alone, never settles under an identity-matched running
generation (delivery is not even consulted), defers an in-flight abort, and
ignores a reservation with no deliveryKey.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(stream): refresh host-action retention on each retry attempt
Codex finding: unacknowledged terminal host-action evidence was capped at the
24h pause TTL, so a host dependency (Mongo) unreachable for longer than that let
the Redis key — and its pending marker — expire with no generation-fenced
acknowledgement, stranding the ScheduleRun where no reconciler is armed.
Measure retention from the LAST retry rather than from the terminal transition:
enumerating a pending host action IS the retry attempt, so both stores refresh
its retention as they hand it to the hook (Redis re-EXPIREs the job key; the
in-memory store stamps terminalHostActionRefreshedAt and bounds from it). Evidence
therefore survives as long as some replica is still actively retrying, while a
deployment that stops sweeping entirely still lets it age out — so this does not
reintroduce the unbounded leak the cap existed to prevent.
Test: after a failed hook, a later cleanup pass keeps the marker pending and moves
its retention basis forward.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): defer settlement when the Stop barrier times out
Codex finding: waitForStopPersistence returned after its 5s poll budget even when
the Stop was still fresh and unacknowledged, and recordScheduleOutcome then went
straight on to terminalize the run — releasing its capacity, deletion, and erasure
barriers while beforePublish may still have been writing. Slow checkpoint cleanup
is indistinguishable from a dead route on that signal, so the timeout was being
treated as if the barrier had been satisfied.
The poll budget now means "undecided", not "clear". On timeout with a fresh,
unacknowledged Stop the barrier DEFERS: recordScheduleOutcome returns false
without recording, leaving the run active/preserved. Settlement then happens
either when the route acknowledges, or once the existing stale-owner cutoff
(ABORT_OWNER_PRESUMED_ALIVE_MS) authorizes a later attempt — which the loop
already treats as clear-to-settle. Callers with durable retry (the approval-expiry
host action, reconciliation) re-drive it, so a deferral converges rather than
stranding the run.
Test: a fresh Stop that never acknowledges within the budget reports not-settled
and records no outcome.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): require definite delivery failure, extract its message, converge deferred Stops
Third Codex round. Three findings in code from this closeout, plus one pre-existing
P1 that is a one-line operator-safety fix.
P1 — dead-delivery settlement demanded too little evidence. `dead` does not prove a
request was rejected: the trigger host marks response timeouts and invalid success
responses `certainty: 'ambiguous'`, and the engine dead-letters those once retries
are exhausted. The erasure sweep treated every dead letter as positive evidence, so
an ambiguous one sitting over a generation a peer had accepted could terminalize the
run and release its capacity mid-flight. It now settles only on a DEFINITE rejection,
unless job absence is deployment-authoritative (safe topology), where the
confirmed-absent job is itself the evidence.
P1 — `lastError` is an `AgentTriggerDeliveryFailure` object, not a string. A
duplicated local interface declared it `string` (against CLAUDE.md's no-duplicate-
types rule), so both the sweep and the engine reconciler passed the object into the
String-typed run/schedule `error` fields; Mongoose would reject the cast, the per-row
catch would swallow it, and the run would keep its global capacity slot. The dep type
now reuses the canonical `AgentTriggerDeliveryFailure` and both call sites pass
`.message`. Re-typing immediately surfaced a stale test that had asserted a string.
P1 (pre-existing) — the base-config global stop is honored in `getLimits` via
`isRuntimeDisabled` rather than a literal `=== false`. The stop has two shapes, and
deepMerge turns base `{ use: false }` plus a principal override of `true` into
`{ use: true }`, so the literal check reported the feature enabled and Run Now
dispatched straight through fireSchedule, bypassing the operator's emergency stop.
Now the same predicate the engine gate already uses.
P2 — a Stop whose settlement DEFERRED past the poll budget had no convergence path
where no reconciler is armed. `acknowledgeScheduledStopPersistence` now optionally
re-drives the terminal outcome once the barrier clears; `recordRunOutcome` is
match-guarded and idempotent, so an owner that already settled makes it a no-op. The
abort route passes it for a running generation; a paused job still settles explicitly.
Tests: ambiguous dead letters refused under unsafe topology but settled when absence
is authoritative, definite rejections settled either way, the failure message carried
through, and the abort route's re-drive present for running / absent for paused.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): converge terminal runs in clustered workers, retry suspension restore
Fourth Codex round on
|
||
|
|
7569404a7c
|
🎛️ feat: Make Max Subagents Configurable via librechat.yaml (#15023)
* feat: make max subagents configurable via endpoints.agents.maxSubagents The per-agent subagent cap was hardcoded at 10 in MAX_SUBAGENTS, leaving orchestration-heavy deployments no option but patching limits.ts and rebuilding. Add an optional endpoints.agents.maxSubagents key to librechat.yaml (default 10, hard ceiling 50) that drives request validation, model spec presets, and the agents panel UI cap. * style: fix import order in OrchestrationHub |
||
|
|
a47ba7168f
|
🪢 feat: Custom Request Headers For Langfuse (#14945)
* ✨ feat: Custom Request Headers For Langfuse Self-hosted Langfuse behind an authenticating proxy or gateway could not be reached: every outbound Langfuse request hardcoded `Authorization` and nothing else. Adds `langfuse.headers`, mirroring `endpoints.custom` headers, and applies it to all four request surfaces — trace/media export (via the agents run config), feedback scores, central project-identity lookup, and admin credential verification. Values resolve through the same pipeline as endpoint headers, so `${ENV_VAR}` interpolation and header-safe encoding come along. `extractEnvVariable` continues to refuse infrastructure secrets, so a config cannot exfiltrate `MONGO_URI` through a header. A header whose variable is unset is dropped with a one-time warning rather than sent as a literal `${...}`, which a gateway would read as a wrong credential instead of a missing one. Headers merge beneath LibreChat's own `Authorization` on the REST surfaces, matching `mergeHeaders`, so a custom header can never displace the Langfuse credential. These are deployment-level and documented as such: trace export batches spans from every user through a single exporter, so unlike endpoint headers they cannot carry per-user placeholders. The central project-id cache key now includes the headers, so the header-less module warm-up cannot record a proxy rejection against the entry the request path later reads. * 🔒 fix: Keep Langfuse Headers Out Of Stored Overrides The generic admin config API accepts any field path inside an allowed section, so `langfuse.headers` could be written through it. Unlike `langfuse.secretKey`, headers are a map rather than one scalar path, so the config secret registry cannot encrypt them at rest or mask them on read — an admin-written map would sit in Mongo in plaintext and come back in plaintext, widening exposure of what are gateway credentials. Rejects them on both the dotted-patch and object-upsert routes, the same way process-backed MCP servers are held to librechat.yaml. This is what makes "deployment-level" true rather than merely documented. * 🐛 fix: Wire Config Middleware And Header Collisions For Langfuse Two codex review findings. P1 — `api/server/routes/admin/langfuse.js` never mounted `configMiddleware`, so `req.config` was undefined in production and credential verification silently ran without the deployment's proxy headers: exactly the deployments this feature targets could not save a connection. The handler unit tests injected `config` into their mock requests, so they stayed green. Mounts the middleware after the access checks (unauthorized callers still short-circuit first) and adds route-level tests that assert the handler actually receives a resolved config — the composition root, not the component. P2 — spreading custom headers under `Authorization` only replaced an exact-case collision. A configured `authorization` survived alongside the managed `Authorization` and fetch appends rather than replaces, sending both credentials in one combined value. All four request sites now use `mergeHeaders`, which already merges case-insensitively with the override winning; tests cover the lower- and upper-case variants. * 🔒 fix: Mask Langfuse Headers On Read And Harden Value Handling Three codex round-2 findings. P1 — the write guard blocked storing `langfuse.headers` in Mongo but did nothing for the read path: `GET /api/admin/config/base` serves the resolved AppConfig through `redactConfigSecrets`, which only knows registered scalar secrets, so a yaml-configured gateway credential was returned in full to any admin with Langfuse read access. Adds a secret-map registry that masks values while keeping key names, so an admin can still see which headers a deployment sets. Masking is safe precisely because these are yaml-only — a masked read cannot be round-tripped back over the real values. A malformed non-object value at that path is dropped rather than serialized. P2 — `mergeHeaders` indexes one spelling per lowercase name, so a config holding both `authorization` and `AUTHORIZATION` had only one displaced; the survivor was then appended by `Headers` into a combined value. Case variants are now collapsed at resolution, before any consumer sees them. P2 — `resolveHeaders` encodes only values it substitutes a user field into, and no user is supplied here, so a literal or interpolated character above U+00FF reached `Headers` unencoded and threw. Final values now go through `encodeHeaderValue`; Latin-1 still passes verbatim. * 🔒 fix: Keep Langfuse Header Credentials Out Of Logs And Validate Names Three codex round-3 findings, plus a documented boundary for the fourth. P1 — `loadCustomConfig` logs the parsed config at startup (`printConfig` defaults true), so a literal gateway credential in `langfuse.headers` was copied into application logs on every boot, undoing the masking the admin read path had just gained. The printed copy now goes through `redactConfigSecretMaps`, reusing the same registry. Scoped to map-valued secrets so scalar-secret log behavior is unchanged; the live config keeps its real values. P2 — a nonempty but invalid field name (` X-Token`, `X Proxy Token`) passed the emptiness check and then threw in the `Headers` constructor, which would break export, verification, lookup, and feedback for the whole deployment rather than that one header. Names are trimmed and validated against the RFC 7230 token grammar, and dropped with a warning otherwise. P2 — unresolved `${VAR}` detection tested the *resolved* value, so a credential legitimately containing `${...}` was mistaken for a failed substitution and dropped. Detection now inspects the configured text and checks the referenced variables directly, which also drops references to denylisted infrastructure secrets instead of forwarding them verbatim. The fourth (fanout gateway forwards only `Authorization`, so a tenant Langfuse behind its own proxy is not covered) is a real limitation in a separate component. Documented on the schema field and in the example config rather than left implied. * 🔒 fix: Scope Langfuse Headers To Configured Origins Three codex round-4 findings. P1 — one header map was attached to every destination a run resolves to. Under fanout that means a credential meant for an internal gateway was also sent to the central destination, typically Langfuse Cloud: an unrelated third-party origin. Headers are now attached only when the destination's origin is one the deployment explicitly configured (a self-hosted base URL, the fanout collector, or a tenant destination set by env). The built-in `*.cloud.langfuse.com` defaults are excluded precisely because nobody pointed at them. For trace export this also means attaching after the export branch settles on a `baseUrl` rather than before, since which destination wins depends on the branch. P2 — `encodeHeaderValue` only encodes above U+00FF, so a newline, CR, or NUL passed through and threw in `Headers`, breaking every request rather than the one header. Values are trimmed (the common trailing-newline case) then validated against the legal field-value bytes; an embedded CRLF is a request-splitting attempt and is dropped, not stripped. P2 — the write guard matched only the exact `headers` property, so `{ langfuse: { "headers.X-Token": "..." } }` and root-level dotted variants slipped through into the Mixed overrides document, where the nested-map redactor never walks them and a later read returns them in plaintext. All dotted spellings are now rejected. * 🔒 fix: Bind Langfuse Headers To One Configured Origin Four codex round-5 findings. P1 — the round-4 allowlist still authorized every configured origin, so a deployment with both a collector and an explicit central host sent the same credential to both. `langfuse.headers` is one map with no way to say which endpoint it authenticates to, so it is only unambiguous when the deployment configures exactly one Langfuse origin. Iterating on which origins to guess was the wrong axis; headers are now sent only when there is a single configured origin and the destination is it, with a warning when several make the intent unresolvable. That covers the self-hosted case this feature exists for; multi-destination deployments need per-destination headers the schema cannot yet express. P1 — `fetch` defaults to following redirects, and Node strips `Authorization` across origins but keeps arbitrary headers, so a redirect off an allowed origin would hand the gateway credential to a host that passed no check. Requests carrying custom headers now refuse redirects; requests without them keep the default, so nothing changes for existing deployments. P2 — `extractEnvVariable`'s whole-string branch is anchored and greedy, so `${CLIENT_ID}:${CLIENT_SECRET}` parsed as one variable name and the raw template was sent as the credential. References are expanded here now, so only literal values reach that path. P2 — a valid token name is not necessarily usable: `Transfer-Encoding` makes `fetch` throw and a fixed `Content-Length` misdescribes the body of every other request sharing the map. Request-framing names are dropped. * 🐛 fix: Expand Langfuse Header References Exactly Once Codex round 6 (P2). After expanding `${VAR}` references myself I still handed the result to `resolveHeaders`, which runs `extractEnvVariable` over it again — so a credential containing `${PATH}`, or any other name that happens to be set, was silently rewritten on export, verification, lookup, and feedback. The round-3 test only used an *unset* embedded name, which the second pass leaves alone, so it could not catch this. Resolution no longer round-trips through `resolveHeaders`. The only part still wanted from it was stripping `{{...}}` user placeholders, which is now applied directly; expansion, encoding, and validation were already local. Adds a test whose embedded variable is set, which fails against the previous pipeline. * 🐛 fix: Process Langfuse Header Templates Before Substitution Codex round 7 (P2), the mirror of round 6. Having stopped re-expanding the resolved credential, the placeholder strip was still running over it: a token containing `{{LIBRECHAT_USER_ID}}` had that span deleted and `abc{{...}}ghi` went out as `abcghi`. Establishes the invariant the last two rounds were circling. Every template operation — placeholder strip, unresolved-reference check, expansion — now runs on the operator's configured text, and the credential is substituted last and never touched again. Gateway credentials are arbitrary strings, so none of their bytes are syntax. Also moves the unresolved-reference check after the strip, so it no longer reports a variable inside a `{{...}}` span that the strip removes. |
||
|
|
57ea1137f6
|
🛡️ feat: Let Admins Restrict Stateful Workspace Scopes (#14910)
* feat: let admins restrict stateful workspace scopes * fix: enforce stateful scope policy across agent paths * fix: close stateful scope policy activation gaps |
||
|
|
7d850c308a
|
🧠 feat: Add Live Reasoning Labels (#14893)
* feat: add live reasoning labels * fix: Stabilize reasoning label checks * fix: Address reasoning label review findings * chore: Bump Agents SDK for reasoning labels * fix: Reset reused reasoning step evidence * fix: Reconcile cleared reasoning labels * fix: Fence reasoning label resets * fix: Reset reasoning ownership before gap labels * fix: Preserve THINK type through label reset * test: Expect run-global reasoning revision |
||
|
|
3bd2358805
|
🗜️ feat: Let Users Toggle Client-Side Image Resizing (#14883)
* feat: allow users to toggle client-side image resizing Client-side image resizing could only be configured in librechat.yaml and defaulted to off, so users had no way to enable it for themselves. Add a "Resize images before upload" toggle in Settings > Chat > Sending, stored per device in localStorage. When librechat.yaml sets clientImageResize.enabled, mergeFileConfig marks the value as enforced and the toggle renders the admin value read-only. Admin resize parameters still apply without locking the toggle when enabled is omitted. Also fix shouldResizeImage, which compared file size against 10% of a 512MB fallback limit and so only triggered above roughly 51MB. It now uses a 512KB floor, which lets the setting affect everyday photos. * fix: harden client image resizing * fix(client): restrict image resizing and localize toast * fix: harden client image resizing * fix(client): recognize static WebP image chunks * fix(client): clamp resized image dimensions * fix(client): enforce safe image resize uploads * fix(client): recheck duplicates after transforming uploads * fix(client): keep selected files after input reset * fix(client): disable image resizing when file config fails * fix(client): decode resize candidates without a base64 copy * fix(client): coordinate upload batches across hook instances * test(client): drop the untyped conversation from shared upload state * fix(client): start the config wait before queueing uploads * fix(client): disable the resize switch while file config is pending The switch stayed clickable during the initial file-config load even though the checked state cannot update until that query settles. * fix(client): only track upload reservations for observable state |