mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-19 22:43:38 +00:00
104 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
7b71c3fc35
|
🥅 fix: Keep Tool Error Prefix Parsing In Bounds (#15857) | ||
|
|
e2f5772961
|
🧮 feat: Surface Retained Tool-Message Context Usage (#15796)
* feat: surface retained tool-message context usage * fix: correct post-snapshot context accounting and lockfile registry Adjudicated the review findings on the retained tool-message context work: - package-lock.json resolved four entries (@librechat/agents, call-bind-apply-helpers, es-object-atoms, get-intrinsic) from a private mirror, breaking `npm ci` outside that network. Restored the public registry hosts; the integrity hashes already match the public tarballs. - `finalizeUsage` migrated every usage atom except `subagentUsageFamily` before dropping the temporary `new` key, so a first-turn subagent total vanished when the conversation id was assigned. - The runway projection divided the pre-invoke `remainingContextTokens` by per-call growth, ignoring the output generated after that snapshot, and the compaction hint subtracted a post-call exchange from a pre-call message total. Both now apply the post-snapshot output, so each side is measured at the same point in time. - The estimate path rendered its tool-call share as a peer row although it is a subset of Input/Output/Estimated, so the visible rows summed past `usedTokens`. It is now an indented subtotal; the counted rows stay provider-accurate. - The Totals block presented conversation-wide session subagent usage beside branch-scoped rows; it is now labeled all-branches. - Rewrote the truncated `rgb-series-8` rationale as one comment on the series block. * fix: keep subset accounting whole across cache, legacy snapshots, and custom themes Adjudicated the second review round on the retained tool-message context work. One cause per finding, each measured on the same basis as what it is compared to: - The all-branches subagent subtotal summed input+output only, so a cached subagent call was under-reported and a fully cached call with no output disappeared from the row while the Cache read/write rows above still counted its traffic. It now sums every normalized bucket. - `collectAnchorSeries` read an absent `remainingContextTokens` as zero and recorded that snapshot as having spent its entire budget, which distorted the growth delta feeding the runway projection. A snapshot saved in the older shape now derives its used total from the breakdown, exactly as the render path's `baseUsed` does, and one that carries neither basis is skipped. Each reading is tagged with its basis and the projection compares only readings of the same basis instead of reporting the difference between two measurements. - `resolveTheme` filled an omitted `rgb-series-8` from the bundled base, so a stored or environment theme that paints its own seven-slot scale got LibreChat's indigo on surfaces it never checked: 2.41:1 against that theme's meter track, under the 3:1 mark floor the scale claims. An owned scale now derives the slot from the theme's own neutral text — 11.5:1 on the same reference theme — and naming `rgb-series-8` still opts out. - Ordered the `--series-*` declarations in the light block by slot. * fix: keep cached shares out of the peer rows and track resolved text for slot 8 Third round on the retained tool-message context work, correcting the two variants of invariants this PR already fixed elsewhere rather than patching them locally: - The snapshot path rendered the reconciling call's Cached and Cache write shares as peer rows with window percentages, although reconciliation already counted those tokens inside the segments above. A fully cached prompt showed its tokens twice and the visible rows summed past the meter. They now render as an indented subtotal, the same treatment the estimate path's tool-call share received. The existing test claimed this invariant in its name but only asserted the rows exist, so it now asserts they are not peers. - The `rgb-series-8` fallback required the theme to name its own text, so a theme that repaints the scale and its surfaces while inheriting LibreChat's text colours still received the bundled indigo. It now derives from the RESOLVED secondary text, so the stop tracks whatever the theme reads its body copy against whether it names one or inherits ours. * fix: settle the subagent share with its run and measure both reclaim sides alike Fourth round on the retained tool-message context work: - The subagent share was committed on arrival while the usage it is a subset of stayed pending, so a run that ended with no salvageable response left its tokens in the Totals after `resetLive` discarded them from every rollup, and a resume re-folded the same events into the cumulative figure. It now accumulates in `pendingSubagentUsageFamily` beside the pending usage and settles with it: committed by `flushPendingInto` when the response keeps its usage, dropped by `resetLive` otherwise, migrated with the conversation id. The view merges committed and in-flight shares, so the live row is unchanged. - `latestExchangeTokens` subtracted a summarizing turn's whole response, including the summarization completion the backend folds into `tokenCount` while the snapshot holds it in `breakdown.summaryTokens` rather than `messageTokens`. The reclaim hint was understated by the summary and could reach zero after a long one; the summary block's size is now excluded from the tail's contribution when that turn compacted. - `sumBranch` exposed the tail's char estimate and tool share even when the tail carried a `tokenCount`, so a resumed partial response stayed in the message total while its tool traffic was dropped from the Tool calls split. Both tail figures are now zero for a counted tail, which also stops the estimate path subtracting an estimate that was never summed. * chore: drop the lockfile churn the base branch already carries The SDK bump this PR set out to make landed on dev as 3.8.5, so the branch no longer changes any package.json and its lockfile edits (the 3.8.4 entry plus the transitive call-bind-apply-helpers/es-object-atoms/get-intrinsic bumps that install pulled in) are churn against the base. * fix: give the eighth series slot a value every bundled theme can carry The base branch grew high-contrast palettes that must declare every registry token, and a test that a series slot can carry `text-on-status` at the 3:1 mark floor, because a slot also fills a badge chip under that glyph. Slot 8 failed both: the high-contrast palettes could not name a token that did not exist when they were written, and the dark indigo #8c98e6 carried white at 2.71:1. Dark slot 8 is muted to #8082be, which reads 3.66:1 under the status label and still clears 3.66:1 on every series surface. The high-contrast palettes take their own neutral text as slot 8 — the same stop `resolveTheme` derives for a third-party theme that paints its own scale — because seven saturated hues is all either canvas affords at their stated CVD separation; it is distinguished by lightness at 21:1 on the canvas instead of by hue. * test: exercise the context split through the mock harness Nine scenarios of the acceptance contract, one tagged Playwright test each, driving the real application: a tool round-trip through the e2e MCP server for the Messages/Tool calls split, its keyboard disclosure of per-tool counts, the peer rows staying inside the meter and surviving a reload; a real subagent run for the all-branches total; seeded conversations for the legacy snapshot whose used figure comes from its breakdown and for the estimate path's nested tool share; and a stored theme that owns the series scale for the eighth slot's fallback and its opt-out. * test: keep the tool and subagent scenarios on the viewports that expose them * 📌 feat: Unified Pinned Section With Sidebar Drag Interactions (#15057) * feat: unified pinned section with drag interactions and sidebar polish - Pinned chats and favorite models/agents/specs render as one interleaved, reorderable list persisted via users.pinnedOrder - Drag conversations onto project rows to file them, onto Chats to unfile, and onto the Pinned section to pin (empty section materializes as a drop zone during a conversation drag) - Row-level unpin button on pinned chats matching the favorites' button, and the row action menu shares the same hover/focus treatment - Agent Marketplace entry moved under the New Chat button in the side rail - Bookmarks trigger sized to the search bar, search focus no longer swaps its border color, tag filtering shows a loading state, and opening a project shows row-shaped skeletons * fix: add pinnedOrder to IUser type and sort imports * fix: pinned drag threshold, unpin focus target, and marketplace link * fix: surface failed pinned reorders and skip no-op order writes * fix: address review findings on unified pinned section Move the pinned-order endpoints into packages/api as TypeScript handlers and route their URLs through the data-provider endpoint registry. Raise the entry and key caps so a valid model favorite and an unbounded pin count can both be persisted. Keep the marketplace entry reachable from the mobile drawer, which is the only sidebar surface on small screens. Merge a reorder into the stored order while a bookmark filter hides part of the list, instead of dropping the hidden keys. Add Alt+Arrow keyboard reordering with a live-region announcement, restore the focus handoff when a favorite is removed, and hold the local arrangement until the stored order catches up. Serialize overlapping order writes and discard the optimistic value when the first one fails. Derive the tag-filter loading state from isPreviousData rather than a flag that could stick on. * fix: release drag while renaming and dedupe repeated favorite keys A draggable ancestor swallows drag-select inside the rename input, so both the row's own drag source and the pinned wrapper's release while a title is being edited. The favorites payload validator accepts a repeated entry, and two rows keyed the same would collide in React, point every drag at the first copy, and fail the order endpoint's uniqueness check. Build the pinned list keyed-unique. * fix: address PR review bot findings Make the model favorite key injective. Joining endpoint and model on '::' collided for components containing the delimiter, so two distinct favorites keyed the same and the dedupe hid one; the endpoint is now length-prefixed. Replace the pinned-order entry cap with a total size guard. Pinning has no membership cap and the sidebar query drains every cursor, so any count limit rejects a legitimate list rather than bounding anything; what needs bounding is the user document. Let only the newest order write touch the cache. An earlier write failing rolled back over a newer arrangement that was still in flight, leaving the UI on a stale order while the server held the current one. A confirmed write now takes the server's answer, and a failed newest write discards the value that was never accepted. Persist the favorites array only once the order write succeeds, so a rejected write cannot leave the two stores disagreeing with the rolled-back display. * fix: address second round of PR review bot findings Read favorites membership at write time rather than capturing it at drag end. A favorite added or removed while the order request was in flight was undone by writing the older list back; only the relative order now comes from the drag. Move the pinned-order queue to module scope. The sidebar unmounts whole sections while a search is active, so a component-local ref let a remounted hook start a second queue alongside a request that was still running. Serialize project assignment per conversation. Every drop target owns its own mutation instance and the write is an unconditional update, so two quick drops let whichever request reached the database last decide the project. Read the pinned-order request body null-safely: destructuring a nullish body threw past the handler into Express's generic error path instead of returning the intended 400. * fix: address third round of PR review bot findings Drop the secondary favorites write. No consumer of the favorites array reads its order, so mirroring the arrangement there bought nothing while racing the membership mutations that share that whole-array endpoint; pinnedOrder is the single source of ordering. Always merge the committed order into the stored one. The visible list is not always the whole list, since a bookmark filter hides rows and the pinned query publishes partial results while draining its cursor, and replacing wholesale dropped every key it could not see. Release the local arrangement when the write settles instead of when the stored order matches it key for key. A pin arriving mid-request made that equality unreachable, stranding the snapshot and hiding the new row until remount. Abandon a queued pinned-order write if the signed-in user changed, so a deferred request cannot write one account's order into another's document. * fix: address fourth round of PR review bot findings Guard the pinned-order completion callbacks on the signed-in account. The pre-send check only covers a write that has not left yet; a response arriving after a session change would publish one account's order into the next one's cache, which this query never refetches on its own. Commit a reorder only when the pinned rows handled the drop. Dragging a pin onto a project row or into Chats sweeps the pointer across sibling rows, whose hover handlers shift the order, so a filing action was also saving a move the user never asked for. * fix: address fifth round of PR review bot findings Deselect pinnedOrder at schema level. It is display-only and allowed to grow large, but every authentication request loads the user document, so it was adding its whole payload to paths that never read it. Refuse to reorder until the stored order has loaded. Merging against an absent order and then cancelling its fetch posted only the visible keys and discarded saved positions for good. Prune keys only once the visible list is known to be the whole list. Merging unconditionally grew the array until the size guard rejected every write, so completeness now gates compaction rather than the filter flag alone. Track every renaming row, not just the most recent. RenameForm has no blur cancellation, so a second form can open while the first is still mounted and the single key reconnected the first row's drag source under its open input. Judge drop eligibility by the queued destination. A drag item carries the project its row had when the drag began, so re-dropping a chat back where it started while an assignment was queued read as a no-op. Take mutation ownership from the user atom AuthContext fills on login rather than the user query, which is still empty in that window, and never treat an unknown owner as a match. Restore the pinned section's 30vh cap. Matching the Projects section's 42vh put 84vh of non-shrinking content above Chats and starved it on short viewports. * fix: address sixth round of PR review bot findings Gate reordering and pruning on successful fetches rather than attempted ones. A failed pinned-order GET still reports as fetched, and a failed later page of the pinned drain still publishes partial data and stops fetching, so both gates were opening on exactly the states they were meant to exclude. Scope the post-unpin focus handoff to this section by its own marker. The whole conversations pane is a labelled region too, and the project lists inside it render the same row, where unpinning only clears the flag and the row stays put, so focus was being moved off a row that never went away. Clear the account tracker when the mutation hook unmounts. The sidebar drops that section during a search, leaving a stale non-null id that also shadowed the cache fallback, so a deferred write could pass its owner check after a session change. * fix: gate pruning on a delivered favorites list A favorites fetch that exhausted its retries leaves isLoading false with the list empty, so an otherwise complete pinned load enabled pruning and dropped every favorite key from the stored order. useFavorites now exposes the query's success state and the pruning gate requires it, matching the pinned-order and pinned-membership sources. * fix: address eighth round of PR review bot findings Identify queued assignments by token rather than by destination. Comparing destination values cannot tell two drops onto the same project apart, so a B, A, B run let the first B's cleanup discard an entry the still-queued later drops owned, and the next drop was then judged against a stale project. Report the rename ending when a row unmounts. A pinned row removed mid-rename, for instance by unpinning the same chat from the project list that also renders it, left its key in the renaming set and came back permanently undraggable. * fix: address ninth round of PR review bot findings Stand the keyboard reorder down wherever the pointer drag does. Alt/Option plus an arrow moves the caret word-wise inside a text field, so the shortcut was moving and saving the whole row out from under an open rename input. Hold a settled assignment destination until a drag item shows the new project. The mutation's cache invalidations are not awaited, so a row still reported its old project for a moment after the write succeeded and a quick drag back to where it started read as a no-op. * fix: address tenth round of PR review bot findings Serialize project assignment inside the mutation instead of at one caller. The drag targets, the row menu and the project dialog each hold their own instance, so a queued drop could still overwrite a newer menu write; moving the queue down covers every surface and lets the drag helper drop its own. Tie the live snapshot's release to the arrangement that created it. An older write settling mid-drag cleared the newer arrangement, snapping the rows back to a stale order while the drag still held its own, so releasing saved an order the user could no longer see. * fix: address eleventh round of PR review bot findings Move assignment destination tracking next to the queue that owns it. Keeping it in the drag helper meant a menu or dialog move left the drag helper still reporting its own older destination, with no row left to observe that would ever clear it, so a later move back was rejected for good. Resolve a conversation's project from the write the mutation confirmed rather than from whichever list rendered the row. The same chat appears in the Pinned section and its expanded project, and those caches refresh independently, so observing one of them was never a sound way to decide the other had caught up. The conversation cache is written synchronously on success, which makes the lazy self-clearing and its settled flag unnecessary. * fix: cancel an in-flight conversation refetch before confirming an assignment Navigation invalidates the exact conversation query, so a refetch that read the old project can be in flight when an assignment resolves and land afterwards, reverting the move in the cache that now answers where a conversation lives. * fix: keep queued pinned-order writes attributable across an unmount Refuse a queued write only when a different account is positively established. Clearing the identity on unmount stopped stale attributions but stranded work that belonged to the same account, since the sidebar drops the section that owns this hook whenever a search is active, and it denied that work its rollback as well. Correct the cache from the request rather than from onError. The observer callbacks do not run once that section unmounts, so an order the server never accepted could survive on a query that refetches on nothing. * fix: publish the session identity from the user atom Queued pinned-order writes were attributed by a tracker that only the pinned section's hook updated, and the sidebar unmounts that section during a search. A retained value went stale across a sign-in, and login clears the user query before the next account's fetch resolves, so a write queued by the previous account could pass its own check and post on the new account's credentials. A Recoil atom effect publishes the identity instead: it runs wherever the atom is set, including sign-out, so attribution comes from the state itself rather than from a component that happens to be mounted. That makes a signed-out session foreign to every owner and lets the check be a strict comparison again, rather than a rule that had to guess when identity was merely unreadable. * fix: discard reorders that ended on a refused external target A drop the pinned rows did not handle usually means a reorder, but a project row or the Chats section refusing one reports the same, so dragging a chat onto the project it already belongs to was saving the incidental shift its pointer caused on the way out. The rows and those targets now report their hovers, so the last one under the pointer settles it, which also allows for a drag that strays over a project and returns to reorder after all. * fix: attribute queued writes to the credentials they will travel with Publishing the account from React state, whether an effect or a Recoil atom effect, always lagged the credential change: setUserContext installs the new Authorization header in one synchronous call, and a write queued by the previous account and sent in that gap read its own owner back and went out as the new one. A test against the atom effect showed onSet is deferred too, so the atom was no better than the effect it replaced. The header is the thing that decides who the server sees, so ownership is read from it. The id claim is compared rather than the token, since a silent refresh rotates the token for the same account and discarding that work would strand something the user is still entitled to. Leaves AuthContext and the user atom untouched. * fix: attribute a reorder to the account that started it onMutate awaits cancelQueries before the request is built, and the mutation function was reading the session again afterwards, so credentials turning over inside that gap sent one account's order as the next. The owner is now recorded in onMutate's first synchronous statement and carried on the order itself, and the optimistic write is skipped when the session moved on, since that cache is shared and showing an order to the wrong account is the same mistake as sending it. Record a pinned hover before the guards rather than after. Hovering the dragged row is a no-op for ordering but still means the pointer is inside the list, so a drag that wandered onto a project and returned to its own position was being discarded as a filing action. * fix: address a further round of PR review bot findings Mark a hover anywhere in the pinned section as internal. The header and the padding around the rows are part of the list, so a drag that wandered onto a project and came back to rest on one of them ran no row hover and was discarded as a filing action. Hold off pruning while a favorites write is unsettled. The earlier GET's success stands through an optimistic removal, so pruning then dropped the ordering key of a write that had not landed, and a failed one came back without its saved position. Declare the reorder shortcut on the elements that take focus. The section's written hint is skipped by anyone tabbing straight to a row, leaving the only non-pointer way to reorder undiscoverable. * fix: reach a real New Chat control, and let the shortcut prop through the memo The unpin fallback looked for a test id no component carries, so once the last pinned row unmounted focus fell to the document rather than to New Chat. It now tries the controls that exist, in sidebar order. The test that covered this had fabricated the missing id, so it proved nothing. Convo is memoized on conversation fields and isGenerating alone, so the shortcut appearing once the pinned order loaded never reached the element that announces it. The comparator now accounts for it. * fix: identify OpenID sessions, and see favorites writes from every hook With OpenID token reuse the bearer is the provider's own id_token, which names its subject as sub under an issuer and carries no LibreChat id, so every such session resolved to nobody and two different accounts compared equal. That is the case the ownership guard exists for, so it read as safe while being exactly wrong. Sessions are identified by whatever the credential offers now, and an owner that cannot be identified is foreign to everything including itself. Count favorites writes across the query client rather than from one observer. A favorite row runs its own useFavorites, so a removal started there was invisible to the pinned section and its ordering key could still be pruned. * fix: close the assignment cache gap, and let a failed drain be retried The assignment released its pending destination as soon as the request settled, while the cache write waited on an awaited cancellation in onSuccess. A drag started in that gap saw neither, fell back to the stale row and refused a valid drop. The write now happens in the request, before the entry is released, which also survives the owning component unmounting. Also clear the pagination guard when the Chats section collapses. A drain that exhausted its retries leaves the conversations array unchanged, so the guard barred every later attempt and the remaining chats stayed unreachable for the session. Reopening the section now tries once more. That guard came from #14860 rather than this branch, so it can be split out if preferred. * fix: compose the marketplace link, and keep focus on a surviving row The marketplace control was a raw link with restated appearance classes, and in restating them it left out the focus ring entirely, so keyboard focus on it was invisible. It composes through the shared button now, which carries the ring, the hover fill and the theme timing. Unpinning from a project list leaves the row in place but removes the pin badge that had focus, and the pinned-section successor search deliberately returns nothing there, so focus fell to the document. It moves to the row's own link. Reconcile the pinned order on window focus and reconnect. It is one shared per-user record, so a second tab would otherwise show a stale arrangement and its next reorder would post a full array undoing the other tab's. * fix: keep a rename from restoring a conversation's other fields A rename request carries only a title, but its response is the whole conversation as the server saw it, and writing all of that back restores the pre-request value of every other field. An assignment confirmed while the rename was in flight was therefore reverted in the cache that now answers which project a conversation belongs to, and a drag back to the original project read as a no-op. Only the title and its timestamp are taken from the response. Cancelling queries cannot help here, since this is another mutation's write rather than a fetch. * fix: compose the unpin badge through the shared row-action button The badge restated a reusable icon control's sizing, radius, hover and focus classes. The client package already had a row-action variant and an icon-xs size that match it exactly, so it uses those instead of its own copy. * fix: reconcile before reordering, and scope assignments to their session A reorder stayed enabled while the order's focus refetch was in flight, so returning to a tab could cancel that refetch and post the arrangement it was about to replace. Pruning had the matching gap: the pinned list keeps its data fresh for five minutes, so a tab whose order had reconciled could prune against membership that had not, dropping the position of a conversation another tab pinned. Both now wait for data that is settled and not stale. Assignments carry the account that started them, like the order writes already did. Conversation ids are per-user, so a queued one travelling with the next account's credentials would act on whatever that id names over there, and its answer would describe one account's conversation to another in a cache that is not scoped by user. The row's overflow trigger derives from the shared row-action variant too, so the two actions in a row stop keeping separate copies of the same recipe. * fix: reject foreign assignment results, and share the section header An assignment settling after a session change returned normally, so the success path still ran: the caller reported success and the response could be applied to the next account's conversation, since ids repeat across accounts. It rejects now, so nothing downstream treats it as a result. The collapsible section header existed as three identical copies, Chats, Projects and Pinned, and this branch added the third. It moves into the shared button as a section-header variant and all three compose from it. * fix: let favorites membership reconcile between tabs The favorites query disabled focus, reconnect and mount refetches, so a second tab kept its old list indefinitely. Its pruning gate waits for a fetch, and none was ever going to happen, so a reorder there could delete the saved position of a favorite the first tab had added. It reconciles on focus and reconnect now, like the order it is arranged alongside. * fix: hold reconciling refetches off while a write is in flight Enabling focus and reconnect refetches on the order and favorites queries let one start after onMutate's single cancellation, read the pre-write state and land after the write, putting the old value back. Both now decline those refetches while a write for that resource is outstanding, keeping the cross-tab reconciliation itself. The check asks the query client at the moment the event fires rather than reading useIsMutating: that hook only reaches the query's options on the next render, which is a race against the very event being gated. A test that dispatches focus mid-write fails without it. * fix: release abandoned assignments and gate pruning on current membership An assignment rejected before entering the try block kept its pending entry forever, so a later session read a destination that was never sent. The ownership check now sits inside the block that releases it, and it runs again after the conversation query cancellation resolves, since the session can turn over while that await is pending. Pruning the pinned order now requires membership at least as current as the order it is pruned against. The pinned query holds its data for five minutes while the order has no such window, so focusing a tab reconciled the order alone and the next reorder dropped the key of a conversation another tab had pinned in between. The assignment spec resets its mocks between tests and gives each test its own conversation id, so one test's unconsumed mock and leftover queue entries no longer decide the next test's result. * fix: address the sixth round of review findings on the pinned section The section header recipe asked for the shared button variant without a size, so CVA emitted the default `h-10 px-4` after it and the merge kept it, putting a 40px control in a 32px header row. The variant opts out of default sizing through a compound now, since a heading is sized by its own text. Restore the search field's focus border. Removing it left keyboard focus signalled only by a background change that hover already uses, so the field had no distinguishable focus state at all. An assignment can reach the database before a concurrent rename and still answer after it, carrying a conversation that predates the rename. Only the project field is merged into the cached conversation and published to the active one now, matching what the rename handler already does. An order refetch that starts mid drag no longer lets the drop post an array built from the order that refetch is about to replace: the arrangement is held and written once the fresh order lands. Reorder writes superseded by a newer arrangement are skipped rather than sent, so holding the shortcut costs two writes instead of one per intermediate position. The order and favorites queries reconcile on mount as well, since a sidebar search unmounts the section and a query with no observer never sees the focus event. * fix: preserve pinned order until favorites membership catches up * fix: reconcile pinned membership and preserve auth and focus state * fix: preserve reconciled keys during active pinned arrangements * style: sort pinned order regression imports * fix: reconcile favorite rollbacks before pinned-order pruning * 🧊 fix: Withhold Rerun Controls on Compaction Turns (#15797) * 🧊 fix: Withhold Rerun Controls on Compaction Turns A compaction turn's response parents onto the leaf it summarized, not onto a user message, so the hover Edit's Rerun replayed an assistant message in the user slot: the request minted a user message under the existing assistant leaf's id and ran the turn with empty text, which fails. Editing offered nothing either, since SUMMARY is not an editable part type. Edit, Regenerate and Continue are now withheld on a compaction turn, and both rerun paths refuse a parent that is not user-created, matching the guard `useChatFunctions.regenerate` already applied. * 🧭 fix: Gate Rerun Controls on the Replayed Parent Turn Withholding Edit, Regenerate and Continue on summary-only content also hit an ordinary turn that auto-summarized and was cancelled before its first answer token: that response is parented to the user's message and is exactly the turn a rerun is for. The row now resolves the message it would replay and withholds the rerun shapes only when a model turn hangs off another model turn, which is the manual compaction shape. An unresolvable parent withholds nothing. `isCompactionTurn` is gone with it, and `isCompactedLeaf` keeps its original body; its docstring now says per part what it always measured per part. * 🧮 fix: Identify a Compaction Turn by Content and Parent Gating the rerun controls on parent authorship alone withheld Edit, Regenerate and Continue from any model turn hanging off another model turn. An imported conversation chains messages sequentially, so a skipped empty human message leaves an ordinary reply parented to the previous reply; its stored content is editable and its controls must stay. It also made every model row resolve its parent out of the flat message list: 100 full-thread scans while mounting a 200-message conversation. A manual compaction is now both halves of its shape — summary-only content and a parent that is not the user's turn — so only such a row consults the thread. `isSummaryOnlyContent` is the content half, in data-provider next to `isCompactedLeaf`. * 🧷 test: Pin the Rerun Refusal on a Chained Model Turn A rerun replays its parent as the user turn, so a model turn parented onto another model turn must submit nothing rather than mint a user message under an existing response's id. Removing the guard makes the case call `ask` once. * 🧱 fix: Gate the Editor on Content and the Rerun on the Parent Identifying a compaction turn by content shape missed the ones the server persists with an error part (a skipped or failed compaction keeps its explanation instead of a summary), which left Edit, Regenerate and Continue on a turn where every rerun shape is inert. The two controls answer two different questions now. A rerun replays the message's parent as the turn's user message, so Regenerate and Continue go whenever a model turn hangs off a model turn. The editor goes only when that turn also has no part to type in: an imported chain's reply is saved directly and stays editable, while a compaction — summary or error part — would open an empty editor with one inert Rerun. `findMessageById` indexes the messages array once per identity, so resolving the parent no longer scans the thread per row: 100 full-array walks while mounting a 200-message conversation become one. * 🏷️ fix: Honor the Server's Compaction Marker on Any Leaf Compact runs on whatever leaf the branch ends with, so a compaction can hang off a user message rather than off the answer it summarized. Inferring the turn from its position alone left the rerun controls on that one: replaying the parent would answer the user's earlier message again instead of redoing the compaction. `isUserInitiatedCompaction` reads the marker the server already stamps — `markCompactionSummary` sets `initiatedBy: 'user'`, and nothing else writes it, so an automatic summary detour keeps its controls. A marked turn is excluded from the rerun shapes wherever it hangs. * 🎨 fix: Ask the Editor What It Can Edit The hover row had its own idea of an editable part and counted artifact-bearing text as one, while `EditContentParts` keeps an artifact in its read-only renderer. A chained model turn whose only content is an artifact therefore still offered Edit, and opening it gave a read-only artifact, a disabled Save and a Rerun the guards refuse. `editableParts.ts` now owns that rule for both: `isEditablePart` is the editor's own filter, `hasEditablePart` asks it per message. The row resolves it lazily — the artifact check parses markdown, and only a turn with nothing to replay needs the answer — so a streaming row never pays for it. * 🚫 fix: Withhold the Editor's Rerun Instead of Refusing It Both editors resolved the parent inside the handler and returned when it was not a user turn, so a chained model turn — an imported thread with no user message between two replies — kept an enabled Rerun that did nothing, with Ctrl/Cmd+Enter bound to it. Each editor resolves that parent at render and withholds the action instead: the button is gone, the shortcut is inert and `aria-keyshortcuts` no longer advertises it, while Save keeps writing the edit. An unresolvable parent keeps the action, matching the hover row. * 🧹 style: Sort the New Imports in Both Editors CI's import-sort check failed on the two files the previous commit touched. * 💬 fix: Report Unsaved Changes When No Rerun Is Offered A save-only editor still warned about the rerun. An answer's draft is discarded by the rerun, and multi-part edits must be saved before one, but neither is true where the action is withheld — a model turn chained onto another model turn — so the status slot claimed a precondition for something the footer does not offer. Both statuses are gated on the same `canRerun` the button is: the plain-text editor reports unsaved changes, and `rerunRequiresSave` cannot hold without a rerun to require it. * 🧪 test: Exercise the Compaction Controls in the Browser Eight mock-harness scenarios drive the real row and the real editors: a finished, failed, error-part and user-parented compaction withhold Edit, Regenerate and Continue while Copy stays; a turn that only auto-summarized keeps all three; a reply chained onto a reply keeps a save-only editor whose status reports unsaved changes; an artifact-only chained reply loses the editor; and the context indicator still offers Compact after a failed compaction. `SeedMessage` carries `content`, `finish_reason` and `model` so a branch can be seeded in the shapes the mock model cannot produce — a persisted summary, an error part, an artifact. * 🩻 feat: improve agent filter controls for insights (#15807) * feat(insights): improve agent filter controls * feat(insights): add search and fixed agent selection controls * fix(insights): show search before all agents control * test(e2e): allow steer consumption before arm response * 🔠 fix: Name Artifact Downloads After Their Titles (#15836) * fix: Use artifact titles for download filenames * fix: Match artifact filenames to downloaded content * fix: Preserve raw artifact extensions and GFM headings * fix: Download complete artifact files and retain filename provenance * fix: Distinguish cached artifact exports from originals * fix: Name unnamed artifact originals by their media type * fix: Recognize legacy CSV MIME in artifact filenames * fix: preserve context insights across reloads and configuration changes * fix: retain primary call details when resuming context snapshots * fix: preserve resumed output and retained tool-result estimates * fix: keep resumed output compatible with older clients --------- Co-authored-by: Ravi Kumar L <upman@users.noreply.github.com> Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
|
ab6cde0c79
|
⏳ fix: Separate Temporary and General Chat Retention (#15734)
* fix: separate temporary and general chat retention * fix: inherit retention when branching messages * fix: harden retention lifecycle boundaries * fix: preserve subagent retention context * fix: trust stored chat retention for uploads * perf: overlap generated file retention lookup * fix: derive retention from trusted chat state * fix: preserve retention across the full turn lifecycle * test: accept the conversation retention read adapter * fix: carry retention deadlines through durable job saves * fix: resolve retention at cached and new chat admission * fix: resolve direct tool retention before execution * fix: preserve retention across assistant writes and direct tools |
||
|
|
ba12eaa418
|
⛔ fix: Bind Tool Approvals to Generation Scope (#15724)
* fix: bind tool approvals to generation scope * fix: gate generation scope on replay lifecycle support * chore: add npmrc for npm v12 support, fix xlsx tarball dl * chore: bump agents sdk to v3.8.3 * fix: require scoped approval replay support |
||
|
|
cdc3339a78
|
🪬 fix: Fence Agent Checkpoint Ownership and Cleanup Recovery (#15680)
* 🔒 fix: Isolate Checkpoint Cleanup by Generation Ownership * 🔒 fix: Harden Checkpoint Cleanup Ownership * 🔒 fix: Retain Checkpoint Cleanup Evidence * 🔒 fix: Persist checkpoint cleanup receipts * 🔒 fix: Bound checkpoint cleanup receipt lifecycle * fix: Bind checkpoint cleanup to durable ownership and deletion intent * fix: Bind event actor checkpoint storage to authenticated owners * fix: Resolve exact actor checkpoint ownership before lifecycle cleanup * fix: Preserve actor storage compatibility with durable ownership and prune work * fix: Defer checkpoint erasure until conversation deletion commits * fix: Reclaim orphan actor ownership and batch deletion lookups * fix: bind actor checkpoint ownership to payload rows * fix: isolate actor readers and await account persistence * fix: gate owned suspensions and drain complete account state * fix: unify checkpoint deletion lifecycle and storage authority * fix: Replay checkpoint deletion from durable owner state * fix: retain checkpoint cleanup identity before topology loss * fix: complete checkpoint catalog capture and retirement lifecycle |
||
|
|
e370ce4b21
|
🎒 fix: Bound Agent Attachment Context (#15694)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
|
||
|
|
0b9dc209b0
|
🧫 fix: Verify Early Stream Recovery (#15676)
* feat: harden early buffer recovery observability * fix: fence recovery to emitted frontier * fix: close recovery observability races * fix: require atomic recovery capabilities * fix: fail incomplete recovery attachments closed * fix: preserve legacy store contract * fix: correlate failed overflow persistence * fix: break recovery type dependency cycle * fix: reconcile distributed recovery lifecycle * fix: preserve content snapshot contract * fix: harden cross-replica recovery lifecycle * fix: lease recovery subscriber state * fix: close recovery lifecycle edge cases * fix: close recovery validation races * test: align subscriber lease cleanup * fix: harden subscriber lease lifecycle * fix: finalize subscriber lease handoffs * fix: reconcile recovery error replacements * fix: preserve durable recovery frontier * test: expect durable frontier gaps * fix: scope durable snapshot protocol * fix: trust durable terminal recovery payloads * fix: close overflow recovery rollout races * test: bound overflow race fixture memory * fix: bound overflow marker refresh * test: reduce recovery fixture memory * fix: fence overflow recovery admission * fix: close overflow recovery admission races * fix: scope overflow admission fencing * fix: preserve owner replay after remote admission * test: await pre-admission append fencing * test: reflect pre-admission durability |
||
|
|
2d794a0439
|
💓 fix: Detect Silently Dead Redis Sockets With Deadline-Bounded Heartbeats (#15539)
* fix: Detect silently dead Redis sockets with deadline-bounded heartbeats A Redis peer can vanish without a FIN or RST (a dropped NAT entry, a proxy failover, a migrated VM). The socket stays `ready` from ioredis's point of view, no error fires, and every command on it waits for the kernel's retransmission timeout — about fifteen minutes at Linux defaults. Observed in production: while the main client's socket was dead, POST /api/agents hung until Cloudflare returned a 524; while the subscriber's socket was dead, every attach failed with "Timed out synchronizing Redis subscription" for sixteen minutes across five conversations. `REDIS_PING_INTERVAL` could not help — it awaits ping() with no deadline, so the probe just joins everything else that is waiting. - Add startRedisHeartbeat: races a PING against a deadline and destroys the socket when the probe goes unanswered, so ioredis reconnects, replays queued commands, and re-subscribes a subscriber's channels. - Destroy the stream rather than disconnect(true), whose FIN a vanished peer never acknowledges; cluster clients fall back to the regular reconnect. - Add createIoRedisSubscriber so every dedicated subscriber gets an error listener and its own heartbeat, and route the generation-stream and both subagent-routing subscribers through it. - Set ioredis keepAlive from REDIS_KEEP_ALIVE (default 10s). - Add REDIS_PING_TIMEOUT, REDIS_SUBSCRIBER_PING_INTERVAL and REDIS_KEEP_ALIVE, documented in .env.example. - Give the fake RESP server a silent mode and a SUBSCRIBE reply so the dead-peer case can be tested against real clients. * fix: Probe each cluster node and refuse a non-positive heartbeat deadline A cluster-level PING is routed to one node, so a healthy reply proved nothing about the other sockets. The heartbeat now resolves its targets each tick — every node for a cluster, the client itself otherwise — and races an independent deadline per target, tearing down only the socket that stops answering. A non-positive deadline disables the heartbeat with a warning instead of forcing a reconnect on every tick, and the single-probe test no longer depends on wall-clock slack. |
||
|
|
d346e5ab5e
|
🚰 fix: Record the Provider Drain When Shutdown Cuts a Generation Short (#15532)
* 🚰 fix: Record the Provider Drain When Shutdown Cuts a Generation Short
* fix: Await generation settlement on shutdown instead of publishing the drain
* fix: Spend the real shutdown budget and cover the initialization window
* fix: Keep the shutdown registration literal the index spec asserts on
* fix: Keep draining settlements registered while shutdown is waiting
* fix: Track provider executions at the begin/drain chokepoints instead of per caller
* fix: Open the provider tracker before the begin CAS and cap cluster workers to the primary deadline
* fix: Propagate the cluster deadline and release trackers on reconfigure and abandonment
* fix: Hold SIGTERM until the worker acknowledges the cluster deadline
* fix: Refuse provider starts after shutdown begins and harden the cluster deadline handoff
* test: Assert the multi-execution shutdown wait with both executions begun before shutdown
|
||
|
|
8a46ca070c
|
🧊 feat: Persist Context Fading Tier Across Agent Runs (#15505)
* 🧊 feat: Persist Context Fading Tier Across Agent Runs Carry the pruner's latched context-fading tier from `@librechat/agents` through `contextMeta` the way `calibrationRatio` already travels, so a historical tool result keeps the same truncated bytes from one run to the next and Anthropic's prefix-based prompt cache survives across turns (danny-avila/agents#497). - Add `IAgentFadingTier` to the message and conversation `contextMeta` types, schemas and the data-provider zod schema - Add `isAgentFadingTier`, `resolvePersistableFadingTier` and `resolveRunContextMeta` in `packages/api`; a tier is persisted once it carries information (masking active or a budget below the window), and a latched tier is kept even at a neutral calibration ratio - Seed `createRun({ fadingTier })` from the parent response's contextMeta regardless of encoding, since caps are character-based; the field rides the existing forward-compatible `runConfig` variable and older SDK versions ignore it, as does `Run.getFadingTier` being absent - Accept a valid `fading` field in event actor context and reject a malformed one, matching the calibration checks - Capture run context meta through one module-level helper for both completion paths instead of two inline copies Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Seed Fading Tier Across HITL Resume, Validate at the DB Layer Address review findings on fading tier persistence: - Leave the "is this tier worth persisting" decision to the SDK, whose `Run.getFadingTier()` now returns only informative tiers; comparing against the client's window misclassified untouched conversations whenever a reserve ratio was configured - Carry `contextMeta` through the HITL pause projection (staged approval, job metadata, resume state) and seed the rebuilt client from it, so a paused turn resumes with the same tier and calibration instead of re-deriving a shallower one and rewriting the prefix - Share run seeding between `chatCompletion` and `resumeCompletion`, and make capture tolerate partial runs and bare resume contexts - Validate `contextMeta.fading` in `commitAgentEventActorState` with a shared `isAgentFadingTier` guard in data-schemas, so a malformed tier is rejected at the DB layer instead of silently cold-starting the actor - Reuse `IAgentEventActorContextMeta` for `IMessage.contextMeta` instead of extending an inline duplicate Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 test: Cover Resume Seeding From Paused Context Meta Assert the resume controller seeds the rebuilt client from the context meta captured at the pause before rebuilding the run, and move the capture helper's JSDoc back above its function. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧹 style: Sort Imports in Fading Tier Plumbing Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Persist Fading Tier Through Redis Jobs and Branches Address Codex and Copilot review findings on fading tier persistence: - Deserialize `contextMeta` in `RedisJobStore`, whose explicit read mapper rebuilt every other pause field but this one, so Redis-backed deployments resumed with no tier to seed after every HITL pause - Copy server-private `contextMeta` onto the assistant message created by `POST /api/messages/branch`, so the next turn seeds its pruner the same way it would from the parallel source response - Constrain the `fading` subdocuments in both Mongoose schemas (version enum, positive budget, required fields) to match the zod schema - Throw a distinct "fading tier is invalid" error from both context meta validators instead of reporting a calibration failure Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Carry Context Meta Through Stop and Re-Pause Addresses the second Codex pass on the fading-tier persistence: - A re-pause whose resumed segment has nothing persistable now clears the job's `contextMeta` in the same transition instead of leaving the first pause's calibration and tier for the next resume to seed from. - The client publishes the run's live calibration and fading tier onto the job after each pre-invoke context snapshot (deduplicated on value), and the Stop path copies `jobData.contextMeta` onto the stopped response, so a follow-up from a stopped turn keeps the prompt prefix stable. - Mid-run captures (HITL pause, Stop) read the graph's live state; `Run` refreshes its own getters only after the stream settles, so the pause used to persist the seeds it started from. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Fence Context Meta Publication Ahead of Stop A Stop handled by another replica reads the job before the owner is signaled, so a fire-and-forget publish could lose the race, and a run stopped before its first context snapshot never published the tier it inherited. The client now publishes the inherited seed before the run streams or resumes, and the context-usage handler awaits each snapshot's publish so the write lands before the model call it describes begins. Failures still only log; the pre-run publish is optional-called so bare resume contexts keep working. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Read Back Context Meta After the Abort Claim The run owner publishes its context meta ahead of each model call and awaits the write, so a publish landing between the abort's initial job read and its terminal claim describes the call whose partial output the abort snapshot carries. abortJob now reads the same-epoch job back after the claim and content re-read and takes its context meta, so the stopped response persists the tier that produced its bytes. The refresh is best-effort and logs on failure like the content refresh beside it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Share In-Flight Context Meta Writes and Clear Neutral Live State Two snapshot callbacks in a parallel-agent run can capture the same tier before the first write settles; the publisher now caches the in-flight promise per serialized value so the second caller awaits the same durable write instead of treating an uncommitted value as published. A live snapshot with nothing left to persist after an earlier publish now writes a neutral record (ratio 1, no tier), since a running job's fields cannot be deleted through the metadata writer, so a Stop no longer persists an earlier non-neutral state onto the response. The pre-run seed publish is unchanged and is told apart from live snapshots by a flag from the context-usage sink. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 chore: Drop Unused Rest Parameter in Context Meta Test Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 feat: Persist Per-Agent Fading Tiers as Compact Context Meta Reconciles the host with @librechat/agents at 9aacad1a, where graph history stays canonical and each Run derives a provider-only projection from the latched tiers. LibreChat therefore persists only compact state: the calibration ratio, the default agent's tier, and now the per-agent tiers from Run.getFadingTiers(), stored as validated entries so agent IDs never become MongoDB field names and restored onto a null-prototype record for RunConfig.fadingTiers. Truncated messages, canonical tool content, projection provenance and per-message truncation state are never persisted; the SDK's latched flag is stripped on capture. The per-agent map travels everywhere the single tier already did: response contextMeta, event-actor checkpoint state, HITL pause and resume job metadata in both stores, the Stop path, and branch creation. A shared Mongoose definition keeps the message and conversation schemas identical, the zod schema validates entries, and commitAgentEventActorState rejects malformed maps. Comments no longer describe graph messages as truncated; only the provider projection is. Tests cover: tool results and inputs reach the persisted content parts at full size; contextMeta carries exactly the compact tier and calibration fields; a persisted snapshot round-trips into the seeded RunConfig unchanged and prototype-safe; per-agent tiers survive pause, Redis job round-trips, and event-actor context validation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Restore Event Actor Types and Publish Seeds Before Abortable Setup The previous commit's doc rewording in the conversation types dropped the event-actor interfaces that precede the fading tier, which broke the declaration bundle in CI; they are restored unchanged. The inherited context meta is now published onto the job as soon as the parent's state is loaded and awaited at the top of chatCompletion and resumeCompletion, ahead of run creation and the other abortable setup stages, so a Stop during setup still persists the parent's tiers onto the stopped response. The partial response saved when every subscriber disconnects now carries the job's context meta like the Stop and pause paths do. The branch route keeps the source's context meta in the saved message but strips it from the HTTP response, matching client reads. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Serialize Distinct Context Meta Publications Overlapping snapshots from parallel agents could carry different tiers; each started its own write and both job stores keep whichever write finished last, so an older snapshot could overwrite a newer one. Each distinct publication is now issued after the previous one settles, in call order, so the newest snapshot is what a Stop reads back. Equal values still share one write. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Project Root Message Reads and Keep Inherited Meta on Setup Failure The root GET /api/messages paths returned raw documents: the single-message query read called getMessages without a projection and the cursor page performed an unprojected find. Both now apply CLIENT_MESSAGE_SELECT, with getMessagesByCursor accepting a select option, so the persisted fading tiers never reach the client through the paginated endpoint; the search-hydration read projects the same way. A turn that loads a parent's context meta but fails before its run exists used to replace the inherited meta with the capture of a missing run, so the persisted error response lost the last valid tiers. Both finalizers now keep the inherited meta when no run was created while a created run's neutral state may still clear it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 refactor: Move Context Meta Publication Into packages/api The publication coordinator (ordering, deduplication, in-flight sharing, failure state) now lives in packages/api as createContextMetaPublisher, with selectRunContextMetaToPublish deciding what a publication carries; the legacy client keeps only the wiring to the job metadata writer. A transiently failing write is retried with a short backoff before the publication is reported and forgotten, so a snapshot's model call no longer proceeds on the first rejection while the job holds the previous record. The SDK swallows handler errors, so retrying is the only way to strengthen the fence without failing the turn. A terminal save now supplies contextMeta: null when the finished run has nothing to carry, and saveMessage unsets a previously stored value in that case, so a disconnect snapshot's record cannot outlive the run that completed neutrally. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Widen the saveMessage Method Type for the Null Unset The exported method interface still typed contextMeta as the message field, so the spec exercising the null unset failed typechecking; it now mirrors the implementation signature. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Publish Every Agent's Snapshot, Sanitize Search Hits, Unbound Agent IDs The context-usage handler now awaits the run's context meta publish for every ON_CONTEXT_USAGE event, hidden sequential agents included, while the visible-snapshot bookkeeping and the client emission keep their gate; a Stop before the next visible snapshot finds the tier a hidden agent's model call latched. Search results are built from the hydrated hit minus its server-private contextMeta, since Meilisearch hydration projects every schema field. The per-agent tier entry guard no longer caps the agent ID length: ephemeral agent IDs encode endpoint, model and sender without a bound, and dropping such an entry would let that agent alone re-derive its tier on the next turn. Also formats the publication spec so the static checks pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Unset Context Meta Atomically, Keep Publication State After a Failed Write - `saveMessage` folds the `contextMeta` unset into the same update that persists the terminal response, on both the plain and the provenance-merge paths, so a failure between two writes can no longer leave a completed row carrying a disconnect snapshot's state. - `createContextMetaPublisher` tracks whether any record has committed separately from the retryable latest publication, so `hasPublished` stays true after a later publication exhausts its retries and a following neutral snapshot still overwrites the earlier record on the job. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Carry Context Meta Through the Resumable Stop Route and Neutral Resumed Completions - The resumable Stop route copies the refreshed `jobData.contextMeta` onto the terminal assistant message it persists, as the abort middleware already does, so a stopped resumable response keeps the tiers that produced its partial output. - A resumed run that completes with neutral context state now saves `contextMeta: null`, which unsets the paused segment's record instead of letting an omitted field keep it on the completed row. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Trust Only Server-Authored Context Meta, Unset It on a Neutral Stop, Read It From the Job on Disconnect - `POST /api/messages/:conversationId` and conversation imports drop any client-supplied `contextMeta`, and the run seeds its calibration and fading tiers only from a server-authored parent response, so a forged tier can never shape a provider projection. - Both Stop paths save `contextMeta: jobData.contextMeta ?? null`, so a job that re-paused with neutral state unsets what an earlier pause stored on the row instead of leaving it behind. - The disconnect partial save reads the same-epoch job record for the run's published context meta, since the client-facing resume snapshot never carries server-private state. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧪 test: Mock the Job Store in the Resume Metadata Spec The disconnect partial save now reads the same-epoch job record through `GenerationJobManager.getJobStore()`, so the resume metadata spec's manager mock provides it as the other disconnect spec already does. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 🧊 fix: Keep Context Meta Off the POST Message Response `POST /api/messages/:conversationId` returned the saved row verbatim, so a client write against an existing server-authored message echoed the row's stored context meta. The route now projects the row through the same client projection the branch route uses, which drops the server-private field. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF * 📦 chore: Bump @librechat/agents to 3.7.16 for Persisted Fading Tiers 3.7.16 is the first release that carries danny-avila/agents#497, so `Run.getFadingTier()` / `getFadingTiers()` now report the latched tier this branch persists and `RunConfig.fadingTier` / `fadingTiers` seed the next run from it. The dependency set is unchanged from 3.7.15; only the package entry moves. Adds an SDK-backed round trip to the fading spec: a first pruner latches an informative tier, `resolveRunContextMeta` reduces it to the persisted shape, and a second pruner seeded from that shape reproduces the first run's projected tool-exchange bytes under instruction and calibration drift, while an unseeded pruner under the same drift relaxes its tier and rewrites them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
29f738dea8
|
⏭️ feat: Honor an Interrupt Before the Model Has Answered (#15491)
* ⏭️ feat: Honor an Interrupt Before the Model Has Answered An interrupt armed while the model is still thinking sat as "Interrupting" until the entire turn finished, then landed as a terminal continuation. The cause is upstream: the SDK reads `shouldPreempt` once per streamed chunk, and its seal gate requires non-empty text — so a silent provider is never polled at all, and a reasoning-only turn is never sealable. `@librechat/agents` gains a discard-and-restart path for exactly that window (`HOOK_PREEMPT_RESTART_CAPABLE`). This wires the host half: - `GenerationJobManager.subscribePreempt` registers run-scoped wake listeners, notified after an ACCEPTED arm — local or cross-replica, since both land in `armPreemptIds`. Fenced on `createdAt` like every other preempt entry point, and a throwing listener never fails the arm a queued steer depends on. - `createSteerPreemptPoll` hands that channel to the SDK as `StreamPreemption.subscribe`, gated on a new `isSteerPreemptRestartSupported` probe so an SDK that cannot act on a wake is never handed one. The wake is a hint only: `isPreemptRequested` stays the single authority, so the level-triggered contract the seal path depends on is unchanged. Inert until `@librechat/agents` publishes the restart contract — the probe returns false against the pinned 3.7.11 and the run keeps today's behavior. * 🩹 fix: Replay an Already-Armed Preempt on Subscribe Preempt requests are level-triggered, and the window between a job becoming steerable and the SDK installing its model-attempt listener is real. An interrupt landing there was recorded with no callbacks to notify, and on a silent or reasoning-only turn no later chunk poll may ever run — leaving the request to wait out the whole turn, the exact stall this channel removes. `subscribePreempt` now replays an existing arm to the newly registered listener only, after registration: waking the whole set would re-notify runs that already looked, and waking before registration would leave a concurrent arm with nowhere to land. The `createdAt` fence is unchanged and still gates the replay. The SDK also reads the level flag once at each attempt's start, so this is belt-and-braces rather than the sole cover — but it makes the host correct on its own terms instead of resting on when the SDK happens to look. * 🔧 chore: Update @librechat/agents dependency to version 3.7.14 in package-lock.json and related package.json files * 🔧 chore: Update dependencies in package-lock.json and package.json - Bump versions for @humanfs/core (0.19.1 to 0.19.2), @humanfs/node (0.16.6 to 0.16.8), @xmldom/xmldom (0.8.13 to 0.8.15), and qs (6.15.2 to 6.16.0). - Add @humanfs/types (0.15.0) as a new dependency for @humanfs/node. - Update dependencies for side-channel and side-channel-list to their latest versions. |
||
|
|
6122d210da
|
🛄 fix: Reclaim Pending MCP OAuth Prompts on Resume (#15471) | ||
|
|
a34d5dbfc7
|
🪂 fix: Preserve Subscribers Through Terminal Handoffs (#15462)
* fix: preserve fenced terminal subscriber handoff * test: parse replay conversation pathname * fix: capture late terminal subscribers |
||
|
|
cdfe54c349
|
🚦 fix: Downgrade Stale Generation Fence Telemetry (#15452)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
Docker Dev Images Build / publish (push) Waiting to run
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
* fix: downgrade stale generation fence telemetry * fix: preserve fenced persistence failures * fix: avoid duplicate fence warnings * fix: propagate terminal publication fences * style: format fencing audit changes * fix: keep successor transitions generation scoped * fix: bound predecessor handoff retirement * fix: reconcile discovered successors completely * fix: preserve successor handoff acknowledgement |
||
|
|
c7f0a8f247
|
📬 feat: Move Agent Queued Turns Server-Side (#15370)
* feat: move agent queued turns server-side * test: update queued turn composition coverage * fix: harden queued-turn lifecycle ownership * fix: centralize queued-turn lifecycle ownership * fix: close queued-turn lifecycle gaps * fix: unify queued turn lifecycle transitions * fix: fence queued-turn deletion admission * fix: preserve queued-turn lifecycle ownership * fix: close queued-turn reconciliation races * fix: reconcile quarantined queued-turn admissions * fix: annotate admission evidence default * fix: deepen queued turn lifecycle admission * fix: fence queued turn provider admission |
||
|
|
dc77b78d3e
|
🔇 refactor: Quiet Framework Logging in Unit Tests (#15363)
* 🔇 refactor: Quiet Framework Logging in Unit Tests Unit test output was overwhelmingly framework log lines rather than test results. A `packages/api` run printed 9,076 lines for 434 suites; 1,596 of them were the identical `at Console.log (winston/transports/console.js:87)` frame that Jest staples onto every winston write. Three causes: - Each winston logger built its Console transport with a hard-coded `level: 'info'`. Winston resolves a transport's explicit level ahead of its parent's, so the logger's own `NODE_ENV`-derived `warn` never applied — in tests or in production. `CONSOLE_LOG_LEVEL` now drives it, defaulting to the previous `info` so deployments are unaffected. `meiliLogger` had the same hard-coded level and is routed through the same helper. - `LOG_TO_FILE` defaults on, so every Jest worker opened DailyRotateFile transports. A run left `packages/api/logs/` holding a 107KB error log, a gzipped rotation, audit JSON, and — from fake-timer suites — files dated 1969 and 2023. - Client suites carried leftover debug `console.log` in shipped source (`Test mode. Skipping silent refresh.` alone accounted for 109 blocks) plus three test-harness defects whose React error dumps buried everything else: a markdown suite rendering Mermaid with no Router, a `~/Providers` mock missing `useSearchContext` (the component's own catch swallowed the throw, so the sources path was dead while the suite passed), and `getBoundingClientRect` stubs without `left`, which made a computed `right` NaN. A shared `config/jest.setup.logging.cjs` sets the two env vars for both backend workspaces. It sets env only and requires nothing: eagerly requiring the logger froze `CREDS_KEY` into `encryptV3` before specs could set it and broke five suites. `TEST_VERBOSE_LOGS=true` restores the logs for debugging. Output per full run: packages/api 9,076 -> 770 lines (winston lines 1,596 -> 0) packages/data-schemas 2,521 -> 946 lines (winston lines 290 -> 0) client 10,384 -> 3,153 lines (console blocks 285 -> 74) Test counts are unchanged: client 476 suites / 5,827 tests green, and the Ariakit `act(...)` warnings the selector suites emitted are gone rather than suppressed. * 🔧 fix: Settle Shutdown Timers Before Counting Them `destroy()` finishes cancelling its fenced retirement timers on the tick after it resolves. The count only returned to the baseline because a console write from the logger happened to yield first — silencing that logging left four timers still pending at the assertion, deterministically failing the suite in CI's sharded, coverage-enabled run. Advance the fake clock by zero before counting, so the assertion proves the timers were cancelled rather than that something incidental yielded. * 🛡️ fix: Validate CONSOLE_LOG_LEVEL Instead of Trusting It Winston resolves a level name against the logger's level map, so a name outside it resolves to `undefined` and the transport drops every message. Passing `CONSOLE_LOG_LEVEL` straight through meant a typo (`warning`) or a stray space silently muted all console output — a misconfiguration that reads as a dead deployment. Validate against the level set, normalize case and whitespace, and fall back to the default with a warning on the way past, since the logger cannot report its own misconfiguration. The level map itself was duplicated verbatim in both loggers; it now lives in `utils` beside the validation that depends on it. * 🎚️ fix: Let CONSOLE_LOG_LEVEL Outrank DEBUG_CONSOLE `DEBUG_CONSOLE=true` forced the console transport to `debug`, so on those deployments every non-`silent` value of `CONSOLE_LOG_LEVEL` was discarded — `CONSOLE_LOG_LEVEL=error` still emitted debug and info, defeating the control it advertises. `DEBUG_CONSOLE` now only moves the *default* level and keeps choosing the debug format, so an explicit level stays in charge of verbosity. With no explicit level the resolved default is still `debug`, unchanged for existing setups. Covers the wiring with a spec that re-imports the logger per environment, rather than only testing the resolver in isolation. |
||
|
|
cd3768ed1f
|
🍵 feat: Continue Late Steers in Warm Agent Runs (#15357)
* feat: continue late steers in warm agent runs * chore: bump agents sdk to v3.7.9 * fix: guard terminal steer admission |
||
|
|
2ae6c8aea9
|
🛎️ feat: Wake Agents on Background Tool Completion (#15350)
* feat: wake agents for background tool completion * fix: isolate background completion contracts * fix: anchor background completion identity * fix: close background completion delivery gaps * fix: preserve manual background polling * fix: preserve legacy tool group identity * fix: close background wakeup identity gaps * test: type background wakeup enqueue mock * fix: preserve background completion identity * test: type activity phase fixture * test: mock phase media query * fix: arbitrate background result ownership * fix: retain tool step routing helper * test: expand phase groups before identity checks * fix: preserve background completion ownership * fix: bound durable background results * fix: type wakeup input budget export * chore: sort background handler imports * fix: persist timed-out background completions * fix: reconcile background completion ownership * fix: type background completion capabilities * fix: harden background completion terminalization * test: type missing completion evidence * fix: require evidence for completion retirement * chore: satisfy background completion static checks * test: cover automatic background completion wakeups * test: assert background wakeup agent identity * fix: wake capability-fenced trigger deliveries * test: keep capability shield fixture public * test: assert capability worker wakeup * fix: close background completion ownership gaps * test: satisfy completion lease static checks * fix: preserve artifact completion wakeups * fix: close background delivery recovery gaps * fix: simplify background receipt guidance * fix: recover dead background completion batches * fix: fence background completion recovery * fix: recheck background recovery ownership * fix: fence unpublished background continuations * test: await background message hydration |
||
|
|
e3ccaba5af
|
🛄 feat: Restore Compaction Guidance Across Continuations (#15356)
* 🧭 feat: preserve compaction guidance across continuations * 🔧 fix: narrow persisted compaction fields |
||
|
|
70f735336d
|
🛎️ fix: Enroll Remote Agent Runs in the Generation Lifecycle (#15349)
* fix: enroll remote agent runs in generation lifecycle * fix: close remote lifecycle ownership gaps * fix: close remote conversation drain races * fix: fence remote runs during conversation deletion * fix: reconcile remote deletion and settlement races * fix: complete owner deletion recovery * fix: preserve remote cleanup receipts * fix: consume deletion receipts before cleanup * fix: expose idempotent deletion option * chore: sort remote lifecycle imports |
||
|
|
7a0061507e
|
🪪 fix: Admit Confirmed Generation Retries Before Message Limits (#15341)
* fix: admit idempotent retries before message limits * fix: bound generation retry admission * fix: keep retry probe store-compatible * fix: exclude normalized resume routes * fix: preserve trusted retry exemptions * fix: bound retry claim admission * style: satisfy generation retry static checks |
||
|
|
3fa33b740b
|
🛫 refactor: Promote Generation Protocol V2 Automatically (#15324)
* refactor: promote generation protocol v2 automatically * fix: remove unused protocol import |
||
|
|
77c2a51cf3
|
🪧 fix: Advertise Detached Event Actor Support from the Generation Store (#15322)
Some checks failed
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Has been cancelled
Frontend Unit Tests / Build packages (push) Has been cancelled
Frontend Unit Tests / TypeScript type checks (client) (push) Has been cancelled
Frontend Unit Tests / Tests: @librechat/client (push) Has been cancelled
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Has been cancelled
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Has been cancelled
Frontend Unit Tests / Vite build verification (push) Has been cancelled
* fix: activate detached event actions automatically * fix: shield detached terminal generations * perf: share capability availability index * style: flatten capability status selection * fix: preserve mixed-version lifecycle shells * fix: complete detached action store adapters * fix: close mixed-version capability races * fix: honor legacy capability success |
||
|
|
487984193a
|
🪂 feat: Make Event Actor Detached Actions Durable (#15307)
Some checks are pending
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
* feat: make Event Actor detached actions durable * fix: align detached actor checks with merged base * fix: close detached actor lifecycle gaps * fix: break detached action outcome cycle * fix: deepen detached action ownership * fix: fence detached launch handoffs * fix: preserve detached rollout fencing * test: type detached recovery failure * fix: close detached ownership handoffs * fix: stage detached action activation * fix: persist detached terminal retries * fix: fence detached durability ownership * fix: fence trigger lane publication * fix: fence detached transition generations * fix: retire failed detached predecessors * fix: close detached rollout consumers * test: type detached retry turns * fix: close detached ownership gaps * fix: fence detached action resumes * fix: fence prior-head event actor resumes * fix: preserve legacy event actor resumes * fix: align detached resume boundaries |
||
|
|
c06b09c945
|
🤖 feat: make Event Actor HITL durable (#15305)
* 🤖 feat: make event actor HITL durable * 🤖 fix: break event actor outcome cycle * 🤖 fix: close durable actor terminal races * fix: harden durable event actor recovery proofs * fix: close event actor resume publication races |
||
|
|
8ba76507ae
|
🧷 fix: Verify Scheduled HITL State Before Pausing (#15284)
Some checks failed
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
Publish `librechat-data-provider` to NPM / pack (push) Has been cancelled
Publish `librechat-data-provider` to NPM / publish-npm (push) Has been cancelled
* fix: require durable storage for scheduled HITL * fix: verify scheduled HITL checkpoints before pause * fix: narrow scheduled HITL capability checks * fix: honor ask tool filters in schedule preflight * fix: enforce durability across scheduled HITL resumes * fix: scope scheduled HITL admission to active tools * refactor: move scheduled HITL admission to TypeScript * fix: close scheduled HITL admission gaps * fix: bind scheduled pauses to current checkpoint * fix: cap pending actions at binding deadline * fix: reject expired pending actions * fix: guard approval deadlines atomically |
||
|
|
44edcbe014
|
🧾 feat: Store Durable Event Actor Receipts (#15265)
* 🧾 feat: Store durable event actor receipts * 📊 fix: Scope actor delivery metrics * 🧱 fix: Close durable receipt recovery gaps * fix: serialize actor action admission * fix: retire terminal batch members * fix: close actor terminal recovery races * fix: reclaim lanes after terminal receipts * fix: close actor receipt recovery gaps * fix: preserve ambiguous legacy handling * fix: recover legacy actor outcomes * fix: admit leased actor actions * fix: serialize actor admission recovery * fix: close actor recovery races * fix: token-fence actor admissions * fix: preserve mixed-version actor fences * fix: harden actor rollout and metrics * fix: gate durable actor receipt rollout * fix: enforce base actor receipt rollout * fix: align experimental actor rollout |
||
|
|
68fc46a055
|
🪢 feat: Resume Bound Event Actors from Checkpoint Forks (#15227)
* feat: resume event actors from checkpoint forks * fix: fence event actor checkpoint uncertainty * fix: satisfy event actor type contracts * fix: make actor reconciliation recoverable * fix: preserve event actor lifecycle transitions * fix: fence event actor lifecycle outcomes * fix: enforce event actor lifecycle ownership * fix: retain event actor settlement proof * 🔒 fix: Retain Event Actor Receipts Through Repair and Bound Their Journal Repair and compensation deleted the reconciliation row they resolved, which was the only durable proof that the invocation had already applied an external action. A delayed duplicate owner could then reacquire the same invocation id and repeat that action. Both resolutions now retire their receipt to `settled` and record how it settled, so the same-id tombstone survives; `history_repaired` and `action_compensated` still force a cold rebuild. Compensation undoes the effect without re-authorizing the delivery, so a legitimate retry must arrive under a new invocation id. A retried repair converges on its own receipt. Bound the journal so a long-lived actor cannot grow its conversation document without limit: a new fence is admitted only when no active lifecycle row exists, so a capped push can evict nothing but the oldest settled receipts. Stop shipping the unbounded source payload on every bound-child continuation. It rode the delivery body regardless of the feature flag while the sibling `fire` body deliberately sends event identity alone, so a large webhook payload could push a previously working delivery past the chat route's body limit. The actor binds an invocation from identity and never builds the prompt from it. Blank the positional token map on warm continuations. It is derived from the full DB history, while a warm run executes on checkpoint-restored state, so its indices address different messages and the pruner never recounts them — misattributing cached counts to the wrong messages in both directions. Keep the replaced-claim exit on its cleanup path when preserving reconciliation fails: the committing CAS already left a blocking row, so the failed status upgrade costs provenance, not safety. * 🧪 test: Pin the Warm Continuation's Map/Summary Asymmetry Give the warm-continuation client test a populated token map and a real cross-run summary so its assertions bite: the positional map must arrive blank (checkpoint-restored state no longer matches DB-derived indices, and the pruner never recounts a populated entry) while `initialSummary` must pass through unchanged — it rides the system tail and summarizes pre-boundary turns that were excluded from the very history the committed checkpoint was built from, so blanking it would silently drop context no warm run can recover. * ⚖️ fix: Honor Compensation in Settlement and Age-Bound the Receipt Journal A compensated receipt still tombstones its invocation id, but its external effect was explicitly undone — the terminal handler nonetheless replayed every settled lifecycle's stored action as authoritative and settled the public outcome as applied, telling an action-aware source the operation stands and suppressing the new-invocation retry compensation requires. The handler now settles a compensated invocation as failed with an explicit compensation error, overriding even fresh applied run evidence from a replayed generation; verified and repaired receipts continue to replay applied. Receipt eviction is now primarily age-based: a stale same-id owner is bounded by time, not by how many newer invocations settle, so the previous count-only slice let a high-rate actor evict a tombstone while its delayed duplicate owner could still wake and repeat the action. Admission prunes only settled receipts older than a retention window that dwarfs every generation, job, and delivery-retry lifetime, and the count cap is demoted to a raised document-size backstop. * 🔀 fix: Serialize Compensation Against Settlement and Never Evict Live Receipts The terminal handler read its lifecycle snapshot, verified history, and then settled the public outcome — so a compensation resolving the same receipt during that window lost: the handler settled applied from its stale snapshot and no retry could ever change the replay-identity-locked outcome. The receipt's status CAS is now the serialization point: verification resolves the receipt BEFORE settling, whichever transition wins determines the public outcome, and a crash between resolve and settle converges through the retained receipt's replay. The verified-replay probe requires the receipt's own resolution, so a compensated receipt can never satisfy a verification retry. This inverts the settle-before-receipt ordering deliberately: that ordering guarded proof that resolution used to delete, and the receipt now retains its full action proof through resolution. The document-size cap is no longer an eviction quota. A receipt inside its retention window is never discarded: when the journal holds a full cap of unexpired receipts, new invocations are refused fail-closed until receipts age out, making duplicate protection and document integrity simultaneous invariants instead of a rate-dependent trade. * 🎓 fix: Keep Skill-Bearing Event Actors on the Legacy Path Skill primes are spliced into the message list directly ahead of the newest message, and a warm continuation forwards only that newest message — so a checkpoint-restored actor would keep serving the prime bodies baked in at its last cold start and never observe an edited or newly attached skill. Until the actor head carries a context fingerprint that forces a cold rebuild when the agent's skill context changes, agents with always-apply or manual skill primes stay on the legacy path, which re-primes fresh bodies every turn: correct on every event, just never warm. * 📜 fix: Gate Fork Mode on the Skills Capability, Not Just Request-Time Primes History-derived re-priming was a third path into the same staleness class: an actor that previously invoked a skill carries no request-time prime arrays, yet primeInvokedSkills re-resolves that skill's current body from history each turn and the warm slice drops the reconstruction — leaving the checkpoint's old body active after edits. The fork gate now keys on the priming hook itself (present exactly when the skills capability is enabled) alongside the request-time arrays, so every skill-body path routes to the legacy rebuild until #15235's context fingerprint restores warm continuation for skill-bearing actors. * 🧾 fix: Capture Applied-Action Proof at Tool Execution, Not After sendMessage The executor read applied-action evidence from the run-step collection the instant sendMessage resolved, but that collection is populated asynchronously — an applied invocation could classify as actionless (runSteps still empty while the tool result already streamed), discarding its fork and stranding the actor cold while the terminal handler later settled the same delivery as applied from the persisted evidence. Authoritative proof is now recorded in graph context the moment the expected tool executes: the request-owned recorder observes the tool-end chain (which ToolNode dispatches synchronously with both input and output) and applies the same fences as run-step evidence — exact tool name with the MCP-suffixed form, the declared argument subset against the execution input, an error-free result, and the background non-execution receipt exclusion. readAppliedAction consults the receipt first; run-step inspection remains the fallback for paths that bypass the tool-end chain. Regression coverage reproduces the observed ordering: the real executor commits the head from the receipt while run steps are empty, warm-continues the next event, and never re-executes the action; recorder fences and the receipt-first controller wiring are covered separately. * 🎯 fix: Supply Execution Arguments to the Tool End Callback The live Vertex + MCP canary exposed a contract mismatch the synthetic fixtures hid: the ON_TOOL_EXECUTE execution path invoked its tool end callback with output only, while the action recorder must verify the declared argument subset against the execution input. The receipt never qualified, every turn fell back to cold history rebuilds, and the tournament advanced with zero actor heads and zero retained checkpoints while looking successful. The execution handler owns both halves at the same moment, so the fix is at the source rather than a correlation store: ToolEndCallbackData gains the executed call's input and every callback site passes tc.args. A handler-level regression drives the real createToolExecuteHandler and asserts the callback receives both fields; recorder regressions pin the production shapes — an output-only tool end must starve an argument-fenced receipt rather than trust an unfenced match, and still qualifies a name-only expected action. * 🕵️ fix: Mark Background Deliveries So They Cannot Impersonate Applied Actions The background-claim callback reports the ORIGINAL tool's name for artifact attribution on the poll turn that harvests a completed task. A name-only expected action could therefore be impersonated by work some earlier turn dispatched: the recorder would attribute that delivery to the current invocation and commit a head whose state never contained the invocation's own action. The run-step evidence path never had this hole — it sees the poll tool's name — so the recorder must match its provenance discipline. Delivery callbacks now carry an explicit backgroundDelivery marker set at the one site that rewrites the name, and the recorder ignores marked deliveries outright. Regressions pin both halves of the contract: the delivery callback must carry the marker with the poll call's arguments, and a marked delivery can never qualify even a name-only expected action. * 🧿 fix: Version Invalidations, Keep Evidence Ahead of Output Policy, Gate Detachable Actions Three closeout-round findings, each converted into an invariant. Every legacy-path invalidation now advances a durable epoch — including for headless and already cold-marked actors, where the marker alone leaves no CAS-visible trace — and the actor-head CAS requires the epoch observed at preparation. A concurrently prepared fork whose history predates an intervening legacy turn can no longer commit past it; the commit reports an ordinary conflict and journals. Execution identity is now emitted before post-execution output policy: when a side-effecting tool succeeds but its returned content is withheld by the output filter, the callback delivers an outputFiltered receipt with blank content — the recorder accepts it as proof (rejecting model-detached calls it cannot distinguish through the blank shape), the artifact path never sees it, and an applied action is no longer reclassified actionless and re-executed on retry. Background-capable expected actions stay off the fork path: dispatch returns a launch handle every evidence fence correctly rejects, and the completion is provenance-marked as another turn's work, so a fork would settle actionless before the external effect lands with nothing to stop a retry from dispatching it again. The gate mirrors the MCP-suffix name matching of the evidence path. * 🚧 fix: Seal the Whole Legacy Turn Behind a Second Epoch Advance The epoch fenced only the legacy turn's start: a fork preparing after the begin invalidation but before the turn's message persistence observed the new epoch and cold marker, rebuilt from history that did not yet contain the turn, and committed cleanly because nothing advanced the epoch again — making the incomplete rebuild authoritative and clearing the marker. Every legacy event turn now seals its invalidation at terminal persistence with a second epoch advance, on the success, replaced-claim, and error exits alike. Sealing deliberately carries no quiescence requirement — it must succeed while a fork fence is active, because that is exactly the mid-turn race it defeats — and a fork that already committed against the begin epoch is healed the same way: the seal re-marks the head cold, so the next event rebuilds with complete history. Seal failure never diverts the turn's own exit; the begin bump still fences everything prepared before the turn. * 🔗 fix: Replace the Best-Effort Epoch Bump With a Durable Legacy-Turn Fence The second epoch advance could not make a legacy turn atomic, and three findings shared that root cause: two conditional updates left a headless gap a fork could create the head inside; the error exit sealed before saveErrorTurn made the error history durable; and any crash or failure between persistence and sealing left an incomplete fork authoritative, because the seal was best-effort and its failure was swallowed. A legacy turn now carries one durable fence. A token is written before execution by a single update-pipeline write — no two-write gap, and the cold marker is applied only where a head exists via $cond/$$REMOVE. While the token is present no fork may prepare (the adapter refuses) or commit (the CAS requires its absence), because the turn's messages are not yet durable. One atomic write clears the exact token and advances the epoch once history is persisted — after saveErrorTurn on the error exit — and success, replacement, and error exits all route through it. Failure is now fail-closed rather than silent: a failed seal leaves the token set, which keeps blocking forks and is logged as such, and a fence abandoned by a crashed turn is reclaimed only once stale, advancing the epoch and marking any head cold so the next event rebuilds from whatever history actually survived. * fix: serialize legacy event actor turns * fix: close legacy actor fence ownership gaps * fix: preserve legacy actor persistence fences |
||
|
|
290b8664d9
|
🧾 feat: Track Authoritative Agent Event Outcomes (#15213)
* feat: track authoritative agent event outcomes * fix: isolate agent event outcome types * fix: declare agent event handler result * fix: simplify agent event status selection * fix: preserve authoritative event outcomes * fix: preserve terminal event evidence * test: use completed run-step envelope * test: scope deferred HITL question locator * fix: settle every agent event terminal path * style: sort terminal host action imports * fix: fence agent event terminal evidence * fix: recover agent event terminal settlement * fix: scope terminal retry hints by generation * fix: settle terminal host actions exactly |
||
|
|
6d499ba3ce
|
⏳ fix: Anchor Resumed Elapsed Time at the Generation's Real Start (#15204)
Some checks failed
Publish `librechat-data-provider` to NPM / pack (push) Waiting to run
Publish `librechat-data-provider` to NPM / publish-npm (push) Blocked by required conditions
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
Publish `@librechat/client` to NPM / pack (push) Has been cancelled
Publish `@librechat/data-schemas` to NPM / pack (push) Has been cancelled
Publish `@librechat/client` to NPM / publish-npm (push) Has been cancelled
Publish `@librechat/data-schemas` to NPM / publish-npm (push) Has been cancelled
* ⏳ fix: Anchor Resumed Elapsed Time at the Generation's Real Start A reload emptied the Recoil anchor, so the indicator fell back to its mount time and visibly reset to 0s over a run that had been generating for much longer. The stream status the resume path already reads carries the server-recorded generation start; the fill now prefers it, so a reattached run reports real elapsed time. The fill remains fill-only: a same-session reattach keeps its original ask baseline, and the indicator's existing clamp absorbs any client/server clock skew. * 🕰️ fix: Rebuild the Resumed Baseline From the Server-Computed Age Codex round 1: anchoring at the server's raw createdAt compares two clocks — a client behind the server froze the resumed reading at 0s for the skew, one ahead inflated it. The status route now also reports the generation's age computed on its own clock, and the client rebuilds a clock-local anchor as Date.now() minus that age, so each machine only ever compares to itself. Raw createdAt stays as the fallback for an older server mid-rolling-deploy. * 📥 fix: Compute Elapsed Age in TypeScript, Anchor It at Status Receipt Codex round 2: the elapsed computation moves into packages/api as getGenerationElapsedMs — the route now just delegates, keeping the response contract type-checked and the /api surface a thin wrapper — and the client subtracts the age from the moment the status response arrived (dataUpdatedAt) rather than from apply time, so a slow history fetch between receipt and apply can no longer shrink the reading. Declined with rationale: a shared clock source across replicas — the residual is inter-replica NTP drift, milliseconds against the minutes-scale client skew this PR eliminates, and the helper gives any future shared-clock upgrade a single home. |
||
|
|
862ebf3235
|
🪢 fix: Persist Failed Agent Turns Before Error Publication (#14118) | ||
|
|
ac2aef00f6
|
🫗 fix: Drain Quoted Excerpts Into Mid-Run Steering (#15175)
* 🧭 fix: Carry Quoted Excerpts Through Mid-Run Steering "Add to chat" quote chips were dropped by every during-run steer path: the steer POST had no quotes concept, so a composer-origin steer left the chip staged (gluing onto the NEXT send) and a queued item steered into the live run lost its quotes silently. Quotes now ride the steer protocol end to end: - POST + admission: `quotes` on the steer body, normalized like the chat route's (getReferencedQuotes caps), part of the idempotency fingerprint only when present so pre-existing receipts still replay. - Injection: merged into the model-bound turn as Markdown blockquotes at both boundaries (text-only and media paths), mirroring prependQuotes. - Persistence + replay: the STEER content part stores `quotes` separately from the typed text; stampSteerPartMedia re-merges them per turn (even with resendFiles off) via the SDK's transient media stamp, with the quote block folded into the token budget. - UI: composer steers/interrupt-steers drain the chips (skill picks stay staged — they configure a NEW turn's run); SteerPart and the in-flight bubble render the same MessageQuotes reference blocks as user bubbles; queued/failed rows show a quote count; reconnect reseeds fall back to the server item's quotes when no local chip survives. - buildMessages keeps its zero-await path to the parallel context kickoff via a synchronous stamp-target probe. * 🧭 fix: Keep Quotes in the Client-Safe Steer Projection toPendingSteer is the projection behind resume-state pendingSteers, abort responses, and terminal leftover claims — dropping quotes there would lose them on exactly the recovery paths the reconnect reseed's server fallback relies on. * 🧪 test: In-Flight Steer Bubble Renders Carried Quotes * 🔁 fix: Re-Stage Quotes When a Pre-Quotes Replica Accepts the Steer Codex flagged the rolling-deploy window: an old replica 202s a quoted steer while dropping the excerpts, so the client cleared the chips for context the model never received. The 202 (fresh and receipt replay) now echoes quotesAccepted from the DURABLE item; a missing echo on a quote-bearing composer-origin steer re-stages the excerpts as composer chips — the pre-steer behavior, so they ride the next send instead of vanishing — and strips them from the surviving chip so a later terminal conversion cannot duplicate them. Queued-origin steers keep quotes on the item, whose restore paths already return it intact. The residual cross-version lost-ACK retry stays fail-closed as a 409 idempotency conflict (failed chip with retry controls). * 🔁 fix: Close the Remaining Cross-Version Quote-Loss Windows Codex round 2: - Send now of a quoted queued item against a pre-quotes replica now re-stages the excerpts too (the row is consumed and the words inject bare, so the composer is their only remaining home); the strip clears the chip's captured origin copy so reclaims and terminal conversions cannot duplicate them. - A quoted retry whose lost first ACK was accepted by a pre-quotes replica now REPLAYS that legacy receipt instead of 409ing: the stored fingerprint matching the quote-less hash of the same words proves the cross-version case, and the replayed 202's missing echo drives the re-stage. Different quotes against a quote-bearing receipt still conflict. - TSteerAppliedEvent.part gains the quotes field (typed SSE consumers). * 🧪 test: Drop the Stale Narrow SteerDrainOutput Alias The spec's local intersection re-declared injectedMessages with content: string, predating the SDK pin that declares the field natively (content: string | MessageContentComplex[]). Under CI's clean install the hook's BaseHookOutput is no longer assignable to that narrower alias; the plain PostToolBatchHookOutput is the correct type for every drain/boundary assertion. Verified against the published 3.6.16 dist and the local one. * 🔁 fix: Honor the Generation Owner's Quote Capability End to End Codex round 4: - steerQuotesCapable rides job metadata (createJob + HITL resume rewrite), mirroring preemptCapable's owner-recorded pattern: an upgraded admission replica no longer stores quotes — or claims them accepted — for a generation whose older owning drain would silently drop them at injection. The missing echo drives the client re-stage, and a later capable handover cannot double-deliver restored context. - Applied events reconcile dropped quotes: when a quote-less applied part settles a quote-bearing chip (the lost-202 ordering the ACK-echo path cannot see), resolveSteerChip and both reconnect settle paths re-stage the chip's excerpts before removing their only copy. mergeRestagedQuotes dedupe keeps every trigger idempotent for the same excerpts. * 🔁 fix: Re-Read Quote Capability at the Last Moment and Cap Restaged Chips Codex round 5: - A HITL resume rewrites steerQuotesCapable without changing the generation's createdAt, so the enqueue fence cannot see a capable-to-legacy handover landing during admission's awaits. Re-read the owner's flag immediately before item construction (paid for only by quote-bearing requests); the residual between re-read and enqueue commit matches preemptCapable's documented race. - mergeRestagedQuotes now respects the 10-quote contract with the staged chips winning: a restored tail that cannot ride the next send is dropped explicitly instead of rendering as a chip the submission would silently discard. MAX_QUOTE_COUNT moves to utils/steer as the single client source; QuoteButton imports it. * 🔁 fix: Steer Quote Coverage for Preflights, Memory, and Single-Scan Stamping Codex round 6: - Stored-message policy inspection now extracts steer-part quotes as quote fragments (path /content/N/quotes/M), so conversation import and shared link preflights inspect the newly persisted field exactly like top-level message.quotes. - The memory copy gets its own quote-merge stamp (text only, resendFiles false): formatAgentMessages ignores part.quotes, so without it a steer whose substance lives in its excerpt reached the chat model but never memory extraction. - collectSteerStampTargets replaces the boolean probe: buildMessages collects once and hands the targets to stampSteerPartMedia, keeping the zero-await fast path without scanning the history twice. * 🔁 fix: Redis Quote Plumbing, Conversion-Race Guard, and Quote-Bound Recovery Proof Codex round 7: - RedisJobStore.deserializeJob now restores steerQuotesCapable (the explicit mapper otherwise dropped it on every read, leaving quote steering inert in Redis deployments), with the round-trip spec extended. - Both Lua parked-steer projections (terminal close + generation replacement) forward item.quotes, matching toPendingSteer — a lost final no longer strips excerpts from durable recovery in Redis mode. - The no-echo restage reads the SURVIVING chip (reclaimRejectedChipQuotes): a terminal conversion that beat the delayed 202 already moved the quotes onto the queued follow-up, and re-staging them again double-delivered. Regression-tested with the conversion-before-ACK ordering. - RecoveredSteerPayload binds normalized, order-significant quotes (builder, validator, TS matcher, and the Lua decode+matcher): a stale client presenting the same recoverySteerId with altered or missing quotes cannot consume the parked source. Quote-less sources keep matching quote-less recoveries. * 🔁 fix: Execution-Bound Quote Capability with an Atomic Enqueue Predicate Codex round 8: - steerQuotesCapable becomes a transient assertion translated (at createJob and in ApprovalLifecycle.resolve) into steerQuotesExecutionId, valid only while it equals the LIVE providerExecutionId. A legacy replica winning a HITL resume rewrites the execution id without knowing the marker, so its stale assertion self-invalidates — a bare boolean could not be cleared by code that predates it. - The fenced enqueue evaluates that equality atomically (all three Redis scripts decode-and-strip like the existing preemptCapable normalization; both InMemory sites mirror it) and returns the persisted item, so the quotesAccepted echo reflects exactly what was stored even when a handover lands between admission's read and the commit. The last-moment re-read is gone — the transaction is the authority. - Tests: capable-resume re-binding, legacy-resume omit-not-clear invalidation, the admission-vs-handover race (capability read true, then execution rewritten before enqueue), and the Redis round-trip of the marker. * 🔁 fix: Full Redis Parking Coverage and Loss-Moment Quote Restaging Codex round 9: - The two remaining Redis parking projections (terminal status CAS and stale-running cleanup) forward item.quotes — every field-picked steer projection now carries them (audited: 2 Lua 'projected' + 2 Lua 'clientItem' + toPendingSteer). - The ordinary no-echo ACK no longer re-stages: the steer has not injected yet, so the quotes stay carried on the pending chip. A quote-less applied event re-stages them at the actual loss; a terminal leftover conversion carries them onto the recovered row, whose normal send delivers quotes on any server — re-staging at the ACK let that leftover auto-send bare text while the excerpts glued onto an unrelated draft. Only the settled receipt replay (already injected, no future event) reclaims immediately. * 🔁 fix: Legacy-Replayable Receipts with Separate Quote Identity Codex round 10: an upgraded-first receipt stored a quote-inclusive fingerprint no pre-quotes replica could recompute, so a lost-ACK retry routed through one 409'd already-accepted words with duplicate-send controls. The durable fingerprint reverts to the quote-independent 3-field hash — the one shape EVERY deployed version computes, replayable across a rolling deploy in both directions — and quote identity moves beside it as requestedQuotesFingerprint (of the REQUESTED quotes, pre any capability strip, so an incapable-owner acceptance still replays its own retries). Absent records (legacy-written or quote-less) accept any same-words retry, preserving the round-5 rule; present records must match exactly, keeping different-quotes clientSteerId reuse a 409 on quote-aware readers. Under the keep-on-chip client contract a legacy replay's missing echo is harmless — the excerpts stay carried on the pending chip. * 🧪 chore: Re-Trigger CI After Dropped Workflow Events |
||
|
|
6a7da61234
|
🥸 chore: Resolve Agents SDK Path Aliases That Masked Backend Types (#15160)
* 🐛 fix: Restore Agents SDK Type Resolution in Backend Type Checks * 🐛 fix: Preserve Typed Prompt Callback Assignability * 🐛 fix: Accept Agents Function Tool Calls in isImageVisionTool * 🐛 fix: Prove the Run Step Wire Contract at Compile Time |
||
|
|
dd146ff74d
|
🧾 fix: Report Complete Agents API Usage (#15127)
* fix: report complete agents api usage * fix: preserve invoked usage context * test: cover absent usage context * fix: type responses usage finalization * fix: preserve reasoning usage aliases * fix: declare reasoning usage alias |
||
|
|
d3e70159ca
|
📡 feat: Stream Detached Subagent Activity (#15111)
* feat: stream detached subagent activity * fix: annotate activity stream limits * fix: isolate subagent activity imports * fix: harden detached subagent activity lifecycle * test: cover synchronous activity transport failure * test: include required subagent activity identity * fix: identify and reconnect subagent activity events * fix: bound subagent activity lifecycles * fix: close subagent activity handoff races * fix: bind and synchronize activity subscriptions * fix: detect fresh activity attachment * fix: complete activity synchronization handoff * fix: bind activity sync and failure circuits * fix: expose subscription-bound synchronization * fix: fence activity reconnect publications * test: make detached timeout settlement deterministic * fix: fence Redis activity attachments * fix: close failed activity streams * perf: reuse fenced activity frontier * style: sort subagent thread imports * fix: preserve queued subagent activity * test: type activity publication counter * fix: disconnect subagent activity subscriber * fix: close background activity lifecycle gaps * fix: preserve streamed activity spacing * fix: preserve bounded live subagent activity * fix: merge durable subagent activity safely * fix: model detached activity coverage * fix: type detached activity inputs * fix: order overlapping subagent activity * chore: sort activity test imports * fix: buffer subagent activity handoff gaps * fix: flush activity after parent close * fix: advance closed activity suffixes * fix: preserve detached activity ordering * fix: close detached activity delivery races * fix: bound shared Redis subscriber readiness * fix: expire shared Redis subscription readiness * fix: clean up late Redis subscriptions * fix: preserve late Redis subscription fallback |
||
|
|
67b7b441b2
|
🛂 feat: Filter Model-Bound Content by Source (#14425)
* feat: introduce optional content protection seam * feat: enforce source-aware content filters * feat: complete source-aware content enforcement * test: activate skill file-text fail-close fixtures * fix: harden source-aware content filters * fix: harden model-bound content filtering * fix: preserve legacy filters and generated files * fix: inspect shared scalar metadata * test: align mocks with current dev dependencies * feat: add persisted content filter safeguards * feat: complete source-aware content filter enforcement * fix: move resume content preflight into TypeScript * fix: close content inspection edge cases * fix: harden content protection boundaries * fix: complete content protection safeguards * test: align persisted memory filter coverage * fix: reconcile content protection with current dev * fix: reconcile content protection with latest dev * fix: close content protection review gaps * fix: enforce source-aware provider boundaries * fix: preserve legacy PII preflight semantics * test: stabilize stored branch preflight fixture * fix: defer agent writes until protected model admission * perf: harden source-aware model-bound filtering * fix: canonicalize provider lineage before validation * fix: satisfy model-bound callback type checks * perf: Bound content protection filtering work * fix: Bound submission array traversal * fix: Stabilize bounded content snapshots * fix: Scope model-bound traversal overflows * fix: Preserve scoped content inspection * fix: Accumulate aggregate traversal scopes * fix: centralize content policy boundaries * test: align deferred tool policy context * test: align controller policy mocks * style: normalize content protection imports * fix: close content policy review gaps * fix: narrow active skill policy config * fix: address content protection review boundaries * fix: retain exact provenance overflow sentinel * fix: preserve literal and scoped provenance updates * fix: narrow persisted edit provenance * fix: isolate exact overflow attribution * fix: centralize stored prompt protection * fix: fail closed on incomplete transcript evidence * fix: align canonical transcript routing * refactor: centralize content policy preflights * fix: isolate upload policy error typing * style: sort policy preflight imports * refactor: centralize content policy boundaries |
||
|
|
8ae94afa91
|
🪡 fix: Thread Parent Message ID Through MCP Request-Scoped Bodies (#15095)
* fix: Unify MCP request-scoped headers * fix: address request-scoped MCP review findings * test: preserve request scope on status errors * fix: treat authorized on-demand MCP servers as ready * refactor: separate MCP readiness from connection state * fix: preserve on-demand MCP readiness labels * test: satisfy OpenAI conversation ownership guard * fix: keep MCP action predicates boolean * fix: close deferred MCP request context gaps * fix: preserve on-demand MCP configuration actions * fix: fail closed on unavailable MCP parent context * test: complete MCP connecting-state mocks * fix: preserve missing MCP parent on continuations * fix: align native MCP request identities * fix: preserve edited MCP parent identity * test: use scoped Agent initializer fixture * test: expose MCP request body helper * fix: preserve MCP turn identity across resume * style: sort stream metadata imports * fix: carry normalized MCP identity to execution |
||
|
|
9f8d71a3c5
|
🪢 fix: Preserve Response Identity and Branch During Resumable SSE Sync (#14788)
* fix(client): preserve resumable response identity Fixes #14787 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(client): align resumable sync regressions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(client): clarify resumable response ownership Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(client): preserve resumed regeneration ordering Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(client): cover missing resumed response row Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(client): preserve resume identity on page reload * fix(client): replace reassigned resume placeholder * fix(client): preserve content during response id handoff * fix(client): limit resume placeholder handoff * fix(client): preserve resume display metadata * fix(client): reconcile resume metadata in one pass * fix(client): reconcile preliminary resume user * fix(client): restore regenerated branch on early abort * test(client): cover external regeneration resume * fix(client): preserve regeneration history on errors * fix(client): replace reused regeneration error ids * fix(client): preserve exact-id regeneration rollback --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
|
c7e355b219
|
🛑 fix: Confirm Scheduled Stops and Separate Terminal Scheduler Failure from Startup (#15051)
* 🛑 fix: Confirm Scheduled Stops and Separate Terminal Scheduler Failure from Startup Fixes #15042, fixes #15043. `resume.js` inferred a confirmed stop from the ABSENCE of `failureReason`, but `abortJob` had four `success: false` paths that returned no reason at all. Those settled the occurrence as `interrupted` and pruned the checkpoint on aborts that never landed — including one where a REPLACEMENT generation owned the conversation, which pruned the successor's checkpoint. Every `success: false` return now names itself (`job_not_found`, `already_settled` added alongside the existing `generation_replaced` / `job_still_active`), and a single canonical `isStopConfirmed` predicate decides whether durable state may be settled. `already_settled` confirms a stop — `awaitProviderDrain` has proven the provider segment can no longer persist — so a permanently terminal generation is not answered with a retry loop. Separately, a schedule engine that failed to arm advertised its permanent outage as a transient 503 with `Retry-After`, so a client obeying it would poll forever. Readiness is now tri-state (`starting` / `armed` / `unavailable`): the retry contract applies only while arming is genuinely pending, and a failed arm returns a terminal `SCHEDULES_UNAVAILABLE` with no `Retry-After` and an error-level log. * 🏷️ fix: Declare Schedule Write Gate Return Types `--isolatedDeclarations` requires an explicit return type on the exported factory and on the middleware it returns (TS9007). Adds a named `ScheduleWriteGate` type matching the existing `ShareMiddleware` shape. |
||
|
|
c5276fc63d
|
⏱️ feat: Run Scheduled Chats Through Durable Agent Triggers (#14939)
* feat: Scheduled Chats — agent-centric scheduled runs creating real conversations
Squash of the full review-hardened branch (PR 14540, supersedes 14373) onto
latest dev, preserving the exact verified tree. History prior to this commit
lived on the pre-squash branch; every invariant below survived 25 Codex review
rounds plus two external audit rounds (R26) with regression tests that fail
without their fixes.
Feature:
- Schedules CRUD + side-panel UI (cadence dialog, run cards, Run Now), roles/
permissions (SCHEDULES:USE), interface.schedules availability, per-user limits
and capacity slots, timezone-aware cadence with DST-conservative floors and
misfire grace.
- Engine: single-process claim/fire loop with leases, loopback POST dispatch
(signed schedule-fire JWT claims, per-occurrence idempotency key inside the
route's clientRequestId charset), overlap/balance/capacity/duplicate skip
policies with auto-disable streaks (too_many_failures, insufficient_balance),
reconciliation from retained terminal-job evidence, erasure sweep.
- Scheduled runs create real conversations through the resumable agents chat
path: HITL pauses surface on the card (requires_action), resumes re-apply the
fire boundary's admission policy (revision fence, enabled, global kill switch,
SCHEDULES:USE, availability) before continuing a billed generation.
Correctness invariants (the audit surface):
- Settlement discipline: every persistence-producing write happens-before a
run's terminal outcome write; Stop/complete/pause race through single-winner
terminal CAS claims (dev's TerminalJobClaim substrate) with retained,
completedAt-less evidence for scheduled fires plus an owner-intended outcome
stamp (scheduleOutcome) the reconciler prefers over re-derived success —
round-tripped through the Redis hash mapper.
- Swallowed generation failures (client error content parts) classify to
error/skipped_balance instead of success on both initial and resumed paths;
stale stamps are refreshed evidence-first when persistence plus the Mongo
outcome write both fail.
- Abort honesty: delivery judged by generation ownership and the CAS's actual
from-status; republication escalates to the transport's acknowledged variant;
the Stop route settles only genuinely paused runs.
- Account deletion: one-way barrier (deletionRequestedAt) with auth-cache
tombstone-before-stamp, boundary rechecks across all auth strategies, quiesce
of scheduled + interactive work with durable per-stream abort fences
(positive-evidence acknowledgement only), owner-side finalization markers for
post-terminal billed writes, deferred-deletion sweep (explicitly ensured
partial index) that completes cascades autonomously. Remote OpenAI-compatible/
Responses requests are documented as outside the quiesce and tracked in issue
14594.
- Store compatibility: finalization markers optional on the legacy IJobStore
contract with coherent degradation (registration fails -> synchronous title
fallback; count reads 0).
* fix: fence terminal response persistence from deletion; retain stale-pause evidence
Two blockers from the third external review round.
Terminal persistence visible to account deletion:
- The finalization marker was registered only when post-terminal TITLE work was
possible, but every persistence-owning terminal CAS opens the same window: the
claim drops the job out of the active set BEFORE the response save (and the
background user-message/convo saves), so a deletion quiesce landing there saw
neither an active job nor a marker and could cascade while the admitted request
could still recreate messages. Both controllers now register the marker before
every persistence-owning claim — the fresh-turn path and the HITL resume — and
release it only after their pending saves (and any post-terminal title) have
landed, on success and failure paths alike. The TTL bounds a crash.
- settleAbortFence no longer clears a complete/error fence while
`terminalPersistencePending` is set: terminal at the CAS is not settled while
the owner is still persisting. The job facade now surfaces the flag.
- The marker trio is REQUIRED by the runtime store contract (assertJobStoreV2
refuses a store without it at configure time, keeping the failure loud and
deterministic) while remaining optional on the legacy public IJobStore type for
source compatibility. The silent degrade path from the previous round is gone —
it was not deletion-safe.
Stale-pause recovery retains scheduled evidence:
- Three crash/timeout recovery paths — ApprovalLifecycle.failStalePausePersistence
and the InMemory/Redis stale-pause cleanups — unconditionally stamped
`completedAt`, putting a scheduled fire's failed-pause error terminal on the
short completed TTL. A Mongo outage longer than that TTL erased the evidence and
the reconciler recovered the run as `interrupted` instead of `error`. All three
now follow the controller-observed path from the previous round: scheduled jobs
omit `completedAt` (retained-evidence TTL) and stamp the error outcome.
The PR description now explicitly narrows the deletion guarantee for the remote
OpenAI-compatible/Responses paths (tracked in issue 14594).
* fix: deletion-fence marker protocol — generation-scoped, atomic, fail-closed, all terminal paths
One consolidated pass over the finalization-marker mechanism, per the fourth
external review round. The invariant it establishes: NO persistence-owning
terminal CAS runs without a durable, generation-qualified marker covering the
window it opens, and every consumer treats a pending terminal as unsettled.
- Generation-scoped markers. Entries were keyed (userId, streamId), so a
Stop-superseded generation finishing late could clear the marker its
replacement registered on the same conversation. Marker fields are now
qualified by the generation's createdAt; clears must present the same
identity, and an unqualified legacy clear cannot drop a qualified entry.
- Atomic Redis registration. HSET-then-EXPIRE loses the fresh marker when the
user's existing hash expires between the two commands (or the process dies
there); registration is now a single Lua script carrying both.
- Fail closed everywhere. Registration failure (after one retry) now REFUSES
the terminal CAS instead of proceeding uncovered: the completion claim throws
into the error path, the error path skips completeJob and leaves the job
ACTIVE — deletion-visible by itself, recovered by the stale-running reaper —
and abortJob returns a new retryable `fence_unavailable` failure the Stop
route answers with 503 and the deletion quiesce treats as an unacknowledged
stop (fence kept). The previous round's log-and-proceed is gone.
- Every terminal path enrolled. abortJob now owns its window (register before
the abort CAS, clear in its finally — the Stop route's checkpoint prune and
partial save run inside beforePublish, between CAS and publication); the
interactive and background generation-error paths register before their
completeJob; the resume controller's error finalization registers before its
completeJob. A lost or thrown claim releases the marker after pending saves
flush instead of holding the user's deletion behind the TTL.
- Every pending terminal unsettled. settleAbortFence defers on
terminalPersistencePending for ALL statuses — including `aborted`, whose
route-side persistence the previous guard missed.
Also: a direct Redis regression for scheduled stale-pause retention (the P2
test gap), and the PR description no longer claims to carry every commit.
* fix: lease-token lifecycle fences — same-generation isolation, admission fence, undelivered-Stop retention, legacy abort enrollment
Fifth external review round; four P1s, handled as the requested consolidated
lifecycle-fence pass.
- Lease tokens. Marker fields were (streamId, createdAt), shared by every
contender on the same generation — completion and Stop, or two racing Stops —
so a losing contender's cleanup erased the winner's still-live marker. Every
registrant now carries a unique lease token in the field and may only ever
clear its own lease; unqualified legacy clears cannot touch qualified entries.
- Admission fence. Authentication can pass before the deletion barrier goes up,
and the durable createJob is several async steps later — a deletion quiesce in
that window saw neither an active job nor a marker and could cascade before
the admitted request created its job. The controller now registers an
admission lease and THEN rereads the deletion barrier: the ordering guarantees
either this request observes the barrier (403, lease released, slot/claim
cleanup) or the quiesce observes the lease and defers. Held until createJob is
durable; released on every refusal and initialization-error path. Fail closed
when the lease itself cannot be registered (503 retryable).
- Undelivered-Stop retention. abortJob released its lease in a finally even when
delivery AND publication had provably failed — the job reads terminal
(invisible to active-set scans), a user Stop writes no durable abort fence,
and the remote owner keeps generating and will persist its abort-catch writes
whenever the signal finally lands. The lease is now retained in exactly that
case, and each resignal attempt heartbeats a fresh lease so the fence outlives
the TTL for as long as delivery is still being driven. The abort-winning
turn's own loser-side pending saves are additionally fenced in the controller
catch (best-effort — those writes are already in flight).
- Legacy abort enrollment. abortMiddleware (assistants abort route fallback for
non-assistants endpoints) awaited abortJob and then spent usage and saved the
stopped response AFTER the abort's lease was released. Both writes now run
inside `beforePublish`, between the abort CAS and publication, covered by the
same lease as every other abort.
Barrier tests, each verified to fail without its fix: same-generation lease
isolation (store), racing two-Stop loser cleanup (manager, stale-read forced
CAS race), undelivered-Stop lease retention, resignal heartbeat, admission
refusal with lease-before-reread ordering plus release-on-durable-create, and
legacy-abort persistence inside beforePublish.
* fix: heartbeat-backed owner-lifecycle leases close the settlement handoff races
Sixth external review round: the remaining P1 interleavings were one structural
problem — lease handoffs that were not atomic — resolved as the requested
consolidated lifecycle-lease pass.
- Quiesce reads leases BEFORE the active-job scan. The admission-lease -> durable
-job handoff is only atomic against a reader in the OPPOSITE order of the
writer: writers hold the lease strictly until the job is active-set visible,
so leases-first shows every interleaving either the lease or the job.
Jobs-first allowed a request to create its job after the scan and release its
lease before the count — hiding both, cascading, and letting the new
generation persist into a deleted account.
- The abort acknowledgement is fenced by the owner-lifecycle lease. Redis ACKed
the moment the owner's AbortController tripped; the stopping side released its
lease on that ACK while the owner's asynchronous abort-catch persistence was
still ahead. The transport now awaits a manager-installed pre-ACK hook that
registers a DETERMINISTIC owner lease (exactly one owner exists per
generation, and determinism is what lets the signal-time registrant and the
owner's catch-side release agree across processes) before the acknowledgement
is persisted or published; an owned same-replica abort bridges to the same
lease before tripping its local controller. Both generation-owner catches
(fresh turn, resume) release it once their writes land.
- A failed replacement handoff no longer orphans the predecessor. The atomic
replacement removes it from active storage, and an unconfirmed handoff
terminalizes the replacement too — leaving nothing a quiesce could discover
while the predecessor's provider may still be generating. Its owner lease is
now retained at the point the receipt fails delivery; the owner replica renews
it through the pre-ACK fence when the signal finally lands.
- Leases HEARTBEAT while held. The five-minute store TTL only bounds a crashed
holder; live persistence — a stalled save, a long deferred title — must never
outlive its own fence. holdUserFinalization registers and renews every minute
until released; the controllers' completion/error/admission leases all hold.
The undelivered-Stop retention moved to a deterministic `stop` lease that
every resignal attempt renews and the first successful one clears (no more
opaque leases accumulating to TTL), and a THROWN abort transition releases the
contender lease instead of leaking it.
- The user-document abort-fence mutations now invalidate the auth user-doc
cache, matching every other user-doc write.
Barrier tests, each verified to fail without its fix: quiesce lease-scan
ordering (plus the observed-lease defer), pre-ACK fence ordering at the
transport, owned-abort owner-lease bridging, replacement-handoff predecessor
retention, held-lease heartbeat past the TTL, and failed-then-successful
resignal reaping the retained stop lease.
* fix: one manager-owned owner-lease span across every abort delivery path
Seventh external review round; four lifecycle-fence gaps, closed by making the
owner-lifecycle lease a single manager-owned, heartbeat-held span.
- Fail-closed acknowledgements. The pre-ACK hook registered a one-shot lease and
the transport ACKed even when it failed; a same-replica owned abort likewise
swallowed registration failure. The hook now acquires a HELD owner lease
(heartbeat until the owner's catch releases it via releaseOwnerLease) and a
rejection SUPPRESSES the acknowledgement — the stopping side stays retryable
behind its retention lease, and every resignal re-drives the handler. The hook
also stopped gating on `job.createdAt === generationId`: during a replacement
handoff the store holds the replacement while the abort targets the
predecessor, and that gate silently skipped exactly the generation being
acknowledged (the store job is owner identity, never a generation gate). A
local owned abort acquires the same held lease before tripping its provider;
post-CAS the trip cannot be withheld, so acquisition failure downgrades
delivery and the retention handoff keeps the user fenced.
- Committed-but-lost-reply disambiguation. A thrown abort transition released
the contender lease as if nothing had happened, but a Lua CAS can commit and
lose its reply — an aborted job invisible to active-set scans whose provider
was never signalled, with no fence left. The throw path now re-reads the exact
generation: only a job still live under the caller's identity proves no
commit; committed or ambiguous outcomes hand the fence to the deterministic
stop lease (kept on the contender lease if even that fails) before rethrowing.
- Replacement handoff covered end to end. A LOCAL replacement abort acquires the
predecessor's held owner lease before the trip (failure reports the receipt
undelivered, engaging retention). Failed-handoff retention is no longer a
swallowed one-shot: it heartbeats with the durable acknowledgement proof as
its renewal predicate — acquisition failures keep retrying for as long as the
fence is needed, and the retainer stands down (without clearing the shared
field) once the remote owner ACKs and thereby holds its own lease.
- A LOCAL resignal delivery hands off to the owner lease BEFORE clearing the
retained stop lease, and keeps the stop lease when that handoff fails.
Barrier tests: hook rejection suppressing the ACK, commit-then-lost-reply
retention with its provably-uncommitted counterpart, local-resignal owner
handoff ordering, pre-ACK owner lease held past the store TTL until release
(and provably stopped after), and failed-handoff retention retrying on its
heartbeat — verified fail-before/pass-after by stashing the fixes.
* fix: finish scheduled chat lifecycle hardening
* fix: close scheduled chat review follow-ups
* fix: generation-fence abort recovery evidence
* test: wait for settled approval tool output
* test: preserve scheduled init reconciliation option
* refactor: rebuild scheduled chats on durable agent triggers
* test: reset MCP cache mock between cases
* test: isolate scheduler startup in server specs
* fix: harden scheduled run lifecycle
* fix: normalize schedule capacity conflicts
* test: type schedule collision fixture
* fix: re-fence scheduled resume and expiry
* fix: fence scheduled resume handoffs
* fix: release superseded manual schedule leases
* fix: release failed run-now claims
* fix: release superseded engine claims
* fix: repair schedule dialog interaction and rework its form
The agent picker was unusable: ControlCombobox portals its popover to the
body by default, which lands it outside the dialog's Radix focus trap. Clicks
passed through it, it could not be tabbed into, and the trap fighting Ariakit
for focus locked the page up on selection. The prop is documented for exactly
this case — pass `portal={false}` and give the dialog `overflow-visible`, as
ProjectButton already does. The time and day dropdowns defaulted the same way.
Alongside that:
- Extract the agent builder's instructions editor (special-variable menu plus
expand-to-fullscreen) into a controlled `VariableEditor` and use it for the
schedule prompt. Insertions now route through `onChange`, so react-hook-form
sees them — the schedule PATCH is built from `dirtyFields`, and a `setValue`
that skipped dirty tracking would have dropped an inserted variable silently.
- Replace the hand-rolled frequency buttons with the shared `Radio`. They marked
the selection with `bg-surface-hover` on an outline button whose hover is the
same token, so the selected option was indistinguishable from a hovered one;
`Radio` is also a real radiogroup rather than four `aria-pressed` toggles.
- Wrap the fields in a real `<form>` and associate the footer button by id, so
Enter submits. Group the frequency, day and time controls in fieldsets.
- Add placeholders for name and prompt, match the textarea fill to the other
fields, and label the hourly case as minutes past the hour.
- Widen the dialog to `md:max-w-3xl` and pair name with agent so the form fits
without scrolling on desktop.
- Move scheduled chats below skills and above prompts in the side nav.
The new dialog spec fails when the portal fix is reverted.
* test: cover scheduled and subagent deletion drains
* refactor: own the form-control appearance in the client primitives
Addresses the codex finding on ScheduleDialog: a feature-local `FIELD_CLASS`
restated the `Input` primitive's border, radius, height and background so it
could be pasted onto the schedule dropdowns, leaving those controls with no
connection to the primitive they were imitating.
Move that appearance into `packages/client/src/components/Field.ts` as the
single source `Input` and `Textarea` now compose, and give `Dropdown` and
`ControlCombobox` a `variant="field"` that applies it. The schedule dialog
passes the variant and carries no class strings of its own.
This also repairs a break the dev merge would otherwise have introduced: the
newer `Dropdown` splits `className` (wrapper) from `triggerClassName`, so the
old pasted classes would have landed on the wrapper and left the triggers
unstyled.
The semantic-token guard now watches the shared module and asserts each
primitive still composes it, which covers more than the two files it read
before.
* fix: keep an explicit schedules disable from becoming an opt-in
`use` is two things at once for a dual-purpose runtime interface field: a
permission bit, which DB overrides strip, and the runtime disable signal that
`getLimits` reads. Stripping it from `{ use: false, maxPerUser: 2 }` leaves an
object, and `getLimits` treats any object without `use: false` as enabled — so
an admin override written to stop scheduled billing for a role or user started
it instead.
Collapse an explicit disable to the boolean form before the strip, on both
paths that accept it: the `interface.schedules` field patch, which admitted the
object wholesale because bare runtime paths deliberately bypass the permission
gate, and the overrides merge, which reached the composite-field branch and
kept `maxPerUser`. Objects that only narrow limits are untouched, so a
principal can still be given a smaller cap.
Both regressions fail without the normalizer.
* fix(schedules): Wave A — null-balance CAS, atomic paused-card clear, clustered erasure sweep
Slice 1 (thread r3804518381): route the existing-null balance initialization
through a { user, tokenCredits: null } compare-and-set instead of a blind $set,
so a concurrent initializer/charge landing between the preflight read and the
write is never handed back its spent starting balance. On a CAS miss the
preflight re-reads the winner. The absent-record $setOnInsert path and the
credited-record refill-config sync are unchanged. Adds the initializeNullBalance
adapter (no upsert) and regression coverage for winner/miss/sync cases.
Slice 2 (thread r3804518388): updateScheduleById now drops a `requires_action`
lastRun projection atomically with the configRevision bump. Any pause present at
edit time was projected under the pre-edit revision and can never be replaced by
its own revision-fenced terminal outcome — a disabling edit would strand the
card on "Needs approval" forever. Implemented as classic-operator CAS branches
(DocumentDB rules out a conditional pipeline $unset), fenced on the card STILL
being the pause so a terminal outcome or newer occurrence that races in is
preserved. Terminal history survives untouched.
Slice 4 (thread r3803826204): expose initializeScheduleErasureSweep from the
schedule runtime facade and start it in every clustered (experimental) worker
after Mongo is up. It re-drives eraseScheduleIfDrained for soft-deleted rows so
a hidden prompt cannot outlive its drain when the delete/erase-on-settle
attempts miss. It arms nothing else and never infers owner death from a
process-local missing job (isTopologySafeToArm gates that).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): Wave B.1 — reversible account-deletion schedule suspension
Thread r3804518383. Account-deletion quiesce marked every schedule `deleting`,
disabled it, and cleared nextRunAt destructively. When a later cascade step (or
the drain itself) failed, the controller cancelled the user-deletion fence —
restoring the user — but the schedules stayed `deleting` and were erased by the
sweep, silently losing all of a live user's scheduled prompts.
Replace the destructive marking with a REVERSIBLE, token-fenced suspension:
- suspendUserSchedulesForDeletion(userId, token) snapshots each schedule's prior
enabled/nextRunAt under a per-attempt token, then fences firing (disable, clear
nextRunAt, rotate claimToken). It never sets `deleting`, so a suspended row is
not erasure-eligible. Snapshotting reads then bulkWrites (a classic update
cannot copy field values under DocumentDB), fenced per row so an already-
suspended/soft-deleted/edited row is left alone; idempotent per token.
- restoreUserSchedulesFromDeletion(userId, token) reverses it, re-enabling and
re-arming only rows still carrying the exact attempt token and not independently
deleted — so an owner-deleted or newer-attempt-suspended schedule is never
resurrected.
- deleteUserController generates the attempt token, passes it to quiesce, and on
any failure that cancels the user-deletion fence restores the suspended rows. A
successful deletion hard-deletes them (and their snapshots) via the existing
cascade and never restores.
Adds a `deletionSuspension` embedded field (excluded from the wire schedule),
data-method regression tests (suspend/restore/fence/idempotency/no-resurrect),
and controller tests (restore on drain-false and post-quiesce cascade failure,
no restore on success).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): Wave B.2 — wire the interactive Stop persistence protocol
Thread r3804255932. The durable Stop primitives (requestRunAbort 'stop',
getScheduleRunAbortState, markRunAbortPersisted) already existed but production
only used requestRunAbort(..., 'deletion'). An interactive Stop flipped the job
to `aborted` and then persisted its partial message + checkpoint inside
`beforePublish`, without ever stamping the schedule Stop or acknowledging it —
so reconciliation, the generation owner, or a concurrent schedule/account
deletion could terminalize the run and release its capacity (and erase data)
mid-write.
Wire the request -> persist -> acknowledge -> settle barrier through the
schedule runtime (the route never touches raw Mongo):
- Expose beginScheduledStop / acknowledgeScheduledStopPersistence on the service.
- The abort route stamps the Stop BEFORE signalling abortJob (a serialized
'in_progress' loser returns 409 STOP_IN_PROGRESS without a second abort),
acknowledges only after beforePublish persistence succeeds, and on a
persistence failure leaves the barrier unresolved so the run stays preserved
(client retries; stale-owner timeout is the bounded recovery). A failed abort
releases the stamp it placed so a replacement/retry is never blocked through
its predecessor.
- recordScheduleOutcome (the owner settlement path) now waits, bounded, for the
Stop acknowledgement before terminalizing; a resolved/non-stop/stale marker
proceeds immediately. The paused Stop settles only after its own ack.
Adds service-layer barrier tests and route-level ordering/persistence-failure/
in-progress tests; the data-layer serialization is already covered.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): Wave C.1 — reconcile durable trigger delivery with the reservation
Threads r3803826192 (manual limiter) and r3804255924 (PII/moderation), plus the
independently-found long-Retry-After race. fireSchedule reserves a `started` run
and a global capacity slot BEFORE the durable trigger delivery reaches the chat
route, where an interactive limiter (manual Run Now), PII, or moderation can
reject it before any generation job exists — dead-lettering the delivery while
the run sat `started` until the 30-minute orphan sweep mislabeled it interrupted.
And a valid delivery deferred by Retry-After (up to 24h) could be orphan-settled
and have its capacity released, then fire anyway.
Translate durable delivery state into the schedule outcome:
- Store the deterministic trigger deliveryKey on the ScheduleRun reservation
(computed from the envelope BEFORE enqueue, so an ambiguous commit still has it).
- Add a getTriggerDelivery engine dep (wired to the merged trigger service's
getDelivery) that reads the durable delivery by key.
- Schedule reconciliation, for a jobless `started` run: staging/pending/leased →
admission is live, never orphan; dead → record `error` from the durable
lastError and release capacity promptly (no 30-minute wait), through the
ordinary outcome/auto-disable path; succeeded or no record → the existing
legacy orphan policy (interrupted only past the cutoff); a delivery lookup
failure defers rather than orphaning a possibly-live delivery. Limiter/PII/
moderation middleware writes no schedule state.
Adds reconcile state-mapping tests (dead/pending/leased/staging/succeeded/none/
lookup-failure) and a fire test that the reservation's deliveryKey equals the
enqueued delivery's idempotency key.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(stream): Wave C.2 — durable retry/ack for terminal host lifecycle actions
Thread r3804518375. Approval expiry won the `requires_action → aborted` CAS and
then invoked the host hook best-effort: `runApprovalExpiredHandler` swallowed a
failure, and because later sweeps enumerate only `requires_action` jobs, the now-
aborted job was never offered again. In the clustered entrypoint (no schedule
reconciler) the ScheduleRun stayed `requires_action` and its retained job
persisted indefinitely.
Make the host lifecycle work durable rather than schedule-specific:
- Add a generic `terminalHostActionPending` marker, set ATOMICALLY in the same
terminal transition (ApprovalLifecycle.expireWithIdentity), only when a host
adapter is installed.
- Retain and index such jobs: both stores keep them out of terminal reaping and
expose getTerminalHostActionJobs(); Redis adds a set + extended (24h-bounded)
TTL, in-memory a bounded 24h retention so a permanently-failing hook cannot leak.
- The manager clears the marker only after the adapter acknowledges success,
fenced by generation identity (clearTerminalHostAction), so a replacement
generation can neither clear nor execute its predecessor's action.
- cleanup()/expireStaleApprovals() enumerates unacknowledged terminal host actions
across restarts and replicas and retries the idempotent hook; the relay only
re-invokes while the marker is unacknowledged, so a successful ack prevents
duplicate work. Store-won expiry marks it too, so a loser-replica relay still
crosses the hook.
- Terminal SSE notification continues regardless of host-hook outcome.
Covers in-memory behavior (retry after failure, restart/other-replica retry, ack
prevents duplicates, identity fence, terminal notification on failure, no marker
accumulation for non-scheduled jobs) and updates the Redis cluster-membership
contract test for the new index.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* style(schedules): satisfy import sorting in fire.ts and fire.spec.ts
CI "Static checks" failed on IMPORT_SORT for the two files Wave C.1 added imports
to (the AgentTriggerEnvelope type import and getAgentTriggerIdempotencyKey).
Applied scripts/sort-imports.mts to exactly those files — imports-only reordering,
no behavior change. Other files reported by a repo-wide check are pre-existing on
dev and deliberately left untouched so this PR is not widened.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): repair the deletion CLI and order restore before the fence release
Addresses three findings from the fresh Codex review.
P1 — config/delete-user.js called methods.disableUserSchedulesForDeletion, which
Wave B removed in favor of suspendUserSchedulesForDeletion. The file is
@ts-nocheck and its spec mocked the removed name, so neither typecheck nor tests
caught it; the real CLI would throw a TypeError before deleting anything and then
only unwind the fence. The CLI now uses the tokenized protocol: it mints a
suspension token, suspends with it, and restores that exact attempt's rows in its
finally block when the deletion does not commit. Its spec mocks the real methods,
so the breakage can no longer hide.
P2 — both the HTTP controller and the CLI released the user-deletion fence BEFORE
restoring schedules. That fence is what refuses new schedule writes/claims, so the
gap let an owner PATCH edit a still-suspended row and have its enabled/next-run
state overwritten by the older snapshot, and let a second deletion attempt
re-suspend under a new token — making the first restore a no-op and stranding the
disabled snapshot permanently. Restore now runs first, while writes are still
fenced.
Hardening for the same defect class across a crash: suspendUserSchedulesForDeletion
now ADOPTS an existing suspension's snapshot when re-suspending a row abandoned by
an earlier attempt, instead of re-capturing the row's current (already-suspended)
state. Without this, an attempt that died before restoring would have its
successor snapshot "disabled, no next run" and permanently strand the schedule.
Tests: CLI restore-before-fence ordering and no-restore-on-success; the same
ordering assertion on both controller post-quiesce failure paths; a data-method
regression that a second attempt adopts the abandoned snapshot and restores the
original enabled/next-run state.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): converge dead deliveries in topology-safe maintenance
Codex finding: the `dead` delivery mapping added in Wave C.1 lives only inside
startScheduleEngine's reconciler, but the clustered entrypoint arms no engine — it
runs erasure-only maintenance. A delivery queued before a restart into clustered
mode and then rejected before generation creation (interactive limiter, PII,
moderation) dead-letters while its ScheduleRun stays `started`, holding a global
capacity slot indefinitely for an ordinary non-deleting schedule.
Add a dead-delivery convergence pass to the erasure sweep, so every topology that
runs schedule maintenance settles it. The pass is POSITIVE-EVIDENCE-ONLY and is
therefore safe where absence-based reconciliation is not: a `dead` delivery is
durable shared state proving no generation owns the reservation. It settles only
when the job is confirmed absent or identity-mismatched (an identity-matched job
still owns the run), defers on an unknown job lookup, on an in-flight abort, and
on an in-flight resume hand-off, ignores legacy reservations with no deliveryKey,
and applies a short grace so an accepted delivery still creating its generation is
never settled mid-handoff. Auto-disable policy is deliberately left to the armed
engine; this path records the failure and frees the slot.
Deliberately does NOT touch api/server/experimental.js — the clustered entrypoint
already starts this sweep, so the convergence arrives through the existing
initializer and the shared-file footprint stays as-is.
Tests: settles a dead delivery as error under an explicitly UNSAFE topology,
leaves live deliveries alone, never settles under an identity-matched running
generation (delivery is not even consulted), defers an in-flight abort, and
ignores a reservation with no deliveryKey.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(stream): refresh host-action retention on each retry attempt
Codex finding: unacknowledged terminal host-action evidence was capped at the
24h pause TTL, so a host dependency (Mongo) unreachable for longer than that let
the Redis key — and its pending marker — expire with no generation-fenced
acknowledgement, stranding the ScheduleRun where no reconciler is armed.
Measure retention from the LAST retry rather than from the terminal transition:
enumerating a pending host action IS the retry attempt, so both stores refresh
its retention as they hand it to the hook (Redis re-EXPIREs the job key; the
in-memory store stamps terminalHostActionRefreshedAt and bounds from it). Evidence
therefore survives as long as some replica is still actively retrying, while a
deployment that stops sweeping entirely still lets it age out — so this does not
reintroduce the unbounded leak the cap existed to prevent.
Test: after a failed hook, a later cleanup pass keeps the marker pending and moves
its retention basis forward.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): defer settlement when the Stop barrier times out
Codex finding: waitForStopPersistence returned after its 5s poll budget even when
the Stop was still fresh and unacknowledged, and recordScheduleOutcome then went
straight on to terminalize the run — releasing its capacity, deletion, and erasure
barriers while beforePublish may still have been writing. Slow checkpoint cleanup
is indistinguishable from a dead route on that signal, so the timeout was being
treated as if the barrier had been satisfied.
The poll budget now means "undecided", not "clear". On timeout with a fresh,
unacknowledged Stop the barrier DEFERS: recordScheduleOutcome returns false
without recording, leaving the run active/preserved. Settlement then happens
either when the route acknowledges, or once the existing stale-owner cutoff
(ABORT_OWNER_PRESUMED_ALIVE_MS) authorizes a later attempt — which the loop
already treats as clear-to-settle. Callers with durable retry (the approval-expiry
host action, reconciliation) re-drive it, so a deferral converges rather than
stranding the run.
Test: a fresh Stop that never acknowledges within the budget reports not-settled
and records no outcome.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): require definite delivery failure, extract its message, converge deferred Stops
Third Codex round. Three findings in code from this closeout, plus one pre-existing
P1 that is a one-line operator-safety fix.
P1 — dead-delivery settlement demanded too little evidence. `dead` does not prove a
request was rejected: the trigger host marks response timeouts and invalid success
responses `certainty: 'ambiguous'`, and the engine dead-letters those once retries
are exhausted. The erasure sweep treated every dead letter as positive evidence, so
an ambiguous one sitting over a generation a peer had accepted could terminalize the
run and release its capacity mid-flight. It now settles only on a DEFINITE rejection,
unless job absence is deployment-authoritative (safe topology), where the
confirmed-absent job is itself the evidence.
P1 — `lastError` is an `AgentTriggerDeliveryFailure` object, not a string. A
duplicated local interface declared it `string` (against CLAUDE.md's no-duplicate-
types rule), so both the sweep and the engine reconciler passed the object into the
String-typed run/schedule `error` fields; Mongoose would reject the cast, the per-row
catch would swallow it, and the run would keep its global capacity slot. The dep type
now reuses the canonical `AgentTriggerDeliveryFailure` and both call sites pass
`.message`. Re-typing immediately surfaced a stale test that had asserted a string.
P1 (pre-existing) — the base-config global stop is honored in `getLimits` via
`isRuntimeDisabled` rather than a literal `=== false`. The stop has two shapes, and
deepMerge turns base `{ use: false }` plus a principal override of `true` into
`{ use: true }`, so the literal check reported the feature enabled and Run Now
dispatched straight through fireSchedule, bypassing the operator's emergency stop.
Now the same predicate the engine gate already uses.
P2 — a Stop whose settlement DEFERRED past the poll budget had no convergence path
where no reconciler is armed. `acknowledgeScheduledStopPersistence` now optionally
re-drives the terminal outcome once the barrier clears; `recordRunOutcome` is
match-guarded and idempotent, so an owner that already settled makes it a no-op. The
abort route passes it for a running generation; a paused job still settles explicitly.
Tests: ambiguous dead letters refused under unsafe topology but settled when absence
is authoritative, definite rejections settled either way, the failure message carried
through, and the abort route's re-drive present for running / absent for paused.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq
* fix(schedules): converge terminal runs in clustered workers, retry suspension restore
Fourth Codex round on
|
||
|
|
5e3c680761
|
🪃 feat: Wake Parent Agents on Child Completion (#14975)
* feat: wake parent agents on child completion * wip: harden child completion wakeup lifecycle * fix: close the completion-wakeup static failures Type the durable-claim store fixture, the continue-envelope test helper, and the terminal message's task metadata so the wakeup suites compile against the shapes they actually exercise. Replace `Array.prototype.at`, which the package target library does not provide. Capture the prepared child thread in a non-optional local before the provider callback closes over it, and narrow the trigger envelope itself on `mode === 'continue'` rather than a separately copied mode, so reading the continue target is sound. Lift the parent-message fallback out of a nested ternary into a named resolver. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H1cCMDrTWaRNkmtKjpWELZ * test: cover the active-predecessor admission fence The Redis job-creation call gained a thirteenth scalar argument, so the spec helper reconstructed the HSET pairs one slot early and rebuilt an invalid job hash; three creation tests failed on that alone. Give the fence itself direct coverage in both store adapters, which it had none of despite deciding whether an automatic continuation may replace a live parent turn. Each proves a running and a requires_action predecessor are refused with the state a controller needs for a finite 409, that an absent or settled predecessor is admitted, and that an ordinary user turn without the policy still replaces its predecessor. The Redis case also asserts a refused continuation leaves the parent's durable job and chunks untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H1cCMDrTWaRNkmtKjpWELZ * fix: harden completion wakeup rollout and claims * fix: close completion wakeup race windows * test: keep the child store fixture exact * fix: close final subagent wakeup gaps * fix: preserve ambiguous completion claims * fix: release pre-admission wakeup claims * fix: stabilize subagent completion recovery --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
547bd8c4bf
|
🧵 feat: Persist View-Only Subagent Threads (#14957) | ||
|
|
aa35cd42b1
|
📬 feat: Add Durable Agent Trigger Delivery (#14925)
* feat: wire trusted agent trigger dispatch * feat: add durable agent trigger delivery * fix: annotate trigger envelope byte limit * test: isolate trigger startup in server specs * fix: fence trigger delivery during account deletion * test: isolate trigger service in user controller specs * fix: close trigger deletion admission race * fix: harden account deletion fences * fix: close durable trigger review gaps * fix: require offline stale-fence recovery * fix: type trigger lane sequence ids * fix: fence admin user deletion triggers * fix: make trigger deletion recovery durable * fix: harden offline user deletion * fix: serialize trigger lane publication * style: sort trigger delivery imports * fix: recover orphaned trigger publications * fix: preserve trigger recovery ordering * fix: fence trigger publication during purge * fix: defer remote trigger deletion fences * fix: close durable delivery cleanup races * fix: drain CLI generation owners before deletion |
||
|
|
7d850c308a
|
🧠 feat: Add Live Reasoning Labels (#14893)
* feat: add live reasoning labels * fix: Stabilize reasoning label checks * fix: Address reasoning label review findings * chore: Bump Agents SDK for reasoning labels * fix: Reset reused reasoning step evidence * fix: Reconcile cleared reasoning labels * fix: Fence reasoning label resets * fix: Reset reasoning ownership before gap labels * fix: Preserve THINK type through label reset * test: Expect run-global reasoning revision |
||
|
|
fb8ae881cf
|
⏱️ feat: Show Run-Step Durations On Tool Cards (#14892)
* ⏱️ feat: Show Run-Step Durations On Tool Cards Surfaces how long each tool call took, derived from the `closed_at` / `created_at` pair already carried by `on_run_step_closed` — the same event #14871 and #14873 use for the terminal status. No new event, no new SDK surface. The duration is stamped onto the content part at the same three sites as `runStepStatus`, so it survives a reload and a resumable reconnect rather than living only on the live React message: - `callbacks.js`, on the aggregated part before the event is forwarded - `RedisJobStore`, in the host-authored replay reconstruction branch - `useStepHandler`, on the live message Rendering lands in the shared `ProgressText`, which nine tool cards already use, rather than in each card: one place decides whether a duration is shown and how it reads, and the cards only forward the number. That keeps this from adding a tenth independent state derivation to a component family whose label/announcement/progress split is already the subject of AI-1810. The value is deliberately absent rather than zero whenever it would be a guess — no `created_at`, non-finite input, or a negative elapsed time from two clocks that disagree, which is now reachable because a step can be opened in one process and closed in another after a checkpoint resume. Sub-second durations are suppressed as noise, and it renders only on a settled, non-error card, where the slot is not already carrying the cancelled icon or the error suffix. For assistive technology the compact form (`3.5s`) is hidden and paired with a spoken equivalent ("took 3.5 seconds"), both inside the button, so the accessible name carries the duration without an `aria-live` region re-announcing it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🎨 style: Sort Imports In Touched Files The import-sort gate runs against the files a PR changes, so pre-existing drift in `ProgressText.tsx` and `RedisJobStore.ts` surfaced on this branch. Both were already unsorted on `dev`; this is the sorter's output, with no semantic change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🐛 fix: Accept Partial Timestamps In Run-Step Duration Helper `getReportableRunStepDurationMs` declared its parameter as `Pick<RunStepClosedEvent, 'created_at' | 'closed_at'>`, where `closed_at` is required. That contradicted the function's own purpose: every guard inside it exists precisely to handle stamps that may be missing. The Redis replay branch reconstructs closures from persisted JSON and holds nothing stronger than "might be a number", so it failed to typecheck against the narrower signature. Widened to an exported `RunStepTimestamps` shape with both stamps optional, rather than asserting at the call site — an assertion would move the decision about what is trustworthy somewhere it cannot be enforced, which is the thing the helper exists to centralize. Callers holding a fully-typed event still pass, since a required field satisfies an optional one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🐛 fix: Suppress Duration When Failure Arrives As errorSuffix Alone At every call site `error` carries cancellation while failure travels through `errorSuffix` with `error` false, so gating the duration on `!error` alone rendered "· 3.5s" beside "· failed" — and announced it. The gate now checks both terminal-failure channels. The original test pinned only the `error: true` path, which is why this survived; the failed-via-suffix path is now pinned separately, both the visible and the announced half. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🧩 refactor: Persist Raw Run-Step Durations, Threshold At Render Only The three stamp sites filtered through the 1-second reportability threshold before persisting, baking a presentation rule into stored data: a 900ms step stored nothing, making "fast" indistinguishable from "not derivable" and unrecoverable if the display rule ever changes. Stamp sites now persist the raw `getRunStepDurationMs` value — absent only when genuinely not derivable — and the renderer alone decides what is worth showing, which `ProgressText` already did. Rendering is unchanged. `getReportableRunStepDurationMs` is removed; it existed only to serve the write-time filter, and a test now pins that sub-threshold durations survive to storage. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🐛 fix: Suppress Duration On Backgrounded Bash And Code Cards A backgrounded call's run step closes when dispatch returns the handle, so the stamped duration is the dispatch time. Rendering it beside "Running/Finished in background" misstated a detached task's runtime as seconds — and violated the "settled card only" rule, since the card is still tracking the detached run. Scope is exactly the two cards that parse background handles. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🌍 fix: Format The Sub-10s Decimal For The Active Locale The fractional seconds value was interpolated as a raw JS number, which hardcodes the en-US decimal point into every language — "1.4s" where the locale writes "1,4 s" — and translators cannot fix a number formatted in code. The value is now formatted via Intl.NumberFormat with i18n.language, following MessageTimestamp's pattern of threading the language into the util; plural-key selection stays on the numeric value. A malformed language tag falls back to the plain number. Also documents the two accepted limits of the derivation, so they read as decisions rather than oversights: positive clock skew is undetectable from a single stamp pair, and the value is wall-clock elapsed, so a step held open across a suspension (checkpoint resume, HITL approval wait) includes that time. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🐛 fix: Persist A Durable `backgrounded` Marker Through Harvest; Localize Minute Digits Codex round 3, both findings confirmed. **Background origin survived only as transient state.** The dispatch handle in `tool_call.output` and the live status-marker attachment are both gone once the harvester patches the settled task's stdout over the handle — so the round-2 suppression (`backgroundHandle == null`) came back on after harvest or reload, showing dispatch time as the task's runtime. Following the same rule as e4bd15d (persist facts, decide at render): the harvest patch now stamps `backgrounded: true` onto the tool call in the same atomic write that erases the handle — on the heal path too, which re-applies over full-row saves that reverted the part. The cards gate on handle-or-marker; the dispatch duration itself stays stored. **Minute-branch digits bypassed locale formatting.** The seconds branch went through Intl.NumberFormat while minutes interpolated raw numbers, so Arabic/Persian locales flipped to ASCII digits above one minute. All interpolated values now flow through the (renamed) formatDurationValue; an ar-EG test pins the localized digits. data-schemas cannot be installed in this environment (same npm ci 403 as packages/api), so message.ts/harvest.ts are syntax-checked with resolution off and otherwise verified by review; CI runs their real typecheck and suites. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🧪 test: Assert The `markBackgrounded` Stamp In Harvest Expectations The successful-harvest test's exact `toHaveBeenCalledWith` object did not include the newly forwarded `markBackgrounded`, so the API suite would fail on it. All three harvest-call expectations now assert `markBackgrounded: true` — the exact-object one of necessity, the two `objectContaining` ones deliberately, since the durable stamp (on the best-effort file-failure path and the reapply heal alike) is now part of the behavior under test. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🎨 style: Wrap Harvest Spec Expectation Per Prettier Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
88747f0ad8
|
🩺 fix: Render Stopped Run Steps From Explicit Status (#14871)
* 🩺 fix: Render Stopped Run Steps From Explicit Status Tool calls decided "still running" vs "stopped" with a whole-message heuristic: const cancelled = !isSubmitting && progress < 1 && !hasError; That inference cannot tell which step actually stopped. An aborted step keeps spinning while `isSubmitting` is still true, and when submitting ends, every unfinished part flips to "Cancelled" at once regardless of which one died. `@librechat/agents` v3.4.6+ emits `on_run_step_closed`, a terminal per-step signal carrying `status` and timestamps — including for steps swept at end-of-run because the caller aborted. The pinned 3.5.1 already ships it; nothing consumed it. - `StepEvents.ON_RUN_STEP_CLOSED` plus `RunStepClosedEvent` / `RunStepStatus` types mirroring the SDK payload. - `PartMetadata.runStepStatus` — a dedicated field, since `status` is already claimed by activity-label and question-form parts. - Server handler forwards the event without the visibility gating the other step handlers apply: a step whose open reached the client must get its close, or the client is left inferring again. - `useStepHandler` writes the terminal status onto the tool call part. - Both decision points (`ToolCall`, the shared `useToolCallState`) prefer explicit status, keeping the heuristic as fallback for messages saved before this and endpoints that do not emit the event. Threaded through the five cards sharing `useToolCallState`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🩹 fix: Address Codex Review On Run Step Closure Rendering - Persist the terminal status server-side. The handler emitted the closure without folding it into `contentParts`, so the status existed only on the live React message: a reload or resumable reconnect dropped it and fell back to the very heuristic this fixes. Now stamped onto the aggregated tool-call part (via `stepMap`, falling back to the event's own index) before forwarding. - Honor terminal status independently of output parsing. Gating on `hasError` meant a `failed` step with unparseable output rendered as "cancelled", while a `failed`/`cancelled` step whose output did parse as an error was not terminal at all and shimmered indefinitely when no completion event arrived. A closed step now forces progress complete and reports `failed` as an error state on its own authority. - Pass the status to the second `BashCall` branch, which rendered the same updated component without it. - Reuse `Agents.RunStepClosedStatus` in `PartMetadata` instead of redeclaring the union, so a future SDK status cannot diverge between the event and the persisted part. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * ♿ fix: Replay Closed Status On Redis Resume, Announce Failures - Apply closure events during Redis reconstruction. The stamp added in the previous commit mutates only the originating process's in-memory `contentParts`; a resumable reconnect landing on another replica rebuilds from `RedisJobStore.getContentParts`, whose allowlist omits `on_run_step_closed`. The status was therefore absent from the sync snapshot and, being snapshot-covered, never redelivered as pending — so multi-replica resume fell back to the whole-message heuristic. Handled as a host-authored event alongside `on_steer_applied` and `on_activity_label`, since the SDK aggregator has no notion of it. - Announce terminal failures in the live region. Forcing terminal progress for a closed step meant a `failed` tool reached the `aria-live` region through `getFinishedText()`, which only special- cased cancellation and otherwise announced "completed function" — telling screen-reader users the opposite of what the card showed. A regression introduced by the previous commit; error states now announce failure before any completion string. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 * 🎯 fix: Resolve Closed Steps By ID, Never By Index The steer and HITL offset wrappers clone and shift only `ON_RUN_STEP` and `ON_AGENT_UPDATE`; every other event passes through untouched. A stored `on_run_step_closed` therefore carries the SDK's unshifted index, while the part it belongs to was rebuilt at the shifted one. Any run containing a steer insertion or HITL resume would stamp the status onto an earlier tool card, or none — leaving the real card on the fallback heuristic while mislabeling a different one. - Redis reconstruction builds a step ID -> index map from the replayed `on_run_step` payloads (which carry the shifted index) and resolves closures against it, mirroring what the live callback does via `stepMap`. - The live handler drops its `?? data.index` fallback for the same reason. Skipping is the safe failure: a missing status degrades to the old heuristic, whereas a misplaced one actively mislabels the wrong card. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014vLhxCFMYkCaTsoFTiAjJ5 --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
b2128a7d18
|
📡 fix: Preserve Redis Abort Terminal Delivery (#14749)
* fix(stream): preserve terminal delivery after Redis fences * fix(stream): cover Redis abort acknowledgment window * chore: sort stream timing imports * fix(stream): follow durable replacement handoffs * fix(stream): bound replacement handoff retirement * test(stream): cover handoff deadline drain * test(stream): cover fenced steer retirement grace * fix(stream): scope fenced retirement lifecycle * fix(stream): retire fenced subscribers safely * test(stream): settle subscription fixtures |
||
|
|
88e08c91e8
|
🧷 fix: Preserve Elicitation Answers Across Aborts (#14745)
* fix: preserve elicitation answers across aborts * chore: sort stream type imports * fix: guard malformed resolved answers * fix: close abort answer race gaps * fix: retain exact answers across pauses * fix: retain answers across resumed pauses * test: satisfy HITL fixture types * fix: retain legacy answers through approvals * fix: preserve answers in reconnect snapshots * test: type legacy answer fixture * fix: bind legacy answers to paused content * test: guard optional resume content * fix: resume questions without streamed content * fix: Preserve legacy answers through abort filtering * test: Narrow reconstructed abort fixture * test: Type abort transform fixture explicitly * fix: Isolate answers with missing ask content * test: Type missing-content ask fixture |
||
|
|
a3cec67e08
|
🪆 feat: Add Parent Activity Phase Summaries (#14721)
* feat: add activity phase summaries * fix: preserve activity phase lifecycle semantics * fix: satisfy activity phase type checks * fix: simplify activity phase status mapping * style: format activity phase changes * fix: rebase activity phase bounds after shaping * fix: link activity phase trace ancestry * fix: reconcile activity phase bounds * style: format activity phase reconciliation test * style: align activity phase assertion * fix: retain reasoning across commentary * fix: preserve activity phase boundary state * fix: detect renderable phase children * test: type parallel phase assertion * chore: bump agents SDK for activity phases * fix: retain unphased lane reasoning * fix: preserve tool group expansion across phases * style: format phase expansion regression * fix: preserve phase interaction state efficiently * perf: skip sparse phase segment holes * perf: partition phase segments with offsets * fix: preserve phase boundaries and cursor state * test: align activity phase regressions with CI * test: keep phase context mock hoist-safe |
||
|
|
58cdd9cd8f
|
⚡ feat: Coalesce Redis Streaming Delta Publications into Windowed Batches (#14614)
* ⚡ feat: Coalesce Redis Streaming Delta Publications into Windowed Batches Every streamed delta currently costs two Redis EVALs (durable append + sequence-allocating publish), and the publish round trip is awaited inside the provider-stream consumption loop. Behind STREAM_DELTA_COALESCE_MS (default off), message/reasoning/run-step deltas now buffer for a small window and flush as one CHUNK_BATCH frame: a single INCRBY reserves consecutive per-event sequences and one EVAL publishes the batch, while a matching batched XADD keeps the durable chunk log on the same cadence so the resume frontier's log-vs-counter timing assumptions hold. Subscribers unpack batch frames at ingress into individually sequenced chunks, so the reorder buffer, duplicate drop, and force-flush behavior are unchanged. Durable, steer-receipt, created, and terminal emissions stay on the awaited per-event path and act as ordering barriers that flush any pending window first; terminal claims flush both sides before the status CAS so a warm tail cannot fence against its own completion. Benchmarked on local Redis (per-scenario RESETSTAT, INFO cpu/commandstats): at 100-200 ev/s a 25ms window cuts EVAL calls 67-82% and Redis engine CPU 52-70%; at the incident's 40 ev/s it halves EVALs while a 20ms window batches nothing (avg 1.0/frame). Producer await stall drops from ~0.9ms/delta to ~0.05ms/delta, matching the previously measured 16-18% USE_REDIS_STREAMS wall-time overhead. Delivery p95 stays under one window (27-28ms at 25ms). * 📝 docs: Document STREAM_DELTA_COALESCE_MS in .env.example * 🚧 fix: Drain Coalesced Windows Before Abort and Shutdown Terminal CAS abortJob and the graceful-shutdown finalizer claim terminal state through their own CAS calls rather than claimTerminalJob, so the pre-CAS coalescer flush did not cover them: a window tail buffered at abort time flushed against the already-aborted status, fenced (-1), and the false receipts retired the healthy runtime and error-closed subscribers before the abort FINAL frame. Extract the flush into flushCoalescedStreamBuffers and call it from all three terminal paths that can interrupt a live emitter (claim, abort, shutdown); the abort call sits ahead of the content snapshot so a chunk-log reconstruction also observes the flushed tail. Regression test aborts mid-window and asserts the tail is delivered with no subscriber error (fails without the fix). Paused-state terminals (approval expiry, pause-persistence timeout) need no flush: the pause's durable barrier already drained the window and nothing streams while paused. * 🛡️ fix: Keep Fence Retire a Lost-Signal Backstop on Aborted Runtimes A cross-replica abort claims its terminal CAS on the aborting replica, so the owner cannot drain its coalesced window pre-CAS; the window flush then fences against the aborted status. When the flush timer lands in the CAS-to-FINAL gap, the false receipts retired the owner runtime and detached its SSE handlers, so the abort FINAL published moments later was dropped and attached clients hung until client-side reconnect. The stop signal reaching the owner (~1ms pub/sub) is proof the abort/replacement flow owns terminal delivery and cleanup, so retireRuntimeAfterDurableFence now returns early for runtimes whose abort signal already landed. The forced teardown remains exactly for its original purpose: a fence observed by a NOT-yet-aborted owner, which is the lost-signal case. Regression test pins the race deterministically via the abort beforePublish hook (which runs between the CAS and the FINAL), forcing the owner flush there: without the guard the FINAL is dropped and the subscriber never completes; with it the FINAL delivers cleanly. * 🧰 fix: Gate, Isolate, and Bound the Coalesced Delta Path Three hardening fixes for the coalescing prototype. The manager now enables the fire-and-forget delta path only when the configured services actually batch — presence of flushPendingChunks/flushPendingAppends is the advertisement — so a custom transport that only implements emitChunk keeps the awaited per-event ordering contract even with STREAM_DELTA_COALESCE_MS set, and a batching transport is never paired with a per-event store (which would let the durable log trail the sequence counter by a full window). Batch unpack isolates each event: a throwing subscriber callback now degrades exactly like a lost individual frame (that sequence stalls until the reorder force-flush) instead of discarding the batch tail whose sequences were already reserved. And the emitter tracks outstanding coalesced receipts per stream, awaiting one once 256 accumulate: healthy settlement is a window plus a round trip so the count sits in single digits and the await never runs, while a stalled Redis now paces the producer exactly like the flag-off awaited path instead of accumulating batches, resolver closures, and queued commands without bound. Unit tests cover the capability gate (hint shape and await behavior for capable, incapable, and window-off configurations) and the backpressure threshold; an integration test pins the unpack isolation (fails without it: the batch tail vanishes instead of recovering via force-flush). Benchmark re-run confirms the counter and gate cost nothing measurable: identical EVAL counts and the serial drain still enqueue-bound. * 🎛️ fix: Make STREAM_DELTA_COALESCE_MS the Single Coalescing Switch The per-instance coalesceWindowMs constructor overrides could disagree with the environment the manager reads: overrides without the env silently did nothing, and an enabled env with an override of 0 selected the un-awaited manager path while both services published and appended per-event. Nothing in the repo passed these options, so remove them — the transport, the job store, and the manager now read STREAM_DELTA_COALESCE_MS through one resolver, making a half-enabled process unrepresentable rather than documented against. The capability-presence gate remains for services that do not implement batching at all. * 🧪 fix: Observe Abort Tail Delivery Before Terminal Teardown in Test The same-replica abort test waited for the coalesced tail only after abortJob returned, but abortJob's finally-block cleanup tears down local subscription state and publish receipts acknowledge Redis execution, not subscriber delivery. Single-node pub/sub delivers sub-millisecond so the frames always won locally; under the CI Redis Cluster they cross the cluster bus and lost the race, timing out the assertion. Await delivery concurrently with the abort instead — the pre-CAS flush publishes the tail several round trips before the teardown, so observing during the call is deterministic in both topologies. Test-only change. |