mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-16 13:00:55 +00:00
6 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
e2f5772961
|
🧮 feat: Surface Retained Tool-Message Context Usage (#15796)
* feat: surface retained tool-message context usage * fix: correct post-snapshot context accounting and lockfile registry Adjudicated the review findings on the retained tool-message context work: - package-lock.json resolved four entries (@librechat/agents, call-bind-apply-helpers, es-object-atoms, get-intrinsic) from a private mirror, breaking `npm ci` outside that network. Restored the public registry hosts; the integrity hashes already match the public tarballs. - `finalizeUsage` migrated every usage atom except `subagentUsageFamily` before dropping the temporary `new` key, so a first-turn subagent total vanished when the conversation id was assigned. - The runway projection divided the pre-invoke `remainingContextTokens` by per-call growth, ignoring the output generated after that snapshot, and the compaction hint subtracted a post-call exchange from a pre-call message total. Both now apply the post-snapshot output, so each side is measured at the same point in time. - The estimate path rendered its tool-call share as a peer row although it is a subset of Input/Output/Estimated, so the visible rows summed past `usedTokens`. It is now an indented subtotal; the counted rows stay provider-accurate. - The Totals block presented conversation-wide session subagent usage beside branch-scoped rows; it is now labeled all-branches. - Rewrote the truncated `rgb-series-8` rationale as one comment on the series block. * fix: keep subset accounting whole across cache, legacy snapshots, and custom themes Adjudicated the second review round on the retained tool-message context work. One cause per finding, each measured on the same basis as what it is compared to: - The all-branches subagent subtotal summed input+output only, so a cached subagent call was under-reported and a fully cached call with no output disappeared from the row while the Cache read/write rows above still counted its traffic. It now sums every normalized bucket. - `collectAnchorSeries` read an absent `remainingContextTokens` as zero and recorded that snapshot as having spent its entire budget, which distorted the growth delta feeding the runway projection. A snapshot saved in the older shape now derives its used total from the breakdown, exactly as the render path's `baseUsed` does, and one that carries neither basis is skipped. Each reading is tagged with its basis and the projection compares only readings of the same basis instead of reporting the difference between two measurements. - `resolveTheme` filled an omitted `rgb-series-8` from the bundled base, so a stored or environment theme that paints its own seven-slot scale got LibreChat's indigo on surfaces it never checked: 2.41:1 against that theme's meter track, under the 3:1 mark floor the scale claims. An owned scale now derives the slot from the theme's own neutral text — 11.5:1 on the same reference theme — and naming `rgb-series-8` still opts out. - Ordered the `--series-*` declarations in the light block by slot. * fix: keep cached shares out of the peer rows and track resolved text for slot 8 Third round on the retained tool-message context work, correcting the two variants of invariants this PR already fixed elsewhere rather than patching them locally: - The snapshot path rendered the reconciling call's Cached and Cache write shares as peer rows with window percentages, although reconciliation already counted those tokens inside the segments above. A fully cached prompt showed its tokens twice and the visible rows summed past the meter. They now render as an indented subtotal, the same treatment the estimate path's tool-call share received. The existing test claimed this invariant in its name but only asserted the rows exist, so it now asserts they are not peers. - The `rgb-series-8` fallback required the theme to name its own text, so a theme that repaints the scale and its surfaces while inheriting LibreChat's text colours still received the bundled indigo. It now derives from the RESOLVED secondary text, so the stop tracks whatever the theme reads its body copy against whether it names one or inherits ours. * fix: settle the subagent share with its run and measure both reclaim sides alike Fourth round on the retained tool-message context work: - The subagent share was committed on arrival while the usage it is a subset of stayed pending, so a run that ended with no salvageable response left its tokens in the Totals after `resetLive` discarded them from every rollup, and a resume re-folded the same events into the cumulative figure. It now accumulates in `pendingSubagentUsageFamily` beside the pending usage and settles with it: committed by `flushPendingInto` when the response keeps its usage, dropped by `resetLive` otherwise, migrated with the conversation id. The view merges committed and in-flight shares, so the live row is unchanged. - `latestExchangeTokens` subtracted a summarizing turn's whole response, including the summarization completion the backend folds into `tokenCount` while the snapshot holds it in `breakdown.summaryTokens` rather than `messageTokens`. The reclaim hint was understated by the summary and could reach zero after a long one; the summary block's size is now excluded from the tail's contribution when that turn compacted. - `sumBranch` exposed the tail's char estimate and tool share even when the tail carried a `tokenCount`, so a resumed partial response stayed in the message total while its tool traffic was dropped from the Tool calls split. Both tail figures are now zero for a counted tail, which also stops the estimate path subtracting an estimate that was never summed. * chore: drop the lockfile churn the base branch already carries The SDK bump this PR set out to make landed on dev as 3.8.5, so the branch no longer changes any package.json and its lockfile edits (the 3.8.4 entry plus the transitive call-bind-apply-helpers/es-object-atoms/get-intrinsic bumps that install pulled in) are churn against the base. * fix: give the eighth series slot a value every bundled theme can carry The base branch grew high-contrast palettes that must declare every registry token, and a test that a series slot can carry `text-on-status` at the 3:1 mark floor, because a slot also fills a badge chip under that glyph. Slot 8 failed both: the high-contrast palettes could not name a token that did not exist when they were written, and the dark indigo #8c98e6 carried white at 2.71:1. Dark slot 8 is muted to #8082be, which reads 3.66:1 under the status label and still clears 3.66:1 on every series surface. The high-contrast palettes take their own neutral text as slot 8 — the same stop `resolveTheme` derives for a third-party theme that paints its own scale — because seven saturated hues is all either canvas affords at their stated CVD separation; it is distinguished by lightness at 21:1 on the canvas instead of by hue. * test: exercise the context split through the mock harness Nine scenarios of the acceptance contract, one tagged Playwright test each, driving the real application: a tool round-trip through the e2e MCP server for the Messages/Tool calls split, its keyboard disclosure of per-tool counts, the peer rows staying inside the meter and surviving a reload; a real subagent run for the all-branches total; seeded conversations for the legacy snapshot whose used figure comes from its breakdown and for the estimate path's nested tool share; and a stored theme that owns the series scale for the eighth slot's fallback and its opt-out. * test: keep the tool and subagent scenarios on the viewports that expose them * 📌 feat: Unified Pinned Section With Sidebar Drag Interactions (#15057) * feat: unified pinned section with drag interactions and sidebar polish - Pinned chats and favorite models/agents/specs render as one interleaved, reorderable list persisted via users.pinnedOrder - Drag conversations onto project rows to file them, onto Chats to unfile, and onto the Pinned section to pin (empty section materializes as a drop zone during a conversation drag) - Row-level unpin button on pinned chats matching the favorites' button, and the row action menu shares the same hover/focus treatment - Agent Marketplace entry moved under the New Chat button in the side rail - Bookmarks trigger sized to the search bar, search focus no longer swaps its border color, tag filtering shows a loading state, and opening a project shows row-shaped skeletons * fix: add pinnedOrder to IUser type and sort imports * fix: pinned drag threshold, unpin focus target, and marketplace link * fix: surface failed pinned reorders and skip no-op order writes * fix: address review findings on unified pinned section Move the pinned-order endpoints into packages/api as TypeScript handlers and route their URLs through the data-provider endpoint registry. Raise the entry and key caps so a valid model favorite and an unbounded pin count can both be persisted. Keep the marketplace entry reachable from the mobile drawer, which is the only sidebar surface on small screens. Merge a reorder into the stored order while a bookmark filter hides part of the list, instead of dropping the hidden keys. Add Alt+Arrow keyboard reordering with a live-region announcement, restore the focus handoff when a favorite is removed, and hold the local arrangement until the stored order catches up. Serialize overlapping order writes and discard the optimistic value when the first one fails. Derive the tag-filter loading state from isPreviousData rather than a flag that could stick on. * fix: release drag while renaming and dedupe repeated favorite keys A draggable ancestor swallows drag-select inside the rename input, so both the row's own drag source and the pinned wrapper's release while a title is being edited. The favorites payload validator accepts a repeated entry, and two rows keyed the same would collide in React, point every drag at the first copy, and fail the order endpoint's uniqueness check. Build the pinned list keyed-unique. * fix: address PR review bot findings Make the model favorite key injective. Joining endpoint and model on '::' collided for components containing the delimiter, so two distinct favorites keyed the same and the dedupe hid one; the endpoint is now length-prefixed. Replace the pinned-order entry cap with a total size guard. Pinning has no membership cap and the sidebar query drains every cursor, so any count limit rejects a legitimate list rather than bounding anything; what needs bounding is the user document. Let only the newest order write touch the cache. An earlier write failing rolled back over a newer arrangement that was still in flight, leaving the UI on a stale order while the server held the current one. A confirmed write now takes the server's answer, and a failed newest write discards the value that was never accepted. Persist the favorites array only once the order write succeeds, so a rejected write cannot leave the two stores disagreeing with the rolled-back display. * fix: address second round of PR review bot findings Read favorites membership at write time rather than capturing it at drag end. A favorite added or removed while the order request was in flight was undone by writing the older list back; only the relative order now comes from the drag. Move the pinned-order queue to module scope. The sidebar unmounts whole sections while a search is active, so a component-local ref let a remounted hook start a second queue alongside a request that was still running. Serialize project assignment per conversation. Every drop target owns its own mutation instance and the write is an unconditional update, so two quick drops let whichever request reached the database last decide the project. Read the pinned-order request body null-safely: destructuring a nullish body threw past the handler into Express's generic error path instead of returning the intended 400. * fix: address third round of PR review bot findings Drop the secondary favorites write. No consumer of the favorites array reads its order, so mirroring the arrangement there bought nothing while racing the membership mutations that share that whole-array endpoint; pinnedOrder is the single source of ordering. Always merge the committed order into the stored one. The visible list is not always the whole list, since a bookmark filter hides rows and the pinned query publishes partial results while draining its cursor, and replacing wholesale dropped every key it could not see. Release the local arrangement when the write settles instead of when the stored order matches it key for key. A pin arriving mid-request made that equality unreachable, stranding the snapshot and hiding the new row until remount. Abandon a queued pinned-order write if the signed-in user changed, so a deferred request cannot write one account's order into another's document. * fix: address fourth round of PR review bot findings Guard the pinned-order completion callbacks on the signed-in account. The pre-send check only covers a write that has not left yet; a response arriving after a session change would publish one account's order into the next one's cache, which this query never refetches on its own. Commit a reorder only when the pinned rows handled the drop. Dragging a pin onto a project row or into Chats sweeps the pointer across sibling rows, whose hover handlers shift the order, so a filing action was also saving a move the user never asked for. * fix: address fifth round of PR review bot findings Deselect pinnedOrder at schema level. It is display-only and allowed to grow large, but every authentication request loads the user document, so it was adding its whole payload to paths that never read it. Refuse to reorder until the stored order has loaded. Merging against an absent order and then cancelling its fetch posted only the visible keys and discarded saved positions for good. Prune keys only once the visible list is known to be the whole list. Merging unconditionally grew the array until the size guard rejected every write, so completeness now gates compaction rather than the filter flag alone. Track every renaming row, not just the most recent. RenameForm has no blur cancellation, so a second form can open while the first is still mounted and the single key reconnected the first row's drag source under its open input. Judge drop eligibility by the queued destination. A drag item carries the project its row had when the drag began, so re-dropping a chat back where it started while an assignment was queued read as a no-op. Take mutation ownership from the user atom AuthContext fills on login rather than the user query, which is still empty in that window, and never treat an unknown owner as a match. Restore the pinned section's 30vh cap. Matching the Projects section's 42vh put 84vh of non-shrinking content above Chats and starved it on short viewports. * fix: address sixth round of PR review bot findings Gate reordering and pruning on successful fetches rather than attempted ones. A failed pinned-order GET still reports as fetched, and a failed later page of the pinned drain still publishes partial data and stops fetching, so both gates were opening on exactly the states they were meant to exclude. Scope the post-unpin focus handoff to this section by its own marker. The whole conversations pane is a labelled region too, and the project lists inside it render the same row, where unpinning only clears the flag and the row stays put, so focus was being moved off a row that never went away. Clear the account tracker when the mutation hook unmounts. The sidebar drops that section during a search, leaving a stale non-null id that also shadowed the cache fallback, so a deferred write could pass its owner check after a session change. * fix: gate pruning on a delivered favorites list A favorites fetch that exhausted its retries leaves isLoading false with the list empty, so an otherwise complete pinned load enabled pruning and dropped every favorite key from the stored order. useFavorites now exposes the query's success state and the pruning gate requires it, matching the pinned-order and pinned-membership sources. * fix: address eighth round of PR review bot findings Identify queued assignments by token rather than by destination. Comparing destination values cannot tell two drops onto the same project apart, so a B, A, B run let the first B's cleanup discard an entry the still-queued later drops owned, and the next drop was then judged against a stale project. Report the rename ending when a row unmounts. A pinned row removed mid-rename, for instance by unpinning the same chat from the project list that also renders it, left its key in the renaming set and came back permanently undraggable. * fix: address ninth round of PR review bot findings Stand the keyboard reorder down wherever the pointer drag does. Alt/Option plus an arrow moves the caret word-wise inside a text field, so the shortcut was moving and saving the whole row out from under an open rename input. Hold a settled assignment destination until a drag item shows the new project. The mutation's cache invalidations are not awaited, so a row still reported its old project for a moment after the write succeeded and a quick drag back to where it started read as a no-op. * fix: address tenth round of PR review bot findings Serialize project assignment inside the mutation instead of at one caller. The drag targets, the row menu and the project dialog each hold their own instance, so a queued drop could still overwrite a newer menu write; moving the queue down covers every surface and lets the drag helper drop its own. Tie the live snapshot's release to the arrangement that created it. An older write settling mid-drag cleared the newer arrangement, snapping the rows back to a stale order while the drag still held its own, so releasing saved an order the user could no longer see. * fix: address eleventh round of PR review bot findings Move assignment destination tracking next to the queue that owns it. Keeping it in the drag helper meant a menu or dialog move left the drag helper still reporting its own older destination, with no row left to observe that would ever clear it, so a later move back was rejected for good. Resolve a conversation's project from the write the mutation confirmed rather than from whichever list rendered the row. The same chat appears in the Pinned section and its expanded project, and those caches refresh independently, so observing one of them was never a sound way to decide the other had caught up. The conversation cache is written synchronously on success, which makes the lazy self-clearing and its settled flag unnecessary. * fix: cancel an in-flight conversation refetch before confirming an assignment Navigation invalidates the exact conversation query, so a refetch that read the old project can be in flight when an assignment resolves and land afterwards, reverting the move in the cache that now answers where a conversation lives. * fix: keep queued pinned-order writes attributable across an unmount Refuse a queued write only when a different account is positively established. Clearing the identity on unmount stopped stale attributions but stranded work that belonged to the same account, since the sidebar drops the section that owns this hook whenever a search is active, and it denied that work its rollback as well. Correct the cache from the request rather than from onError. The observer callbacks do not run once that section unmounts, so an order the server never accepted could survive on a query that refetches on nothing. * fix: publish the session identity from the user atom Queued pinned-order writes were attributed by a tracker that only the pinned section's hook updated, and the sidebar unmounts that section during a search. A retained value went stale across a sign-in, and login clears the user query before the next account's fetch resolves, so a write queued by the previous account could pass its own check and post on the new account's credentials. A Recoil atom effect publishes the identity instead: it runs wherever the atom is set, including sign-out, so attribution comes from the state itself rather than from a component that happens to be mounted. That makes a signed-out session foreign to every owner and lets the check be a strict comparison again, rather than a rule that had to guess when identity was merely unreadable. * fix: discard reorders that ended on a refused external target A drop the pinned rows did not handle usually means a reorder, but a project row or the Chats section refusing one reports the same, so dragging a chat onto the project it already belongs to was saving the incidental shift its pointer caused on the way out. The rows and those targets now report their hovers, so the last one under the pointer settles it, which also allows for a drag that strays over a project and returns to reorder after all. * fix: attribute queued writes to the credentials they will travel with Publishing the account from React state, whether an effect or a Recoil atom effect, always lagged the credential change: setUserContext installs the new Authorization header in one synchronous call, and a write queued by the previous account and sent in that gap read its own owner back and went out as the new one. A test against the atom effect showed onSet is deferred too, so the atom was no better than the effect it replaced. The header is the thing that decides who the server sees, so ownership is read from it. The id claim is compared rather than the token, since a silent refresh rotates the token for the same account and discarding that work would strand something the user is still entitled to. Leaves AuthContext and the user atom untouched. * fix: attribute a reorder to the account that started it onMutate awaits cancelQueries before the request is built, and the mutation function was reading the session again afterwards, so credentials turning over inside that gap sent one account's order as the next. The owner is now recorded in onMutate's first synchronous statement and carried on the order itself, and the optimistic write is skipped when the session moved on, since that cache is shared and showing an order to the wrong account is the same mistake as sending it. Record a pinned hover before the guards rather than after. Hovering the dragged row is a no-op for ordering but still means the pointer is inside the list, so a drag that wandered onto a project and returned to its own position was being discarded as a filing action. * fix: address a further round of PR review bot findings Mark a hover anywhere in the pinned section as internal. The header and the padding around the rows are part of the list, so a drag that wandered onto a project and came back to rest on one of them ran no row hover and was discarded as a filing action. Hold off pruning while a favorites write is unsettled. The earlier GET's success stands through an optimistic removal, so pruning then dropped the ordering key of a write that had not landed, and a failed one came back without its saved position. Declare the reorder shortcut on the elements that take focus. The section's written hint is skipped by anyone tabbing straight to a row, leaving the only non-pointer way to reorder undiscoverable. * fix: reach a real New Chat control, and let the shortcut prop through the memo The unpin fallback looked for a test id no component carries, so once the last pinned row unmounted focus fell to the document rather than to New Chat. It now tries the controls that exist, in sidebar order. The test that covered this had fabricated the missing id, so it proved nothing. Convo is memoized on conversation fields and isGenerating alone, so the shortcut appearing once the pinned order loaded never reached the element that announces it. The comparator now accounts for it. * fix: identify OpenID sessions, and see favorites writes from every hook With OpenID token reuse the bearer is the provider's own id_token, which names its subject as sub under an issuer and carries no LibreChat id, so every such session resolved to nobody and two different accounts compared equal. That is the case the ownership guard exists for, so it read as safe while being exactly wrong. Sessions are identified by whatever the credential offers now, and an owner that cannot be identified is foreign to everything including itself. Count favorites writes across the query client rather than from one observer. A favorite row runs its own useFavorites, so a removal started there was invisible to the pinned section and its ordering key could still be pruned. * fix: close the assignment cache gap, and let a failed drain be retried The assignment released its pending destination as soon as the request settled, while the cache write waited on an awaited cancellation in onSuccess. A drag started in that gap saw neither, fell back to the stale row and refused a valid drop. The write now happens in the request, before the entry is released, which also survives the owning component unmounting. Also clear the pagination guard when the Chats section collapses. A drain that exhausted its retries leaves the conversations array unchanged, so the guard barred every later attempt and the remaining chats stayed unreachable for the session. Reopening the section now tries once more. That guard came from #14860 rather than this branch, so it can be split out if preferred. * fix: compose the marketplace link, and keep focus on a surviving row The marketplace control was a raw link with restated appearance classes, and in restating them it left out the focus ring entirely, so keyboard focus on it was invisible. It composes through the shared button now, which carries the ring, the hover fill and the theme timing. Unpinning from a project list leaves the row in place but removes the pin badge that had focus, and the pinned-section successor search deliberately returns nothing there, so focus fell to the document. It moves to the row's own link. Reconcile the pinned order on window focus and reconnect. It is one shared per-user record, so a second tab would otherwise show a stale arrangement and its next reorder would post a full array undoing the other tab's. * fix: keep a rename from restoring a conversation's other fields A rename request carries only a title, but its response is the whole conversation as the server saw it, and writing all of that back restores the pre-request value of every other field. An assignment confirmed while the rename was in flight was therefore reverted in the cache that now answers which project a conversation belongs to, and a drag back to the original project read as a no-op. Only the title and its timestamp are taken from the response. Cancelling queries cannot help here, since this is another mutation's write rather than a fetch. * fix: compose the unpin badge through the shared row-action button The badge restated a reusable icon control's sizing, radius, hover and focus classes. The client package already had a row-action variant and an icon-xs size that match it exactly, so it uses those instead of its own copy. * fix: reconcile before reordering, and scope assignments to their session A reorder stayed enabled while the order's focus refetch was in flight, so returning to a tab could cancel that refetch and post the arrangement it was about to replace. Pruning had the matching gap: the pinned list keeps its data fresh for five minutes, so a tab whose order had reconciled could prune against membership that had not, dropping the position of a conversation another tab pinned. Both now wait for data that is settled and not stale. Assignments carry the account that started them, like the order writes already did. Conversation ids are per-user, so a queued one travelling with the next account's credentials would act on whatever that id names over there, and its answer would describe one account's conversation to another in a cache that is not scoped by user. The row's overflow trigger derives from the shared row-action variant too, so the two actions in a row stop keeping separate copies of the same recipe. * fix: reject foreign assignment results, and share the section header An assignment settling after a session change returned normally, so the success path still ran: the caller reported success and the response could be applied to the next account's conversation, since ids repeat across accounts. It rejects now, so nothing downstream treats it as a result. The collapsible section header existed as three identical copies, Chats, Projects and Pinned, and this branch added the third. It moves into the shared button as a section-header variant and all three compose from it. * fix: let favorites membership reconcile between tabs The favorites query disabled focus, reconnect and mount refetches, so a second tab kept its old list indefinitely. Its pruning gate waits for a fetch, and none was ever going to happen, so a reorder there could delete the saved position of a favorite the first tab had added. It reconciles on focus and reconnect now, like the order it is arranged alongside. * fix: hold reconciling refetches off while a write is in flight Enabling focus and reconnect refetches on the order and favorites queries let one start after onMutate's single cancellation, read the pre-write state and land after the write, putting the old value back. Both now decline those refetches while a write for that resource is outstanding, keeping the cross-tab reconciliation itself. The check asks the query client at the moment the event fires rather than reading useIsMutating: that hook only reaches the query's options on the next render, which is a race against the very event being gated. A test that dispatches focus mid-write fails without it. * fix: release abandoned assignments and gate pruning on current membership An assignment rejected before entering the try block kept its pending entry forever, so a later session read a destination that was never sent. The ownership check now sits inside the block that releases it, and it runs again after the conversation query cancellation resolves, since the session can turn over while that await is pending. Pruning the pinned order now requires membership at least as current as the order it is pruned against. The pinned query holds its data for five minutes while the order has no such window, so focusing a tab reconciled the order alone and the next reorder dropped the key of a conversation another tab had pinned in between. The assignment spec resets its mocks between tests and gives each test its own conversation id, so one test's unconsumed mock and leftover queue entries no longer decide the next test's result. * fix: address the sixth round of review findings on the pinned section The section header recipe asked for the shared button variant without a size, so CVA emitted the default `h-10 px-4` after it and the merge kept it, putting a 40px control in a 32px header row. The variant opts out of default sizing through a compound now, since a heading is sized by its own text. Restore the search field's focus border. Removing it left keyboard focus signalled only by a background change that hover already uses, so the field had no distinguishable focus state at all. An assignment can reach the database before a concurrent rename and still answer after it, carrying a conversation that predates the rename. Only the project field is merged into the cached conversation and published to the active one now, matching what the rename handler already does. An order refetch that starts mid drag no longer lets the drop post an array built from the order that refetch is about to replace: the arrangement is held and written once the fresh order lands. Reorder writes superseded by a newer arrangement are skipped rather than sent, so holding the shortcut costs two writes instead of one per intermediate position. The order and favorites queries reconcile on mount as well, since a sidebar search unmounts the section and a query with no observer never sees the focus event. * fix: preserve pinned order until favorites membership catches up * fix: reconcile pinned membership and preserve auth and focus state * fix: preserve reconciled keys during active pinned arrangements * style: sort pinned order regression imports * fix: reconcile favorite rollbacks before pinned-order pruning * 🧊 fix: Withhold Rerun Controls on Compaction Turns (#15797) * 🧊 fix: Withhold Rerun Controls on Compaction Turns A compaction turn's response parents onto the leaf it summarized, not onto a user message, so the hover Edit's Rerun replayed an assistant message in the user slot: the request minted a user message under the existing assistant leaf's id and ran the turn with empty text, which fails. Editing offered nothing either, since SUMMARY is not an editable part type. Edit, Regenerate and Continue are now withheld on a compaction turn, and both rerun paths refuse a parent that is not user-created, matching the guard `useChatFunctions.regenerate` already applied. * 🧭 fix: Gate Rerun Controls on the Replayed Parent Turn Withholding Edit, Regenerate and Continue on summary-only content also hit an ordinary turn that auto-summarized and was cancelled before its first answer token: that response is parented to the user's message and is exactly the turn a rerun is for. The row now resolves the message it would replay and withholds the rerun shapes only when a model turn hangs off another model turn, which is the manual compaction shape. An unresolvable parent withholds nothing. `isCompactionTurn` is gone with it, and `isCompactedLeaf` keeps its original body; its docstring now says per part what it always measured per part. * 🧮 fix: Identify a Compaction Turn by Content and Parent Gating the rerun controls on parent authorship alone withheld Edit, Regenerate and Continue from any model turn hanging off another model turn. An imported conversation chains messages sequentially, so a skipped empty human message leaves an ordinary reply parented to the previous reply; its stored content is editable and its controls must stay. It also made every model row resolve its parent out of the flat message list: 100 full-thread scans while mounting a 200-message conversation. A manual compaction is now both halves of its shape — summary-only content and a parent that is not the user's turn — so only such a row consults the thread. `isSummaryOnlyContent` is the content half, in data-provider next to `isCompactedLeaf`. * 🧷 test: Pin the Rerun Refusal on a Chained Model Turn A rerun replays its parent as the user turn, so a model turn parented onto another model turn must submit nothing rather than mint a user message under an existing response's id. Removing the guard makes the case call `ask` once. * 🧱 fix: Gate the Editor on Content and the Rerun on the Parent Identifying a compaction turn by content shape missed the ones the server persists with an error part (a skipped or failed compaction keeps its explanation instead of a summary), which left Edit, Regenerate and Continue on a turn where every rerun shape is inert. The two controls answer two different questions now. A rerun replays the message's parent as the turn's user message, so Regenerate and Continue go whenever a model turn hangs off a model turn. The editor goes only when that turn also has no part to type in: an imported chain's reply is saved directly and stays editable, while a compaction — summary or error part — would open an empty editor with one inert Rerun. `findMessageById` indexes the messages array once per identity, so resolving the parent no longer scans the thread per row: 100 full-array walks while mounting a 200-message conversation become one. * 🏷️ fix: Honor the Server's Compaction Marker on Any Leaf Compact runs on whatever leaf the branch ends with, so a compaction can hang off a user message rather than off the answer it summarized. Inferring the turn from its position alone left the rerun controls on that one: replaying the parent would answer the user's earlier message again instead of redoing the compaction. `isUserInitiatedCompaction` reads the marker the server already stamps — `markCompactionSummary` sets `initiatedBy: 'user'`, and nothing else writes it, so an automatic summary detour keeps its controls. A marked turn is excluded from the rerun shapes wherever it hangs. * 🎨 fix: Ask the Editor What It Can Edit The hover row had its own idea of an editable part and counted artifact-bearing text as one, while `EditContentParts` keeps an artifact in its read-only renderer. A chained model turn whose only content is an artifact therefore still offered Edit, and opening it gave a read-only artifact, a disabled Save and a Rerun the guards refuse. `editableParts.ts` now owns that rule for both: `isEditablePart` is the editor's own filter, `hasEditablePart` asks it per message. The row resolves it lazily — the artifact check parses markdown, and only a turn with nothing to replay needs the answer — so a streaming row never pays for it. * 🚫 fix: Withhold the Editor's Rerun Instead of Refusing It Both editors resolved the parent inside the handler and returned when it was not a user turn, so a chained model turn — an imported thread with no user message between two replies — kept an enabled Rerun that did nothing, with Ctrl/Cmd+Enter bound to it. Each editor resolves that parent at render and withholds the action instead: the button is gone, the shortcut is inert and `aria-keyshortcuts` no longer advertises it, while Save keeps writing the edit. An unresolvable parent keeps the action, matching the hover row. * 🧹 style: Sort the New Imports in Both Editors CI's import-sort check failed on the two files the previous commit touched. * 💬 fix: Report Unsaved Changes When No Rerun Is Offered A save-only editor still warned about the rerun. An answer's draft is discarded by the rerun, and multi-part edits must be saved before one, but neither is true where the action is withheld — a model turn chained onto another model turn — so the status slot claimed a precondition for something the footer does not offer. Both statuses are gated on the same `canRerun` the button is: the plain-text editor reports unsaved changes, and `rerunRequiresSave` cannot hold without a rerun to require it. * 🧪 test: Exercise the Compaction Controls in the Browser Eight mock-harness scenarios drive the real row and the real editors: a finished, failed, error-part and user-parented compaction withhold Edit, Regenerate and Continue while Copy stays; a turn that only auto-summarized keeps all three; a reply chained onto a reply keeps a save-only editor whose status reports unsaved changes; an artifact-only chained reply loses the editor; and the context indicator still offers Compact after a failed compaction. `SeedMessage` carries `content`, `finish_reason` and `model` so a branch can be seeded in the shapes the mock model cannot produce — a persisted summary, an error part, an artifact. * 🩻 feat: improve agent filter controls for insights (#15807) * feat(insights): improve agent filter controls * feat(insights): add search and fixed agent selection controls * fix(insights): show search before all agents control * test(e2e): allow steer consumption before arm response * 🔠 fix: Name Artifact Downloads After Their Titles (#15836) * fix: Use artifact titles for download filenames * fix: Match artifact filenames to downloaded content * fix: Preserve raw artifact extensions and GFM headings * fix: Download complete artifact files and retain filename provenance * fix: Distinguish cached artifact exports from originals * fix: Name unnamed artifact originals by their media type * fix: Recognize legacy CSV MIME in artifact filenames * fix: preserve context insights across reloads and configuration changes * fix: retain primary call details when resuming context snapshots * fix: preserve resumed output and retained tool-result estimates * fix: keep resumed output compatible with older clients --------- Co-authored-by: Ravi Kumar L <upman@users.noreply.github.com> Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
|
d18d62e7c1
|
🪙 refactor: Reconcile Context Gauge to Actual Provider Tokens (#13780)
* 🪙 fix: Reconcile Context Gauge to Actual Provider Tokens The context gauge could read several× too high (e.g. 213K when the real prompt was 56K) and stay there across reloads. Root cause: the SDK's calibrationRatio is `cumulativeProviderReported / cumulativeRawSent`, but a provider's server-side web search injects large fetched content into the prompt that the SDK never sent or counted — pinning the ratio at its cap (5) and multiplying every later message estimate, including post-summary ones. The gauge rendered (and persisted) that inflated estimate, never the provider's actual token count. Fix: reconcile the snapshot to the call's ACTUAL prompt tokens (input + cache), which already arrive in on_token_usage. Only messageTokens is calibration-scaled (instructions/summary are raw tiktoken), so keep those and set messageTokens to the remainder, recomputing free space. Shared `promptTokensFromUsage` + `reconcileContextUsage` in data-provider; applied server-side in buildPersistedContextUsage (reload-stable) and client-side in useUsageHandler on each primary usage (corrects at turn-end, no follow-up needed). Also drop the summary double-count from the Breakdown Messages row. Deferred (separate agents PR): the SDK over-calibration also fires summarization prematurely; fixing it needs decoupling real-content estimation from server-side injection headroom without weakening pruning-overflow safety. * 🪙 fix: Harden Token Reconciliation for Provider-less + Resume Paths Codex review on the reconciliation: - promptTokensFromUsage: when the provider is absent (custom/OpenAI-compatible payloads), fall back to the same magnitude heuristic normalizeUsageUnits uses (cache ≤ input ⇒ already included) so cached events aren't re-inflated. - Resume: backfillUsage restores a primary call's usage without replaying a live on_token_usage (Redis mode), so the live reconcile never ran and a reconnected session stayed on the inflated estimate. New reconcileBackfill reconciles the restored snapshot from the final primary call after contextHandler installs it. * 🪙 fix: Reconcile Resume Snapshot Server-Side, Not via Backfill Codex: the client reconcileBackfill scanned the resumed run's collectedUsage and applied the final primary to the latest snapshot — but on a mid-call resume that usage belongs to an EARLIER call, corrupting the restored gauge. Move the resume reconciliation server-side: GenerationJobManager.persistTokenUsage reconciles the stored contextUsage to a primary usage's actual prompt tokens as it arrives. That usage is the post-invoke truth for the call the latest stored snapshot precedes (no snapshot is captured between a call's pre-invoke dispatch and its usage), so it's correct by construction and run-matched. A mid-call resume (no usage yet) keeps the raw snapshot instead of mis-applying an earlier call's tokens; it reconciles once the call completes. Removed client reconcileBackfill; the live-path reconcile (non-resume) stays. * 🪙 fix: Guard Reconciliation Against Replays and Snapshot Races Two Codex concurrency findings on the reconciliation: - Client: reconcile only on a NEWLY folded primary usage. A replayed duplicate (folded=false on resume) can be an earlier tool-loop call sharing the run id, which would overwrite the latest snapshot with an earlier, smaller prompt. Moved the reconcile after the folded guard. - Server: serialize the context-usage write through the same per-stream queue as the token-usage write. persistTokenUsage reconciles the stored snapshot (read-modify-write); an unserialized trackContextUsage could store a newer snapshot between the read and write — or a stale reconciled write could land after a newer snapshot — clobbering the newer run's gauge when calls interleave. FIFO keeps each call's snapshot ahead of its own usage and behind the next. * chore: import order in GenerationJobManager.ts |
||
|
|
44c253d48a
|
🪙 fix: Correct Context Usage Gauge After Summarization (#13744)
* 🪙 fix: Persist Context Snapshot + Summary Marker After Summarization The post-summarization context is correctly compacted by the SDK, but the breakdown wasn't reliably reaching the client, leaving the gauge on the whole-history estimate (stuck at 100% forever once a conversation compacts). Two server changes in buildResponseMetadata: - Snapshot guard: persist the breakdown when a PRIMARY usage event follows the latest snapshot (tracked via contextUsageSink.latestUsageIndex, recorded in the on_context_usage handler) instead of a brittle snapshot-vs-primary count. A summarization detour adds an extra snapshot whose only following usage is tagged 'summarization', which the count guard could miscount and drop. - Summary marker: whenever a turn compacts (summaryTokens > 0), persist a lightweight metadata.summaryUsedTokens (the pre-invoke compacted context size) UNCONDITIONALLY — so even when the full snapshot can't be saved (interrupted final call) or never reaches the client, the per-message estimate has a signal to cap the discarded history. Tests: client.contextMetadata.spec (guard + marker, incl. marker-survives-drop) and a real-pipeline summarization integration test. * 🪙 fix: Cap the Context Estimate at the Summary Marker When the gauge falls back to the per-message estimate (no usable snapshot on the branch), sumBranch summed the ENTIRE branch history — after a summarization that discarded most of it, this over-counts and pins the gauge at 100% in perpetuity. sumBranch now stops at the deepest summarized response (metadata.summaryUsedTokens) and records it as summaryBaseline; the walk counts only post-summary messages, and useTokenUsage adds the baseline. So the estimate reflects the compacted context (summary + recent turns), not the discarded history. USD/default behavior unchanged when no marker is present. Test: sumBranch caps a huge pre-summary history at the compacted baseline. * 🪙 fix: Address Codex Review on the Summarization Marker - Branch cost/usage is no longer truncated at the summary marker — sumBranch caps only the CONTEXT-window count there and keeps accumulating provider usage/cost to the root (cumulative spend isn't discarded by compaction). - findBranchSnapshotAnchor stops at a summarized response with no snapshot of its own, so it can't recover a stale PRE-summary snapshot and show discarded history; the summary-baseline estimate is used instead. - Abort path: buildAbortedResponseMetadata now persists the summaryUsedTokens marker (pre-invoke, no completedOutputTokens ambiguity, so safe on abort) so a STOPPED summarized turn isn't re-summed on reload. - Marker baseline fallback now includes summaryTokens (a separate breakdown field) so it doesn't under-report the compacted size. DRY'd into a shared computeSummaryUsedTokens used by the completion and abort paths. - Estimate popover surfaces the summary baseline as a row so the displayed rows reconcile with the header total. Tests: sumBranch cost-not-truncated + anchor-stops-at-marker (client); computeSummaryUsedTokens fallback + abort marker (packages/api). * 🪙 fix: Attribute Persisted Context Usage to the Snapshot Run Match the post-snapshot primary usage to the latest snapshot's runId before persisting metadata.contextUsage. Parallel/direct runs interleave snapshots and usage (A snapshot → B snapshot → A usage → B no-usage); the prior index-only guard persisted B's snapshot with A's output. finalCallOutputTokens now filters completedOutputTokens to the snapshot's run. Untagged events (older lib/resume) match any run for back-compat. * 🪙 fix: Harden Summary Marker Against Tool-Loops, Stale Anchors, and Emit Races Codex round on the summarization marker: - Avoid double-counting earlier tool-loop outputs in the summary marker: those outputs sit in BOTH the latest snapshot's pre-invoke baseline AND the response message's tokenCount the client estimate adds on top. computeSummaryUsedTokens now subtracts the run's prior primary outputs (priorRunOutputTokens) — the live path bounds them by the snapshot's usage index, the abort path by all primaries (an interrupted final call emits none). Single-call turns subtract 0. - Stop treating pre-summary anchors as active: sumBranch no longer sets containsAnchor once the context is capped at a summary marker, so a stale pre-summary snapshot can't override the summary-baseline estimate. - Capture latestUsageIndex BEFORE awaiting emitEvent: a yield (resumable SSE / Redis) during parallel runs could let this call's own usage advance the index past the event that proves the snapshot completed, dropping a valid breakdown. * 🪙 fix: Subtract Summarization Output from the Summary Marker recordCollectedUsage folds the summarization call's completion into the response message's tokenCount, while the generated summary is also in the snapshot baseline as summaryTokens. The client estimate (summaryBaseline + responseTokenCount) thus counted the summary twice — inflating the gauge after compaction even on a single-call turn whenever the full snapshot is unavailable. priorRunOutputTokens now also counts summarization-tagged output (still excluding subagent/sequential, which recordCollectedUsage keeps out of the reported total), so the marker subtracts it. Updated unit + guard tests. * 🪙 fix: Refine Marker Subtraction for Summarization RunId and Abort Boundary Two Codex follow-ups on the marker-subtraction logic: - Subtract summarization output regardless of runId: the summarize detour is its own model-end call that may carry a distinct runId, but its output still lands in this response's tokenCount AND the snapshot baseline (summaryTokens). It is now counted unconditionally (still within the response's own usageEmitSink), while primaries keep the parallel-run runId filter. - Don't subtract primaries on the abort path: the job stores no snapshot/usage boundary, so a primary that completed AFTER the latest snapshot is NOT in the baseline; subtracting it would cancel real output and under-report. priorRun- OutputTokens gains an includePrimary flag (false for abort) — abort subtracts only the always-pre-snapshot summarization output. * 🪙 fix: Run-Scope Summary Subtraction and Stop Subtracting on Abort Two Codex follow-ups, resolved by reverting the round-4 detour: - Run-scope the summarization subtraction: the summarize detour inherits the graph run id (traceConfig spreads config.metadata.run_id), so its usage shares the answer snapshot's runId — it is NOT a distinct run. priorRunOutputTokens now filters summarization by runId like primaries, so a parallel sibling run's summary (different runId, in the sibling's baseline) is no longer subtracted from this branch's marker. Drops the includePrimary flag added last round. - Stop subtracting on the abort path: abort tokenCount is countTokens(text) (abortMiddleware) or absent (agents route) — it does not fold in summarization or earlier-call output the way recordCollectedUsage does, so the marker must keep the full baseline. buildAbortedResponseMetadata now subtracts nothing. |
||
|
|
4ee68d5240
|
💸 feat: Per-Agent Endpoint Token Config in Multi-Endpoint Billing (#13738)
* 💸 feat: Per-Agent Endpoint Token Config in Multi-Endpoint Billing
Price each collected/emitted usage item with the producing agent's resolved
endpoint token config, instead of the primary agent's for the whole graph.
Previously AgentClient.recordCollectedUsage and the subagent usage emitter used
a single this.options.endpointTokenConfig (the primary's) for every usage item.
A connected agent or subagent on a different custom endpoint that shares a model
id with an entry in the primary's tokenConfig was therefore mis-priced (a model
absent from it already fell back to the built-in rate map — no regression).
- Tag each usage with its producing agent: ModelEndHandler stamps
usage.agentId = agentContext.agentId; createSubagentUsageSink stamps the
child's subagentAgentId (UsageMetadata gains an optional agentId).
- buildAgentToolContext retains endpointTokenConfig so initialize.js can build
an agentId -> endpointTokenConfig map from agentToolContexts (the one map that
holds every agent, including pure subagents pruned from agentConfigs).
- AgentClient.resolveAgentEndpointTokenConfig(usage) looks up that map by
agentId, falling back to the primary config; used by both the billing path
(new optional resolveEndpointTokenConfig on recordCollectedUsage) and the
subagent cost emitter.
- recordCollectedUsage's resolver is optional and falls back to the batch
endpointTokenConfig, so the shared responses.js/openai.js call sites are
unchanged.
- Tests: two-endpoint graph with a colliding model id prices per-agent; resolver
nullish falls back to batch; subagent sink tags the child agent id.
* fix: Align emit-path cost with per-agent billing; honor known-agent built-in pricing
Addresses Codex review on the per-agent endpoint token config:
- Emit path (callbacks.js) now prices each on_token_usage event with the
producing agent's config (resolved via usageCost.resolveEndpointTokenConfig),
so streamed/persisted metadata.usage.cost matches the per-agent balance
transaction. The agentId tag is resolved server-side and stripped from the
emitted/persisted payload.
- Resolver (resolveAgentTokenConfig) now treats a known agent's config as
authoritative, including undefined → built-in pricing, so a known non-custom
agent in a custom-primary graph is no longer charged the primary's rates.
Only untagged/unknown usage falls back to the primary config.
- endpointTokenConfigByAgentId records every known agent (value may be
undefined) so the resolver distinguishes known-no-rates from unknown.
|
||
|
|
b03b2a0a29
|
💾 feat: Persist Context Breakdown & Branch/Total Usage Cost (#13734)
* 💾 feat: Persist Context Breakdown & Branch/Total Usage Cost Persist the granular context breakdown and per-response usage/cost on the response message metadata, and re-derive branch + total usage/cost from a per-message index so the popover survives reloads and is branch-aware live. - Add aggregateEmittedUsage + buildPersistedContextUsage helpers in packages/api; capture the latest visible snapshot and every emitted on_token_usage payload via contextUsageSink/usageEmitSink. - Attach metadata.contextUsage (Part A) and metadata.usage (Part B) on the agents response message in sendCompletion. - Carry per-message usage on the token index; add sumTotalUsage/setEntryUsage and branch-scoped usage on sumBranch. - Repurpose the session accumulator into a single in-flight pending holder; flush it into the index at finalize; hydrate breakdowns on load. - Render branch cost with a conditional all-branches total in the breakdown. * 🧹 chore: Remove orphaned com_ui_session_cost i18n key * 🩹 fix: Address Codex review — normalize usage server-side, fix reload deltas - Persist per-event-normalized display units in metadata.usage (TResponseUsage) so reloaded mixed-provider turns match the live session; client reads them directly instead of re-normalizing with a single stamped provider (P2). - Persist completedOutputTokens (final call output) on metadata.contextUsage so a reloaded multi-call turn adds the post-snapshot delta, not the full tokenCount the snapshot already counts (P2). - buildIndex preserves a prior entry's immutable usage when a rebuilt cache message lacks metadata.usage, so a mid-session rebuild (regenerate) keeps a sibling branch's flushed cost (fixes the e2e regenerate failure). - Track costKnown so turns saved with contextCost off don't render $0.00 when cost display is later enabled (P3). - Use an epsilon for the all-branches cost comparison to avoid a spurious total row from float summation order (P3). - Update unit/integration/e2e tests for the new shapes; regenerate e2e asserts the all-branches total after reload (deterministic via persisted metadata). * 🩹 fix: Address Codex round 2 — pending leak, cost coverage, reload delta - Clear the in-flight pending usage on terminal abort/error (resetLive), so a stopped generation's tokens no longer merge into the next response (P2). - costKnown now means COMPLETE coverage (ANDed): a branch mixing cost-bearing and cost-less turns is flagged incomplete and the cost row is hidden rather than rendering an under-reported total (P2). - Drop the tokenCount fallback for completedOutputTokens on reload: only the persisted post-snapshot delta is used, so a multi-call turn whose provider emitted no usage_metadata no longer double-counts earlier output (P2). - Update tokens.spec for AND coverage semantics + incomplete-cost case. * 🩹 fix: Address Codex round 3 — no-usage snapshots, total coverage, provider-less cache - Skip persisting metadata.contextUsage when the response emitted no primary usage event: without a known post-snapshot output the granular gauge would undercount the reply on reload, so fall back to the coarse per-message estimate instead (P2). - Gate the all-branches cost row on totalUsage.costKnown so an incomplete total (a sibling saved without cost) never renders an under-reported figure (P2). - aggregateEmittedUsage/finalCallOutputTokens now normalize per-event with the client's magnitude fallback (normalizeEventUnits) instead of billing splitUsage, so provider-less cached events match live on reload (P2). - Add backend test for the provider-less cached case. * 🩹 fix: Address Codex round 4 — abort attribution, complete cost coverage - aggregateEmittedUsage persists cost only when EVERY call was priced; a partial pricing failure now omits cost so the client treats coverage as unknown rather than reading an under-reported sum as authoritative (P2). - finalizeUsage flushes pending into the response entry only when events were folded this session (eventCount > 0), so a late/second resumable subscriber carrying persisted metadata.usage keeps it instead of being overwritten with an empty pending record (P2). - On user stop, attribute the in-flight pending usage to the partial response (new attributePending handler) instead of discarding it in resetLive — the stopped reply's billed tokens are kept and still can't leak into the next response; resetLive's discard remains for the error path (P2). * 🐛 fix: Persist branch cost across branch switches via sticky usage history Branch cost vanished on switching to a sibling branch (until a new turn) — the cost analog of the granularity bug. buildIndex rebuilds the token index from the messages cache; a sibling generated this session whose cache message lacks metadata.usage (and is transiently dropped from the cache during regenerate) lost its live-flushed usage, so sumBranch found none and the cost row hid. Fix: a sticky per-response usage map (conversationId → messageId → usage), written by setEntryUsage and never rebuilt from the cache — the usage counterpart of snapshotsByAnchorFamily for the breakdown. buildIndex/upsertEntries restore an entry's usage from it when the message carries none; cleared on convo switch and migrated with the index. Add unit coverage for the drop-then-readd regression and an e2e assertion that branch cost survives a branch switch. * 🐛 fix: Re-index on branch switch so branch cost survives the switch The sticky usage history alone didn't fix the reported branch-switch cost drop: on a branch switch no cache `updated` event fires, so the index subscriber never re-ran, and the post-regenerate rebuild was skipped while `isSubmitting` was still true — leaving the index stale and missing the now-viewed branch's response entirely (sticky can only restore entries present in a rebuild). Re-index from the messages cache on every tail change (created/finalize AND branch switch), not just while submitting. The cache holds the full message set at switch time, so the viewed branch's response is re-added and its usage restored from metadata.usage or the sticky history → sumBranch finds it and the branch cost renders. Verified locally: the branch-switch e2e now passes (the cost section shows both the branch row and the all-branches total). Also fixed that e2e assertion to target a single cost value (strict-mode safe). * 🩹 fix: Handle stopped-stream usage — reset pending + persist abort metadata Codex round (stop/abort edges): - Resumable explicit-stop (intentional SSE close) reset UI state but never cleared pendingUsageFamily, so usage folded before the stop leaked into the next response in the conversation. Discard pending on intentional close (resetLive); a resume re-folds via backfillUsage, so nothing is lost. - The abort save path (abortMiddleware) persisted the stopped response without metadata.usage/contextUsage, so its cost + breakdown vanished on reload. Rebuild both from the job's persisted tokenUsage (emitted payloads incl. cost) and contextUsage snapshot — parity with the normal sendCompletion path; breakdown gated on a primary usage event like buildResponseMetadata. Deferred (per scope decision): mid-stream branch-switch transiently shows the streaming branch's pending on the viewed sibling (cosmetic, until finalize). * 🩹 fix: Persist abort metadata on the real agents route + tighten snapshot gate Codex round (corrects last round's wrong-path fixes): - Stopped AGENTS responses are saved by routes/agents/index.js (/chat/abort), not abortMiddleware — so last round's metadata fix never ran for them. Moved the rollup/snapshot builder into packages/api as buildAbortedResponseMetadata (shared, unit-tested) and applied it in BOTH abort save paths, so a stopped agent reply keeps its cost + breakdown on reload. - Persist the breakdown only when the FINAL visible call emitted usage: track a per-response snapshot count and require primaryUsageCount >= snapshotCount. Previously any earlier primary usage event passed the gate, so a multi-call turn whose final call emitted no usage_metadata used an earlier call's output as completedOutputTokens (already counted by the latest snapshot) → reload over-reported. Now it falls back to the coarse estimate. Resumable stop pending-reset (prior round, 3cde6fe035) already flows through clearAllSubmissions → SSE close → the intentional-close handler's resetLive. Deferred per scope: mid-stream branch-switch pending attribution (tracked). * 🩹 fix: Abort breakdown over-count + resume re-fold after pending discard Codex round (on the re-applied abort/snapshot work): - buildAbortedResponseMetadata now persists ONLY the usage/cost rollup, not the context breakdown. The abort path can't tell whether the final call emitted usage (the job stores only the latest snapshot, not a count), so persisting the breakdown risked reusing an earlier call's output as completedOutputTokens (already in the snapshot) → reload over-count. Stopped/incomplete responses now fall back to the coarse gauge estimate, which is safe and apt. - resetLive now also forgets the conversation's folded usage-event identities (clearUsageFolded). Discarding pending on a terminal/intentional close left the folded keys set, so a later resume's backfillUsage saw the persisted events as duplicates and never rebuilt pending — leaving the response's usage missing until a full reload. Clearing them lets the resume re-fold. |
||
|
|
db7011d567
|
📊 feat: Real-Time Context Window & Token Usage Tracking (#13670)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
* 📊 feat: Real-Time Context Window & Token Usage Tracking
* 🧪 fix: Align Pricing Spec Dep Signatures with TxDeps
* 🩹 fix: Resolve Codex Findings for Context Usage Tracking
* 📊 feat: Granular Tool Token Breakdown with Deferred Splits
* 🧪 test: Cover Session Cost in Mock E2E and Scope Usage Selectors
* 🧪 test: Live Host-Pipeline Usage Verification (Env-Gated)
* 🧪 test: Local Real-Provider Multi-Turn E2E Harness
* 🪙 fix: Keep Tagged Usage Buckets Out of the Live Context Estimate
* 🩹 fix: Scoped Token-Config Fallback and Sequential Visibility for Usage Events
* 🩹 fix: Address Usage Review Findings — Cost Timing, Scoped Caches, Finalized Output
- carry the post-snapshot output estimate into the context snapshot at
finalize so the gauge keeps the last response after live resets
- accumulate per-rate billable units and price the session cost at
render, so usage events arriving before the token-config load still
count once it resolves
- pass user-scoped token-config cache keys through loadConfigModels
fetches and drop the controller's unscoped fallback to prevent serving
another user's resolved config
- tag emitted usage events with a per-run seq so resume dedupe never
drops a distinct call with an identical payload
- admit the static tokenConfig override in the custom endpoint schema so
it survives zod parsing into req.config
* 🩹 fix: Align Client Usage Accounting with Backend Cost Semantics
- classify cache tokens by provider (shared inputTokensIncludesCache from
data-provider, consumed by both the backend billing path and the client)
instead of a magnitude heuristic, so Anthropic/Bedrock turns where cache
is smaller than uncached input no longer under-bill input
- mirror resolveCompletionTokens on the client so Vertex-style hidden
thinking tokens are reflected in the Output row and session cost
- prefer endpoint pricing over adapter-provider pricing so a custom
endpoint can price a known model name without built-in rates shadowing it
- carry static cacheRead/cacheWrite overrides through the tokenConfig
schema and buildTokenConfigMap
* 🩹 fix: Honor Static Token Config in Billing; Tighten Usage Freshness
- initializeCustom now uses a static endpoint tokenConfig as the agent's
endpointTokenConfig (billing + balance checks), not just the advertised
UI config — previously the gauge showed admin rates while the agent
billed against built-in tables
- invalidate the token-config query alongside models on user-key add/
revoke so context windows and pricing refresh without a reload
- include maxContextTokens in ChatForm's stabilized conversation memo so
the gauge reflects a changed context-window setting immediately
- feed the live output estimate from the legacy content path (direct and
assistants streams), setting from cumulative part text rather than
accumulating deltas
* 🩹 fix: Resume Usage Dedup, Agent Pricing, and Partial Override Billing
- fold usage events idempotently by (runId, seq) so resume backfill no
longer resets the conversation totals — a mid-stream reconnect keeps the
usage of prompts already completed earlier in the session
- tap replayed pending message/reasoning/content events so output streamed
past the resume snapshot reaches the live estimate, not just the message
- resolve cost against the agent's backing endpoint (Agents conversations
report endpoint `agents` / provider `openAI`, neither of which keys a
custom endpoint's tokenConfig)
- getMultiplier/getCacheMultiplier fall back to the standard tables for
models absent from a partial endpointTokenConfig, so a partial static
override no longer bills non-listed models at defaultRate while the UI
shows the correct pattern rate
* 🩹 fix: Repaired Output in Gauge, Cache-Rate Keys, Config Gate, Usage Cleanup
- live/completed gauge counts the repaired completion (normalized output),
so under-reporting providers don't drop the response from used context
- translate static tokenConfig cacheWrite/cacheRead onto the write/read
keys getCacheMultiplier reads, so cache tokens bill at the configured
rate instead of the prompt-rate fallback
- clear the token index and usage atoms when leaving a conversation, so
visited histories don't accumulate in memory for the tab's lifetime
- wait for startupConfig before mounting the gauge, so a deployment with
contextUsage disabled never briefly mounts it or fires the token-config
query on first load
* 🩹 fix: Move Token-Config Resolution to TS; Key Live Usage by Created Convo
- extract the token-config resolution (override gathering + cache lookup +
buildTokenConfigMap) into resolveTokenConfigMap in packages/api, leaving
the /api controller a thin request-scoped wrapper (CLAUDE.md TS rule)
- getConvoKey prefers the user message's real conversationId once the
`created` event stamps it, so a new chat's first-response live gauge and
totals land under the id TokenUsage subscribes to instead of NEW_CONVO
* 🩹 fix: Clear Stale Redis Job Usage; Live-Tap Legacy Streams; Share Fetched Config
- DEL the Redis job hash before re-creating it so a reused streamId can't
inherit a prior run's contextUsage/tokenUsage and backfill stale usage
- tap the legacy {message,text} stream branch (non-agent OpenAI/Anthropic
streams) into the live estimate, not just the content path
- copy a deduped fetch's token config to every sibling endpoint sharing the
baseURL/key/headers, so /token-config resolves each by its own name
* ⏪ revert: Don't DEL Redis job hash in createJob (breaks cross-replica resume)
createJob is an idempotent join — a second replica calls it for the same
streamId to share an in-flight stream's state. DELeting the hash wiped the
prior replica's persisted created/usage state, so a joining replica missed
the created event (GenerationJobManager cross-replica integration test).
Reverts the F1 change from
|