From e2f5772961d240f04ecec4f8c8dd1dfa459df32f Mon Sep 17 00:00:00 2001 From: Marco Beretta <81851188+berry-13@users.noreply.github.com> Date: Fri, 11 Sep 2026 21:53:29 +0200 Subject: [PATCH] =?UTF-8?q?=F0=9F=A7=AE=20feat:=20Surface=20Retained=20Too?= =?UTF-8?q?l-Message=20Context=20Usage=20(#15796)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat: surface retained tool-message context usage * fix: correct post-snapshot context accounting and lockfile registry Adjudicated the review findings on the retained tool-message context work: - package-lock.json resolved four entries (@librechat/agents, call-bind-apply-helpers, es-object-atoms, get-intrinsic) from a private mirror, breaking `npm ci` outside that network. Restored the public registry hosts; the integrity hashes already match the public tarballs. - `finalizeUsage` migrated every usage atom except `subagentUsageFamily` before dropping the temporary `new` key, so a first-turn subagent total vanished when the conversation id was assigned. - The runway projection divided the pre-invoke `remainingContextTokens` by per-call growth, ignoring the output generated after that snapshot, and the compaction hint subtracted a post-call exchange from a pre-call message total. Both now apply the post-snapshot output, so each side is measured at the same point in time. - The estimate path rendered its tool-call share as a peer row although it is a subset of Input/Output/Estimated, so the visible rows summed past `usedTokens`. It is now an indented subtotal; the counted rows stay provider-accurate. - The Totals block presented conversation-wide session subagent usage beside branch-scoped rows; it is now labeled all-branches. - Rewrote the truncated `rgb-series-8` rationale as one comment on the series block. * fix: keep subset accounting whole across cache, legacy snapshots, and custom themes Adjudicated the second review round on the retained tool-message context work. One cause per finding, each measured on the same basis as what it is compared to: - The all-branches subagent subtotal summed input+output only, so a cached subagent call was under-reported and a fully cached call with no output disappeared from the row while the Cache read/write rows above still counted its traffic. It now sums every normalized bucket. - `collectAnchorSeries` read an absent `remainingContextTokens` as zero and recorded that snapshot as having spent its entire budget, which distorted the growth delta feeding the runway projection. A snapshot saved in the older shape now derives its used total from the breakdown, exactly as the render path's `baseUsed` does, and one that carries neither basis is skipped. Each reading is tagged with its basis and the projection compares only readings of the same basis instead of reporting the difference between two measurements. - `resolveTheme` filled an omitted `rgb-series-8` from the bundled base, so a stored or environment theme that paints its own seven-slot scale got LibreChat's indigo on surfaces it never checked: 2.41:1 against that theme's meter track, under the 3:1 mark floor the scale claims. An owned scale now derives the slot from the theme's own neutral text โ€” 11.5:1 on the same reference theme โ€” and naming `rgb-series-8` still opts out. - Ordered the `--series-*` declarations in the light block by slot. * fix: keep cached shares out of the peer rows and track resolved text for slot 8 Third round on the retained tool-message context work, correcting the two variants of invariants this PR already fixed elsewhere rather than patching them locally: - The snapshot path rendered the reconciling call's Cached and Cache write shares as peer rows with window percentages, although reconciliation already counted those tokens inside the segments above. A fully cached prompt showed its tokens twice and the visible rows summed past the meter. They now render as an indented subtotal, the same treatment the estimate path's tool-call share received. The existing test claimed this invariant in its name but only asserted the rows exist, so it now asserts they are not peers. - The `rgb-series-8` fallback required the theme to name its own text, so a theme that repaints the scale and its surfaces while inheriting LibreChat's text colours still received the bundled indigo. It now derives from the RESOLVED secondary text, so the stop tracks whatever the theme reads its body copy against whether it names one or inherits ours. * fix: settle the subagent share with its run and measure both reclaim sides alike Fourth round on the retained tool-message context work: - The subagent share was committed on arrival while the usage it is a subset of stayed pending, so a run that ended with no salvageable response left its tokens in the Totals after `resetLive` discarded them from every rollup, and a resume re-folded the same events into the cumulative figure. It now accumulates in `pendingSubagentUsageFamily` beside the pending usage and settles with it: committed by `flushPendingInto` when the response keeps its usage, dropped by `resetLive` otherwise, migrated with the conversation id. The view merges committed and in-flight shares, so the live row is unchanged. - `latestExchangeTokens` subtracted a summarizing turn's whole response, including the summarization completion the backend folds into `tokenCount` while the snapshot holds it in `breakdown.summaryTokens` rather than `messageTokens`. The reclaim hint was understated by the summary and could reach zero after a long one; the summary block's size is now excluded from the tail's contribution when that turn compacted. - `sumBranch` exposed the tail's char estimate and tool share even when the tail carried a `tokenCount`, so a resumed partial response stayed in the message total while its tool traffic was dropped from the Tool calls split. Both tail figures are now zero for a counted tail, which also stops the estimate path subtracting an estimate that was never summed. * chore: drop the lockfile churn the base branch already carries The SDK bump this PR set out to make landed on dev as 3.8.5, so the branch no longer changes any package.json and its lockfile edits (the 3.8.4 entry plus the transitive call-bind-apply-helpers/es-object-atoms/get-intrinsic bumps that install pulled in) are churn against the base. * fix: give the eighth series slot a value every bundled theme can carry The base branch grew high-contrast palettes that must declare every registry token, and a test that a series slot can carry `text-on-status` at the 3:1 mark floor, because a slot also fills a badge chip under that glyph. Slot 8 failed both: the high-contrast palettes could not name a token that did not exist when they were written, and the dark indigo #8c98e6 carried white at 2.71:1. Dark slot 8 is muted to #8082be, which reads 3.66:1 under the status label and still clears 3.66:1 on every series surface. The high-contrast palettes take their own neutral text as slot 8 โ€” the same stop `resolveTheme` derives for a third-party theme that paints its own scale โ€” because seven saturated hues is all either canvas affords at their stated CVD separation; it is distinguished by lightness at 21:1 on the canvas instead of by hue. * test: exercise the context split through the mock harness Nine scenarios of the acceptance contract, one tagged Playwright test each, driving the real application: a tool round-trip through the e2e MCP server for the Messages/Tool calls split, its keyboard disclosure of per-tool counts, the peer rows staying inside the meter and surviving a reload; a real subagent run for the all-branches total; seeded conversations for the legacy snapshot whose used figure comes from its breakdown and for the estimate path's nested tool share; and a stored theme that owns the series scale for the eighth slot's fallback and its opt-out. * test: keep the tool and subagent scenarios on the viewports that expose them * ๐Ÿ“Œ feat: Unified Pinned Section With Sidebar Drag Interactions (#15057) * feat: unified pinned section with drag interactions and sidebar polish - Pinned chats and favorite models/agents/specs render as one interleaved, reorderable list persisted via users.pinnedOrder - Drag conversations onto project rows to file them, onto Chats to unfile, and onto the Pinned section to pin (empty section materializes as a drop zone during a conversation drag) - Row-level unpin button on pinned chats matching the favorites' button, and the row action menu shares the same hover/focus treatment - Agent Marketplace entry moved under the New Chat button in the side rail - Bookmarks trigger sized to the search bar, search focus no longer swaps its border color, tag filtering shows a loading state, and opening a project shows row-shaped skeletons * fix: add pinnedOrder to IUser type and sort imports * fix: pinned drag threshold, unpin focus target, and marketplace link * fix: surface failed pinned reorders and skip no-op order writes * fix: address review findings on unified pinned section Move the pinned-order endpoints into packages/api as TypeScript handlers and route their URLs through the data-provider endpoint registry. Raise the entry and key caps so a valid model favorite and an unbounded pin count can both be persisted. Keep the marketplace entry reachable from the mobile drawer, which is the only sidebar surface on small screens. Merge a reorder into the stored order while a bookmark filter hides part of the list, instead of dropping the hidden keys. Add Alt+Arrow keyboard reordering with a live-region announcement, restore the focus handoff when a favorite is removed, and hold the local arrangement until the stored order catches up. Serialize overlapping order writes and discard the optimistic value when the first one fails. Derive the tag-filter loading state from isPreviousData rather than a flag that could stick on. * fix: release drag while renaming and dedupe repeated favorite keys A draggable ancestor swallows drag-select inside the rename input, so both the row's own drag source and the pinned wrapper's release while a title is being edited. The favorites payload validator accepts a repeated entry, and two rows keyed the same would collide in React, point every drag at the first copy, and fail the order endpoint's uniqueness check. Build the pinned list keyed-unique. * fix: address PR review bot findings Make the model favorite key injective. Joining endpoint and model on '::' collided for components containing the delimiter, so two distinct favorites keyed the same and the dedupe hid one; the endpoint is now length-prefixed. Replace the pinned-order entry cap with a total size guard. Pinning has no membership cap and the sidebar query drains every cursor, so any count limit rejects a legitimate list rather than bounding anything; what needs bounding is the user document. Let only the newest order write touch the cache. An earlier write failing rolled back over a newer arrangement that was still in flight, leaving the UI on a stale order while the server held the current one. A confirmed write now takes the server's answer, and a failed newest write discards the value that was never accepted. Persist the favorites array only once the order write succeeds, so a rejected write cannot leave the two stores disagreeing with the rolled-back display. * fix: address second round of PR review bot findings Read favorites membership at write time rather than capturing it at drag end. A favorite added or removed while the order request was in flight was undone by writing the older list back; only the relative order now comes from the drag. Move the pinned-order queue to module scope. The sidebar unmounts whole sections while a search is active, so a component-local ref let a remounted hook start a second queue alongside a request that was still running. Serialize project assignment per conversation. Every drop target owns its own mutation instance and the write is an unconditional update, so two quick drops let whichever request reached the database last decide the project. Read the pinned-order request body null-safely: destructuring a nullish body threw past the handler into Express's generic error path instead of returning the intended 400. * fix: address third round of PR review bot findings Drop the secondary favorites write. No consumer of the favorites array reads its order, so mirroring the arrangement there bought nothing while racing the membership mutations that share that whole-array endpoint; pinnedOrder is the single source of ordering. Always merge the committed order into the stored one. The visible list is not always the whole list, since a bookmark filter hides rows and the pinned query publishes partial results while draining its cursor, and replacing wholesale dropped every key it could not see. Release the local arrangement when the write settles instead of when the stored order matches it key for key. A pin arriving mid-request made that equality unreachable, stranding the snapshot and hiding the new row until remount. Abandon a queued pinned-order write if the signed-in user changed, so a deferred request cannot write one account's order into another's document. * fix: address fourth round of PR review bot findings Guard the pinned-order completion callbacks on the signed-in account. The pre-send check only covers a write that has not left yet; a response arriving after a session change would publish one account's order into the next one's cache, which this query never refetches on its own. Commit a reorder only when the pinned rows handled the drop. Dragging a pin onto a project row or into Chats sweeps the pointer across sibling rows, whose hover handlers shift the order, so a filing action was also saving a move the user never asked for. * fix: address fifth round of PR review bot findings Deselect pinnedOrder at schema level. It is display-only and allowed to grow large, but every authentication request loads the user document, so it was adding its whole payload to paths that never read it. Refuse to reorder until the stored order has loaded. Merging against an absent order and then cancelling its fetch posted only the visible keys and discarded saved positions for good. Prune keys only once the visible list is known to be the whole list. Merging unconditionally grew the array until the size guard rejected every write, so completeness now gates compaction rather than the filter flag alone. Track every renaming row, not just the most recent. RenameForm has no blur cancellation, so a second form can open while the first is still mounted and the single key reconnected the first row's drag source under its open input. Judge drop eligibility by the queued destination. A drag item carries the project its row had when the drag began, so re-dropping a chat back where it started while an assignment was queued read as a no-op. Take mutation ownership from the user atom AuthContext fills on login rather than the user query, which is still empty in that window, and never treat an unknown owner as a match. Restore the pinned section's 30vh cap. Matching the Projects section's 42vh put 84vh of non-shrinking content above Chats and starved it on short viewports. * fix: address sixth round of PR review bot findings Gate reordering and pruning on successful fetches rather than attempted ones. A failed pinned-order GET still reports as fetched, and a failed later page of the pinned drain still publishes partial data and stops fetching, so both gates were opening on exactly the states they were meant to exclude. Scope the post-unpin focus handoff to this section by its own marker. The whole conversations pane is a labelled region too, and the project lists inside it render the same row, where unpinning only clears the flag and the row stays put, so focus was being moved off a row that never went away. Clear the account tracker when the mutation hook unmounts. The sidebar drops that section during a search, leaving a stale non-null id that also shadowed the cache fallback, so a deferred write could pass its owner check after a session change. * fix: gate pruning on a delivered favorites list A favorites fetch that exhausted its retries leaves isLoading false with the list empty, so an otherwise complete pinned load enabled pruning and dropped every favorite key from the stored order. useFavorites now exposes the query's success state and the pruning gate requires it, matching the pinned-order and pinned-membership sources. * fix: address eighth round of PR review bot findings Identify queued assignments by token rather than by destination. Comparing destination values cannot tell two drops onto the same project apart, so a B, A, B run let the first B's cleanup discard an entry the still-queued later drops owned, and the next drop was then judged against a stale project. Report the rename ending when a row unmounts. A pinned row removed mid-rename, for instance by unpinning the same chat from the project list that also renders it, left its key in the renaming set and came back permanently undraggable. * fix: address ninth round of PR review bot findings Stand the keyboard reorder down wherever the pointer drag does. Alt/Option plus an arrow moves the caret word-wise inside a text field, so the shortcut was moving and saving the whole row out from under an open rename input. Hold a settled assignment destination until a drag item shows the new project. The mutation's cache invalidations are not awaited, so a row still reported its old project for a moment after the write succeeded and a quick drag back to where it started read as a no-op. * fix: address tenth round of PR review bot findings Serialize project assignment inside the mutation instead of at one caller. The drag targets, the row menu and the project dialog each hold their own instance, so a queued drop could still overwrite a newer menu write; moving the queue down covers every surface and lets the drag helper drop its own. Tie the live snapshot's release to the arrangement that created it. An older write settling mid-drag cleared the newer arrangement, snapping the rows back to a stale order while the drag still held its own, so releasing saved an order the user could no longer see. * fix: address eleventh round of PR review bot findings Move assignment destination tracking next to the queue that owns it. Keeping it in the drag helper meant a menu or dialog move left the drag helper still reporting its own older destination, with no row left to observe that would ever clear it, so a later move back was rejected for good. Resolve a conversation's project from the write the mutation confirmed rather than from whichever list rendered the row. The same chat appears in the Pinned section and its expanded project, and those caches refresh independently, so observing one of them was never a sound way to decide the other had caught up. The conversation cache is written synchronously on success, which makes the lazy self-clearing and its settled flag unnecessary. * fix: cancel an in-flight conversation refetch before confirming an assignment Navigation invalidates the exact conversation query, so a refetch that read the old project can be in flight when an assignment resolves and land afterwards, reverting the move in the cache that now answers where a conversation lives. * fix: keep queued pinned-order writes attributable across an unmount Refuse a queued write only when a different account is positively established. Clearing the identity on unmount stopped stale attributions but stranded work that belonged to the same account, since the sidebar drops the section that owns this hook whenever a search is active, and it denied that work its rollback as well. Correct the cache from the request rather than from onError. The observer callbacks do not run once that section unmounts, so an order the server never accepted could survive on a query that refetches on nothing. * fix: publish the session identity from the user atom Queued pinned-order writes were attributed by a tracker that only the pinned section's hook updated, and the sidebar unmounts that section during a search. A retained value went stale across a sign-in, and login clears the user query before the next account's fetch resolves, so a write queued by the previous account could pass its own check and post on the new account's credentials. A Recoil atom effect publishes the identity instead: it runs wherever the atom is set, including sign-out, so attribution comes from the state itself rather than from a component that happens to be mounted. That makes a signed-out session foreign to every owner and lets the check be a strict comparison again, rather than a rule that had to guess when identity was merely unreadable. * fix: discard reorders that ended on a refused external target A drop the pinned rows did not handle usually means a reorder, but a project row or the Chats section refusing one reports the same, so dragging a chat onto the project it already belongs to was saving the incidental shift its pointer caused on the way out. The rows and those targets now report their hovers, so the last one under the pointer settles it, which also allows for a drag that strays over a project and returns to reorder after all. * fix: attribute queued writes to the credentials they will travel with Publishing the account from React state, whether an effect or a Recoil atom effect, always lagged the credential change: setUserContext installs the new Authorization header in one synchronous call, and a write queued by the previous account and sent in that gap read its own owner back and went out as the new one. A test against the atom effect showed onSet is deferred too, so the atom was no better than the effect it replaced. The header is the thing that decides who the server sees, so ownership is read from it. The id claim is compared rather than the token, since a silent refresh rotates the token for the same account and discarding that work would strand something the user is still entitled to. Leaves AuthContext and the user atom untouched. * fix: attribute a reorder to the account that started it onMutate awaits cancelQueries before the request is built, and the mutation function was reading the session again afterwards, so credentials turning over inside that gap sent one account's order as the next. The owner is now recorded in onMutate's first synchronous statement and carried on the order itself, and the optimistic write is skipped when the session moved on, since that cache is shared and showing an order to the wrong account is the same mistake as sending it. Record a pinned hover before the guards rather than after. Hovering the dragged row is a no-op for ordering but still means the pointer is inside the list, so a drag that wandered onto a project and returned to its own position was being discarded as a filing action. * fix: address a further round of PR review bot findings Mark a hover anywhere in the pinned section as internal. The header and the padding around the rows are part of the list, so a drag that wandered onto a project and came back to rest on one of them ran no row hover and was discarded as a filing action. Hold off pruning while a favorites write is unsettled. The earlier GET's success stands through an optimistic removal, so pruning then dropped the ordering key of a write that had not landed, and a failed one came back without its saved position. Declare the reorder shortcut on the elements that take focus. The section's written hint is skipped by anyone tabbing straight to a row, leaving the only non-pointer way to reorder undiscoverable. * fix: reach a real New Chat control, and let the shortcut prop through the memo The unpin fallback looked for a test id no component carries, so once the last pinned row unmounted focus fell to the document rather than to New Chat. It now tries the controls that exist, in sidebar order. The test that covered this had fabricated the missing id, so it proved nothing. Convo is memoized on conversation fields and isGenerating alone, so the shortcut appearing once the pinned order loaded never reached the element that announces it. The comparator now accounts for it. * fix: identify OpenID sessions, and see favorites writes from every hook With OpenID token reuse the bearer is the provider's own id_token, which names its subject as sub under an issuer and carries no LibreChat id, so every such session resolved to nobody and two different accounts compared equal. That is the case the ownership guard exists for, so it read as safe while being exactly wrong. Sessions are identified by whatever the credential offers now, and an owner that cannot be identified is foreign to everything including itself. Count favorites writes across the query client rather than from one observer. A favorite row runs its own useFavorites, so a removal started there was invisible to the pinned section and its ordering key could still be pruned. * fix: close the assignment cache gap, and let a failed drain be retried The assignment released its pending destination as soon as the request settled, while the cache write waited on an awaited cancellation in onSuccess. A drag started in that gap saw neither, fell back to the stale row and refused a valid drop. The write now happens in the request, before the entry is released, which also survives the owning component unmounting. Also clear the pagination guard when the Chats section collapses. A drain that exhausted its retries leaves the conversations array unchanged, so the guard barred every later attempt and the remaining chats stayed unreachable for the session. Reopening the section now tries once more. That guard came from #14860 rather than this branch, so it can be split out if preferred. * fix: compose the marketplace link, and keep focus on a surviving row The marketplace control was a raw link with restated appearance classes, and in restating them it left out the focus ring entirely, so keyboard focus on it was invisible. It composes through the shared button now, which carries the ring, the hover fill and the theme timing. Unpinning from a project list leaves the row in place but removes the pin badge that had focus, and the pinned-section successor search deliberately returns nothing there, so focus fell to the document. It moves to the row's own link. Reconcile the pinned order on window focus and reconnect. It is one shared per-user record, so a second tab would otherwise show a stale arrangement and its next reorder would post a full array undoing the other tab's. * fix: keep a rename from restoring a conversation's other fields A rename request carries only a title, but its response is the whole conversation as the server saw it, and writing all of that back restores the pre-request value of every other field. An assignment confirmed while the rename was in flight was therefore reverted in the cache that now answers which project a conversation belongs to, and a drag back to the original project read as a no-op. Only the title and its timestamp are taken from the response. Cancelling queries cannot help here, since this is another mutation's write rather than a fetch. * fix: compose the unpin badge through the shared row-action button The badge restated a reusable icon control's sizing, radius, hover and focus classes. The client package already had a row-action variant and an icon-xs size that match it exactly, so it uses those instead of its own copy. * fix: reconcile before reordering, and scope assignments to their session A reorder stayed enabled while the order's focus refetch was in flight, so returning to a tab could cancel that refetch and post the arrangement it was about to replace. Pruning had the matching gap: the pinned list keeps its data fresh for five minutes, so a tab whose order had reconciled could prune against membership that had not, dropping the position of a conversation another tab pinned. Both now wait for data that is settled and not stale. Assignments carry the account that started them, like the order writes already did. Conversation ids are per-user, so a queued one travelling with the next account's credentials would act on whatever that id names over there, and its answer would describe one account's conversation to another in a cache that is not scoped by user. The row's overflow trigger derives from the shared row-action variant too, so the two actions in a row stop keeping separate copies of the same recipe. * fix: reject foreign assignment results, and share the section header An assignment settling after a session change returned normally, so the success path still ran: the caller reported success and the response could be applied to the next account's conversation, since ids repeat across accounts. It rejects now, so nothing downstream treats it as a result. The collapsible section header existed as three identical copies, Chats, Projects and Pinned, and this branch added the third. It moves into the shared button as a section-header variant and all three compose from it. * fix: let favorites membership reconcile between tabs The favorites query disabled focus, reconnect and mount refetches, so a second tab kept its old list indefinitely. Its pruning gate waits for a fetch, and none was ever going to happen, so a reorder there could delete the saved position of a favorite the first tab had added. It reconciles on focus and reconnect now, like the order it is arranged alongside. * fix: hold reconciling refetches off while a write is in flight Enabling focus and reconnect refetches on the order and favorites queries let one start after onMutate's single cancellation, read the pre-write state and land after the write, putting the old value back. Both now decline those refetches while a write for that resource is outstanding, keeping the cross-tab reconciliation itself. The check asks the query client at the moment the event fires rather than reading useIsMutating: that hook only reaches the query's options on the next render, which is a race against the very event being gated. A test that dispatches focus mid-write fails without it. * fix: release abandoned assignments and gate pruning on current membership An assignment rejected before entering the try block kept its pending entry forever, so a later session read a destination that was never sent. The ownership check now sits inside the block that releases it, and it runs again after the conversation query cancellation resolves, since the session can turn over while that await is pending. Pruning the pinned order now requires membership at least as current as the order it is pruned against. The pinned query holds its data for five minutes while the order has no such window, so focusing a tab reconciled the order alone and the next reorder dropped the key of a conversation another tab had pinned in between. The assignment spec resets its mocks between tests and gives each test its own conversation id, so one test's unconsumed mock and leftover queue entries no longer decide the next test's result. * fix: address the sixth round of review findings on the pinned section The section header recipe asked for the shared button variant without a size, so CVA emitted the default `h-10 px-4` after it and the merge kept it, putting a 40px control in a 32px header row. The variant opts out of default sizing through a compound now, since a heading is sized by its own text. Restore the search field's focus border. Removing it left keyboard focus signalled only by a background change that hover already uses, so the field had no distinguishable focus state at all. An assignment can reach the database before a concurrent rename and still answer after it, carrying a conversation that predates the rename. Only the project field is merged into the cached conversation and published to the active one now, matching what the rename handler already does. An order refetch that starts mid drag no longer lets the drop post an array built from the order that refetch is about to replace: the arrangement is held and written once the fresh order lands. Reorder writes superseded by a newer arrangement are skipped rather than sent, so holding the shortcut costs two writes instead of one per intermediate position. The order and favorites queries reconcile on mount as well, since a sidebar search unmounts the section and a query with no observer never sees the focus event. * fix: preserve pinned order until favorites membership catches up * fix: reconcile pinned membership and preserve auth and focus state * fix: preserve reconciled keys during active pinned arrangements * style: sort pinned order regression imports * fix: reconcile favorite rollbacks before pinned-order pruning * ๐ŸงŠ fix: Withhold Rerun Controls on Compaction Turns (#15797) * ๐ŸงŠ fix: Withhold Rerun Controls on Compaction Turns A compaction turn's response parents onto the leaf it summarized, not onto a user message, so the hover Edit's Rerun replayed an assistant message in the user slot: the request minted a user message under the existing assistant leaf's id and ran the turn with empty text, which fails. Editing offered nothing either, since SUMMARY is not an editable part type. Edit, Regenerate and Continue are now withheld on a compaction turn, and both rerun paths refuse a parent that is not user-created, matching the guard `useChatFunctions.regenerate` already applied. * ๐Ÿงญ fix: Gate Rerun Controls on the Replayed Parent Turn Withholding Edit, Regenerate and Continue on summary-only content also hit an ordinary turn that auto-summarized and was cancelled before its first answer token: that response is parented to the user's message and is exactly the turn a rerun is for. The row now resolves the message it would replay and withholds the rerun shapes only when a model turn hangs off another model turn, which is the manual compaction shape. An unresolvable parent withholds nothing. `isCompactionTurn` is gone with it, and `isCompactedLeaf` keeps its original body; its docstring now says per part what it always measured per part. * ๐Ÿงฎ fix: Identify a Compaction Turn by Content and Parent Gating the rerun controls on parent authorship alone withheld Edit, Regenerate and Continue from any model turn hanging off another model turn. An imported conversation chains messages sequentially, so a skipped empty human message leaves an ordinary reply parented to the previous reply; its stored content is editable and its controls must stay. It also made every model row resolve its parent out of the flat message list: 100 full-thread scans while mounting a 200-message conversation. A manual compaction is now both halves of its shape โ€” summary-only content and a parent that is not the user's turn โ€” so only such a row consults the thread. `isSummaryOnlyContent` is the content half, in data-provider next to `isCompactedLeaf`. * ๐Ÿงท test: Pin the Rerun Refusal on a Chained Model Turn A rerun replays its parent as the user turn, so a model turn parented onto another model turn must submit nothing rather than mint a user message under an existing response's id. Removing the guard makes the case call `ask` once. * ๐Ÿงฑ fix: Gate the Editor on Content and the Rerun on the Parent Identifying a compaction turn by content shape missed the ones the server persists with an error part (a skipped or failed compaction keeps its explanation instead of a summary), which left Edit, Regenerate and Continue on a turn where every rerun shape is inert. The two controls answer two different questions now. A rerun replays the message's parent as the turn's user message, so Regenerate and Continue go whenever a model turn hangs off a model turn. The editor goes only when that turn also has no part to type in: an imported chain's reply is saved directly and stays editable, while a compaction โ€” summary or error part โ€” would open an empty editor with one inert Rerun. `findMessageById` indexes the messages array once per identity, so resolving the parent no longer scans the thread per row: 100 full-array walks while mounting a 200-message conversation become one. * ๐Ÿท๏ธ fix: Honor the Server's Compaction Marker on Any Leaf Compact runs on whatever leaf the branch ends with, so a compaction can hang off a user message rather than off the answer it summarized. Inferring the turn from its position alone left the rerun controls on that one: replaying the parent would answer the user's earlier message again instead of redoing the compaction. `isUserInitiatedCompaction` reads the marker the server already stamps โ€” `markCompactionSummary` sets `initiatedBy: 'user'`, and nothing else writes it, so an automatic summary detour keeps its controls. A marked turn is excluded from the rerun shapes wherever it hangs. * ๐ŸŽจ fix: Ask the Editor What It Can Edit The hover row had its own idea of an editable part and counted artifact-bearing text as one, while `EditContentParts` keeps an artifact in its read-only renderer. A chained model turn whose only content is an artifact therefore still offered Edit, and opening it gave a read-only artifact, a disabled Save and a Rerun the guards refuse. `editableParts.ts` now owns that rule for both: `isEditablePart` is the editor's own filter, `hasEditablePart` asks it per message. The row resolves it lazily โ€” the artifact check parses markdown, and only a turn with nothing to replay needs the answer โ€” so a streaming row never pays for it. * ๐Ÿšซ fix: Withhold the Editor's Rerun Instead of Refusing It Both editors resolved the parent inside the handler and returned when it was not a user turn, so a chained model turn โ€” an imported thread with no user message between two replies โ€” kept an enabled Rerun that did nothing, with Ctrl/Cmd+Enter bound to it. Each editor resolves that parent at render and withholds the action instead: the button is gone, the shortcut is inert and `aria-keyshortcuts` no longer advertises it, while Save keeps writing the edit. An unresolvable parent keeps the action, matching the hover row. * ๐Ÿงน style: Sort the New Imports in Both Editors CI's import-sort check failed on the two files the previous commit touched. * ๐Ÿ’ฌ fix: Report Unsaved Changes When No Rerun Is Offered A save-only editor still warned about the rerun. An answer's draft is discarded by the rerun, and multi-part edits must be saved before one, but neither is true where the action is withheld โ€” a model turn chained onto another model turn โ€” so the status slot claimed a precondition for something the footer does not offer. Both statuses are gated on the same `canRerun` the button is: the plain-text editor reports unsaved changes, and `rerunRequiresSave` cannot hold without a rerun to require it. * ๐Ÿงช test: Exercise the Compaction Controls in the Browser Eight mock-harness scenarios drive the real row and the real editors: a finished, failed, error-part and user-parented compaction withhold Edit, Regenerate and Continue while Copy stays; a turn that only auto-summarized keeps all three; a reply chained onto a reply keeps a save-only editor whose status reports unsaved changes; an artifact-only chained reply loses the editor; and the context indicator still offers Compact after a failed compaction. `SeedMessage` carries `content`, `finish_reason` and `model` so a branch can be seeded in the shapes the mock model cannot produce โ€” a persisted summary, an error part, an artifact. * ๐Ÿฉป feat: improve agent filter controls for insights (#15807) * feat(insights): improve agent filter controls * feat(insights): add search and fixed agent selection controls * fix(insights): show search before all agents control * test(e2e): allow steer consumption before arm response * ๐Ÿ”  fix: Name Artifact Downloads After Their Titles (#15836) * fix: Use artifact titles for download filenames * fix: Match artifact filenames to downloaded content * fix: Preserve raw artifact extensions and GFM headings * fix: Download complete artifact files and retain filename provenance * fix: Distinguish cached artifact exports from originals * fix: Name unnamed artifact originals by their media type * fix: Recognize legacy CSV MIME in artifact filenames * fix: preserve context insights across reloads and configuration changes * fix: retain primary call details when resuming context snapshots * fix: preserve resumed output and retained tool-result estimates * fix: keep resumed output compatible with older clients --------- Co-authored-by: Ravi Kumar L Co-authored-by: Danny Avila --- .../agents/__tests__/toolContext.spec.js | 176 ++++++++ .../__tests__/usageEvents.integration.spec.js | 8 + .../Chat/Input/TokenUsage/Breakdown.spec.tsx | 263 +++++++++++- .../Chat/Input/TokenUsage/Breakdown.tsx | 400 ++++++++++++++--- .../Chat/Input/TokenUsage/index.tsx | 4 +- .../Chat/__tests__/useTokenUsage.spec.tsx | 283 ++++++++++++ client/src/hooks/Chat/useTokenUsage.ts | 204 +++++++-- .../SSE/__tests__/useUsageHandler.spec.tsx | 131 +++++- client/src/hooks/SSE/useUsageHandler.ts | 78 +++- client/src/locales/en/translation.json | 12 + client/src/store/usage.ts | 26 ++ client/src/style.css | 2 + client/src/utils/tokens.spec.ts | 257 ++++++++++- client/src/utils/tokens.ts | 402 +++++++++++++++--- .../scenarios/persisted-context-usage.spec.ts | 261 ++++++++++++ .../mock/scenarios/series-eight-theme.spec.ts | 134 ++++++ .../scenarios/subagent-context-total.spec.ts | 118 +++++ .../mock/scenarios/tool-context-split.spec.ts | 236 ++++++++++ packages/api/src/agents/usage.spec.ts | 120 +++++- packages/api/src/agents/usage.ts | 94 +++- .../api/src/stream/GenerationJobManager.ts | 12 +- .../src/stream/__tests__/usageResume.spec.ts | 95 ++++- .../client/src/components/SegmentedMeter.tsx | 3 + packages/client/src/theme/registry.spec.ts | 83 ++++ packages/client/src/theme/registry.ts | 27 +- .../client/src/theme/semanticTokens.spec.ts | 2 +- packages/client/src/theme/themes/dark.ts | 9 +- packages/client/src/theme/themes/default.ts | 5 +- .../client/src/theme/themes/highContrast.ts | 19 +- packages/client/src/theme/types/index.ts | 3 + .../src/theme/utils/createTailwindColors.js | 1 + packages/data-provider/src/types/runs.spec.ts | 162 ++++++- packages/data-provider/src/types/runs.ts | 176 +++++++- .../data-schemas/src/methods/message.spec.ts | 79 ++++ 34 files changed, 3639 insertions(+), 246 deletions(-) create mode 100644 api/server/controllers/agents/__tests__/toolContext.spec.js create mode 100644 client/src/hooks/Chat/__tests__/useTokenUsage.spec.tsx create mode 100644 e2e/specs/mock/scenarios/persisted-context-usage.spec.ts create mode 100644 e2e/specs/mock/scenarios/series-eight-theme.spec.ts create mode 100644 e2e/specs/mock/scenarios/subagent-context-total.spec.ts create mode 100644 e2e/specs/mock/scenarios/tool-context-split.spec.ts diff --git a/api/server/controllers/agents/__tests__/toolContext.spec.js b/api/server/controllers/agents/__tests__/toolContext.spec.js new file mode 100644 index 0000000000..be207a21ab --- /dev/null +++ b/api/server/controllers/agents/__tests__/toolContext.spec.js @@ -0,0 +1,176 @@ +const { + AIMessage, + HumanMessage, + ToolMessage, + FunctionMessage, +} = require('@langchain/core/messages'); +const { syncBudgetDerivedFields } = require('@librechat/agents'); + +function snapshot(messageTokens = 1000, calibrationRatio = 1) { + return { + calibrationRatio, + contextBudget: 1100, + effectiveInstructionTokens: 100, + remainingContextTokens: 1000 - messageTokens, + breakdown: { messageTokens }, + }; +} + +function result(id, name) { + return new ToolMessage({ content: 'result', tool_call_id: id, name }); +} + +describe('agents tool context accounting', () => { + test('counts retained results and tool-only invocations without double counting raw calls', () => { + const usage = snapshot(); + const messages = [ + new HumanMessage('question'), + new AIMessage({ + content: '', + tool_calls: [{ id: 'a', name: 'read_file', args: {} }], + additional_kwargs: { + tool_calls: [ + { id: 'a', type: 'function', function: { name: 'read_file', arguments: '{}' } }, + ], + }, + }), + result('a'), + new AIMessage('answer'), + ]; + syncBudgetDerivedFields(usage, messages, () => 10); + expect(usage.breakdown.toolMessageTokens).toBe(20); + expect(usage.breakdown.toolMessageTokenCounts).toEqual({ read_file: 10 }); + }); + + test('keeps visible text and media assistant content outside the tool share', () => { + const content = [ + 'explaining', + ['explaining'], + [{ type: 'image_url', image_url: { url: 'https://example.com/image.png' } }], + ]; + for (const mixed of content) { + const usage = snapshot(); + const messages = [ + new AIMessage({ content: mixed, tool_calls: [{ id: 'a', name: 'read_file', args: {} }] }), + result('a'), + ]; + syncBudgetDerivedFields(usage, messages, () => 10); + expect(usage.breakdown.toolMessageTokens).toBe(10); + expect(usage.breakdown.toolMessageTokenCounts).toEqual({ read_file: 10 }); + } + }); + + test('counts tool-only reasoning and inline provider results without attributing invocation overhead', () => { + const usage = snapshot(); + syncBudgetDerivedFields( + usage, + [ + new AIMessage({ + content: [ + { type: 'thinking', thinking: 'reasoning' }, + { type: 'tool_use', id: 'a', name: 'file_search', input: {} }, + { type: 'web_search_tool_result', tool_use_id: 'a', content: [] }, + ], + }), + result('a', 'file_search'), + ], + () => 10, + ); + expect(usage.breakdown.toolMessageTokens).toBe(20); + expect(usage.breakdown.toolMessageTokenCounts).toEqual({ file_search: 10 }); + }); + + test('attributes raw calls, legacy functions, and unknown results without guessing an unrelated name', () => { + const usage = snapshot(); + syncBudgetDerivedFields( + usage, + [ + new AIMessage({ + content: '', + additional_kwargs: { + tool_calls: [ + { id: 'raw', type: 'function', function: { name: 'raw_tool', arguments: '{}' } }, + ], + }, + }), + result('raw'), + new AIMessage({ + content: '', + additional_kwargs: { function_call: { name: 'legacy_tool', arguments: '{}' } }, + }), + new FunctionMessage({ content: 'legacy result', name: '' }), + result('missing', 'explicit_tool'), + result('missing'), + ], + () => 10, + ); + expect(usage.breakdown.toolMessageTokens).toBe(60); + expect(usage.breakdown.toolMessageTokenCounts).toEqual({ + raw_tool: 10, + legacy_tool: 10, + explicit_tool: 10, + unknown_tool: 10, + }); + }); + + test('preserves prototype-sensitive tool names through JSON serialization', () => { + const usage = snapshot(); + syncBudgetDerivedFields( + usage, + [result('a', '__proto__'), result('b', 'constructor'), result('c', 'toString')], + () => 10, + ); + const counts = JSON.parse(JSON.stringify(usage)).breakdown.toolMessageTokenCounts; + expect(Object.hasOwn(counts, '__proto__')).toBe(true); + expect(counts.__proto__).toBe(10); + expect(counts.constructor).toBe(10); + expect(counts.toString).toBe(10); + }); + + test('apportions fractional calibration and budget clamping without exceeding the tool total', () => { + for (const ratio of [0.5, 1, 1.5, 5]) { + for (const available of [0, 1, 2, 10]) { + const usage = snapshot(available, ratio); + syncBudgetDerivedFields( + usage, + [result('a', 'a'), result('b', 'b'), result('c', 'c')], + () => 1, + ); + const { toolMessageTokens, toolMessageTokenCounts } = usage.breakdown; + expect(toolMessageTokens).toBe(Math.min(available, Math.round(3 * ratio))); + const counts = Object.values(toolMessageTokenCounts ?? {}); + expect(counts.every((count) => Number.isSafeInteger(count) && count >= 0)).toBe(true); + expect(counts.reduce((sum, count) => sum + count, 0)).toBe(toolMessageTokens); + } + } + }); + + test('distinguishes a known empty share from an unavailable counter', () => { + const known = snapshot(); + syncBudgetDerivedFields(known, [new HumanMessage('hello')], () => 10); + expect(known.breakdown.toolMessageTokens).toBe(0); + expect(known.breakdown.toolMessageTokenCounts).toBeUndefined(); + const unavailable = snapshot(); + syncBudgetDerivedFields(unavailable, [result('a')]); + expect(unavailable.breakdown.toolMessageTokens).toBeUndefined(); + }); + + test('drops an unavailable tool share without failing the model call', () => { + for (const value of [NaN, Infinity, -1, Number.MAX_SAFE_INTEGER + 1]) { + const usage = snapshot(); + usage.breakdown.toolMessageTokens = 20; + usage.breakdown.toolMessageTokenCounts = { read_file: 10 }; + syncBudgetDerivedFields(usage, [result('a')], () => value); + expect(usage.breakdown.toolMessageTokens).toBeUndefined(); + expect(usage.breakdown.toolMessageTokenCounts).toBeUndefined(); + expect(usage.breakdown.messageTokens).toBe(1000); + } + }); + + test('accepts approximate fractional token counters', () => { + const usage = snapshot(); + syncBudgetDerivedFields(usage, [result('a', 'file_search')], () => 1.5); + expect(usage.breakdown.toolMessageTokens).toBe(2); + expect(usage.breakdown.toolMessageTokenCounts).toEqual({ file_search: 2 }); + }); +}); diff --git a/api/server/controllers/agents/__tests__/usageEvents.integration.spec.js b/api/server/controllers/agents/__tests__/usageEvents.integration.spec.js index caa14f6ea7..cd61f4e167 100644 --- a/api/server/controllers/agents/__tests__/usageEvents.integration.spec.js +++ b/api/server/controllers/agents/__tests__/usageEvents.integration.spec.js @@ -232,6 +232,14 @@ describe('usage events through the real agents pipeline', () => { expect(remainingContextTokens).toBeLessThan(contextBudget); expect(breakdown.toolTokenCounts.add).toBeGreaterThan(0); } + expect(contextEvents[0].data.breakdown.toolMessageTokens).toBe(0); + const toolBreakdown = contextEvents[1].data.breakdown; + expect(toolBreakdown.toolMessageTokens).toBeGreaterThan(0); + expect(toolBreakdown.toolMessageTokens).toBeLessThanOrEqual(toolBreakdown.messageTokens); + expect(toolBreakdown.toolMessageTokenCounts.add).toBeGreaterThan(0); + expect( + Object.values(toolBreakdown.toolMessageTokenCounts).reduce((sum, count) => sum + count, 0), + ).toBeLessThanOrEqual(toolBreakdown.toolMessageTokens); /** Tool loop grows the context between calls */ expect(contextEvents[1].data.prePruneContextTokens).toBeGreaterThan( diff --git a/client/src/components/Chat/Input/TokenUsage/Breakdown.spec.tsx b/client/src/components/Chat/Input/TokenUsage/Breakdown.spec.tsx index 74dd039280..6841b51ac4 100644 --- a/client/src/components/Chat/Input/TokenUsage/Breakdown.spec.tsx +++ b/client/src/components/Chat/Input/TokenUsage/Breakdown.spec.tsx @@ -2,7 +2,7 @@ import '@testing-library/jest-dom/extend-expect'; import { Provider } from 'jotai'; import userEvent from '@testing-library/user-event'; import { Constants, Tools } from 'librechat-data-provider'; -import { render, screen, within, fireEvent } from '@testing-library/react'; +import { render, screen, within, fireEvent, waitFor } from '@testing-library/react'; import type { TokenUsageView } from '~/hooks/Chat/useTokenUsage'; import Breakdown from './Breakdown'; @@ -23,6 +23,8 @@ const view = { total: 1, estTokens: 0, tailEstTokens: 0, + estToolTokens: 0, + tailEstToolTokens: 0, containsAnchor: false, summaryBaseline: 0, usage: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, cost: 0, costKnown: true }, @@ -77,6 +79,14 @@ const snapshotView = { }, } as unknown as TokenUsageView; +/** Same snapshot with a reported tool-call split present: 150 of the 550 + * message tokens are tool exchanges, so Messages shows 400 and a Tool calls + * row appears. */ +const toolSplitView = JSON.parse(JSON.stringify(snapshotView)) as unknown as TokenUsageView; +toolSplitView.snapshot!.breakdown.toolMessageTokens = 150; +toolSplitView.snapshot!.breakdown.toolMessageTokenCounts = { read_file: 100 }; +toolSplitView.toolMessageTokenCounts = toolSplitView.snapshot!.breakdown.toolMessageTokenCounts; + const renderBreakdown = (props: Partial> = {}) => render( @@ -139,14 +149,14 @@ describe('TokenUsage Breakdown', () => { ), ).toEqual([ 'bg-series-1', // messages - 'bg-series-2', // system prompt - 'bg-series-3', // system tools - 'bg-series-3', // system tools, deferred - 'bg-series-4', // mcp tools - 'bg-series-4', // mcp tools, deferred - 'bg-series-5', // skills - 'bg-series-6', // subagents - 'bg-series-7', // summary + 'bg-series-3', // system prompt + 'bg-series-4', // system tools + 'bg-series-4', // system tools, deferred + 'bg-series-5', // mcp tools + 'bg-series-5', // mcp tools, deferred + 'bg-series-6', // skills + 'bg-series-7', // subagents + 'bg-series-8', // summary ]); }); @@ -157,7 +167,7 @@ describe('TokenUsage Breakdown', () => { renderBreakdown({ view: noSkills }); expect(screen.getByRole('progressbar').children).toHaveLength(8); - expect(screen.getByRole('progressbar').querySelector('.bg-series-5')).toBeNull(); + expect(screen.getByRole('progressbar').querySelector('.bg-series-6')).toBeNull(); }); it('gives the deferred rows their family slot and a hatch', async () => { @@ -172,9 +182,8 @@ describe('TokenUsage Breakdown', () => { const deferred = rowFor('com_ui_context_tools_system_deferred') .firstElementChild as HTMLElement; - expect(system).toHaveClass('bg-series-3'); - expect(deferred).toHaveClass('bg-series-3'); - expect(system.getAttribute('style') ?? '').not.toContain('repeating-linear-gradient'); + expect(system).toHaveClass('bg-series-4'); + expect(deferred).toHaveClass('bg-series-4'); expect(deferred.getAttribute('style')).toContain('repeating-linear-gradient'); }); @@ -271,6 +280,208 @@ describe('TokenUsage Breakdown', () => { expect(estimate).toBeInTheDocument(); expect(estimate.querySelector('[class*="bg-series-"]')).toBeNull(); }); + + it('keeps the estimate rows summing to used tokens when a tool share is known', async () => { + const estimateWithTools = { + ...view, + usedTokens: 600, + maxTokens: 2000, + branchTotals: { ...view.branchTotals, input: 300, output: 200 }, + estimatedTokens: 100, + messageTokens: 600, + toolCallTokens: 150, + } as TokenUsageView; + + renderBreakdown({ view: estimateWithTools }); + await userEvent.click(toggle()); + + const estimate = screen.getByTestId('context-estimate'); + const peerTotal = Array.from(estimate.children) + .filter((child) => !child.className.includes('pl-6')) + .reduce((sum, child) => sum + Number(child.lastElementChild?.textContent ?? 0), 0); + + /** 300 input + 200 output + 100 estimated = 600 used. The tool share is a + * subset of those rows, so it may only appear as an indented subtotal โ€” + * as a peer row the visible rows would claim 750 of a 600-token window. */ + expect(peerTotal).toBe(600); + + const toolRow = within(estimate).getByText('com_ui_context_tool_calls').parentElement + ?.parentElement as HTMLElement; + expect(toolRow.parentElement?.className).toContain('pl-6'); + expect(toolRow.textContent).toContain('150'); + }); + it('splits tool-call usage out of the messages row when reported', async () => { + renderBreakdown({ view: toolSplitView }); + await userEvent.click(toggle()); + + const breakdown = screen.getByTestId('context-breakdown'); + const rowFor = (label: string) => + within(breakdown).getByText(label).parentElement?.parentElement as HTMLElement; + + /** 1000 used โˆ’ 400 instructions โˆ’ 50 summary โˆ’ 150 tool calls = 400 */ + expect(rowFor('com_ui_context_messages').textContent).toContain('400'); + expect(rowFor('com_ui_context_tool_calls').textContent).toContain('150'); + }); + it('shows every named tool result, including known zeroes, in the disclosure', async () => { + const manyTools = JSON.parse(JSON.stringify(toolSplitView)) as TokenUsageView; + manyTools.snapshot!.breakdown.toolMessageTokenCounts = { + alpha: 30, + beta: 20, + gamma: 10, + delta: 8, + epsilon: 7, + zeta: 5, + eta: 3, + zero_tool: 0, + }; + manyTools.toolMessageTokenCounts = manyTools.snapshot!.breakdown.toolMessageTokenCounts; + + renderBreakdown({ view: manyTools }); + await userEvent.click(toggle()); + await userEvent.click(screen.getByRole('button', { name: /com_ui_context_tool_calls/ })); + + const details = screen.getByText('com_ui_context_tool_breakdown') + .parentElement as HTMLElement; + for (const name of [ + 'alpha', + 'beta', + 'gamma', + 'delta', + 'epsilon', + 'zeta', + 'eta', + 'zero_tool', + ]) { + expect(within(details).getByText(name)).toBeInTheDocument(); + } + expect( + within(details).getByText('zero_tool').parentElement?.parentElement?.textContent, + ).toContain('0'); + }); + + it('renders a reported zero tool-call share instead of treating it as unavailable', async () => { + const knownZero = JSON.parse(JSON.stringify(snapshotView)) as TokenUsageView; + knownZero.snapshot!.breakdown.toolMessageTokens = 0; + + renderBreakdown({ view: knownZero }); + await userEvent.click(toggle()); + const breakdown = screen.getByTestId('context-breakdown'); + expect(within(breakdown).getByText('com_ui_context_tool_calls')).toBeInTheDocument(); + expect( + within(breakdown).getByText('com_ui_context_tool_calls').parentElement?.parentElement + ?.textContent, + ).toContain('0'); + }); + + it('uses a native disclosure button for the expandable tool list', async () => { + renderBreakdown({ view: toolSplitView }); + await userEvent.click(toggle()); + + const toolButton = screen.getByRole('button', { name: /com_ui_context_tool_calls/ }); + expect(toolButton).toHaveAttribute('aria-expanded', 'false'); + + toolButton.focus(); + await userEvent.keyboard('{Enter}'); + + expect(toolButton).toHaveAttribute('aria-expanded', 'true'); + expect(screen.getByText('com_ui_context_tool_breakdown')).toBeInTheDocument(); + expect(document.getElementById(toolButton.getAttribute('aria-controls')!)).toContainElement( + screen.getByText('com_ui_context_tool_breakdown'), + ); + await userEvent.keyboard(' '); + expect(toolButton).toHaveAttribute('aria-expanded', 'false'); + }); + + it('keeps the messages row unsplit when the snapshot lacks the tool split', async () => { + renderBreakdown({ view: snapshotView }); + await userEvent.click(toggle()); + + const breakdown = screen.getByTestId('context-breakdown'); + expect(within(breakdown).queryByText('com_ui_context_tool_calls')).toBeNull(); + expect( + within(breakdown).getByText('com_ui_context_messages').parentElement?.parentElement + ?.textContent, + ).toContain('550'); + }); + + it('shows cached prompt shares as a subtotal, not beside the context rows', async () => { + const cached = { + ...snapshotView, + cacheRead: 30, + cacheWrite: 10, + } as TokenUsageView; + renderBreakdown({ view: cached }); + await userEvent.click(toggle()); + + const breakdown = screen.getByTestId('context-breakdown'); + const cachedRow = within(breakdown).getByText('com_ui_context_cached').parentElement + ?.parentElement as HTMLElement; + + /** Reconciliation already counted the cached prompt inside the segments + * above, so these may only appear indented โ€” as peer rows a fully cached + * prompt would show its tokens twice and the visible rows would sum past + * the meter. */ + expect(cachedRow.parentElement?.className).toContain('pl-6'); + expect(cachedRow.textContent).toContain('30'); + const peers = Array.from(breakdown.children).filter( + (child) => !child.className.includes('pl-6'), + ); + expect(peers.some((peer) => peer.textContent?.includes('com_ui_context_cache_write'))).toBe( + false, + ); + expect(screen.getByRole('progressbar')).toHaveAttribute('aria-valuenow', '50'); + }); + + it('hides the compaction hint when the operation is unavailable', async () => { + renderBreakdown({ + view: { ...toolSplitView, runwayTurns: 2, compactionReclaim: 90000 }, + compactionAvailable: false, + }); + await userEvent.click(toggle()); + await userEvent.click(screen.getByTestId('context-insights-toggle')); + const hints = await screen.findByTestId('context-hints'); + expect(hints.textContent).toContain('com_ui_context_runway'); + expect(hints.textContent).not.toContain('com_ui_context_compaction'); + }); + + it('warns under pressure inline, with insights behind the โ“˜ button', async () => { + const pressured = JSON.parse(JSON.stringify(toolSplitView)) as TokenUsageView; + pressured.percent = 85; + pressured.snapshot!.breakdown.toolMessageTokenCounts = { + grep: 1500, + read_file: 500, + }; + pressured.toolMessageTokenCounts = pressured.snapshot!.breakdown.toolMessageTokenCounts; + pressured.runwayTurns = 2; + pressured.compactionReclaim = 90000; + + renderBreakdown({ view: pressured, compactionAvailable: true }); + await userEvent.click(toggle()); + + /** Pressure warns inline; insights stay hidden until the โ“˜ is hovered */ + expect(screen.getByText('com_ui_context_pressure_warn')).toBeInTheDocument(); + expect(screen.queryByTestId('context-hints')).toBeNull(); + + const info = screen.getByTestId('context-insights-toggle'); + await userEvent.click(info); + const hints = await screen.findByTestId('context-hints'); + expect(hints.textContent).toContain('com_ui_context_largest_tool'); + expect(hints.textContent).toContain('com_ui_context_runway'); + expect(hints.textContent).toContain('com_ui_context_compaction'); + + await userEvent.keyboard('{Escape}'); + await waitFor(() => expect(screen.queryByTestId('context-hints')).toBeNull()); + }); + + it('shows the danger tint at the hard threshold', async () => { + const critical = JSON.parse(JSON.stringify(toolSplitView)) as TokenUsageView; + critical.percent = 96; + + renderBreakdown({ view: critical }); + await userEvent.click(toggle()); + + expect(screen.getByText('com_ui_context_pressure_danger')).toBeInTheDocument(); + }); }); describe('totals', () => { @@ -284,6 +495,32 @@ describe('TokenUsage Breakdown', () => { }), ).toBeInTheDocument(); }); + + it('counts a cached subagent call in the all-branches subtotal', async () => { + /** A fully cached subagent call reports its prompt under cacheRead and + * returns nothing: summing input+output alone would hide the row while + * the cache rows above still count the same traffic. */ + const cachedSubagent = { + ...view, + branchUsage: { ...view.branchUsage, cacheRead: 900 }, + subagentUsage: { + input: 0, + output: 0, + cacheRead: 900, + cacheWrite: 100, + cost: 0, + costKnown: true, + }, + } as TokenUsageView; + + renderBreakdown({ view: cachedSubagent }); + await userEvent.click(toggle()); + + const totals = within(screen.getByTestId('token-usage-totals')); + expect( + totals.getByText('com_ui_context_subagents_all').parentElement?.nextElementSibling, + ).toHaveTextContent('1K'); + }); }); describe('langfuse', () => { diff --git a/client/src/components/Chat/Input/TokenUsage/Breakdown.tsx b/client/src/components/Chat/Input/TokenUsage/Breakdown.tsx index 53959fafc9..83c69b2657 100644 --- a/client/src/components/Chat/Input/TokenUsage/Breakdown.tsx +++ b/client/src/components/Chat/Input/TokenUsage/Breakdown.tsx @@ -1,8 +1,12 @@ -import { useState } from 'react'; +import { Fragment, useId, useMemo, useState } from 'react'; import { useAtom } from 'jotai'; -import { ChevronDown, ExternalLink } from 'lucide-react'; +import { ChevronDown, ExternalLink, Info, TriangleAlert } from 'lucide-react'; import { Button, + HoverCard, + HoverCardPortal, + HoverCardContent, + HoverCardTrigger, Collapsible, MeterSwatch, SegmentedMeter, @@ -12,7 +16,7 @@ import { import type { MeterSegment } from '@librechat/client'; import type { TokenUsageView } from '~/hooks/Chat/useTokenUsage'; import type { CurrencyConfig } from '~/utils'; -import { groupToolTokens, formatTokens, formatCost } from '~/utils'; +import { groupToolTokens, formatTokens, formatCost, normalizeTokenCount, cn } from '~/utils'; import { contextBreakdownExpandedAtom } from '~/store/usage'; import { useLocalize } from '~/hooks'; @@ -27,16 +31,28 @@ interface RowProps { /** Meter segment this row keys; hovering it highlights its slice of the bar */ id?: string; onHoverChange?: (id: string | null) => void; + onClick?: () => void; + expanded?: boolean; + controls?: string; } -function Row({ label, value, max, segment, track, id, onHoverChange }: RowProps) { - const percent = max != null && max > 0 ? Math.min((value / max) * 100, 100) : null; - return ( -
onHoverChange?.(id) : undefined} - onPointerLeave={id != null ? () => onHoverChange?.(null) : undefined} - > +function Row({ + label, + value, + max, + segment, + track, + id, + onHoverChange, + onClick, + expanded, + controls, +}: RowProps) { + const safeValue = normalizeTokenCount(value); + const safeMax = max != null ? normalizeTokenCount(max) : 0; + const percent = safeMax > 0 ? Math.min((safeValue / safeMax) * 100, 100) : null; + const content = ( + <> {segment != null && } {track === true && ( @@ -45,16 +61,41 @@ function Row({ label, value, max, segment, track, id, onHoverChange }: RowProps) className="size-2 flex-none rounded-sm bg-surface-tertiary ring-1 ring-inset ring-border-medium" /> )} - {label} + {label} - - {formatTokens(value)} + + {formatTokens(safeValue)} {percent != null && ( )} + + ); + const className = cn( + 'flex w-full items-center justify-between gap-4 text-left text-sm', + onClick != null && + 'rounded-sm focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-text-primary', + ); + const handlers = { + onPointerEnter: id != null ? () => onHoverChange?.(id) : undefined, + onPointerLeave: id != null ? () => onHoverChange?.(null) : undefined, + }; + return onClick != null ? ( + + ) : ( +
+ {content}
); } @@ -62,6 +103,7 @@ function Row({ label, value, max, segment, track, id, onHoverChange }: RowProps) interface BreakdownProps { view: TokenUsageView; showCost: boolean; + compactionAvailable?: boolean; currency?: CurrencyConfig; langfuseSessionUrl?: string; } @@ -69,31 +111,137 @@ interface BreakdownProps { export default function Breakdown({ view, showCost, + compactionAvailable = false, currency, langfuseSessionUrl, }: BreakdownProps) { const localize = useLocalize(); const [expanded, setExpanded] = useAtom(contextBreakdownExpandedAtom); const [hoveredSegment, setHoveredSegment] = useState(null); - const { usedTokens, maxTokens, percent, snapshot, snapshotActive, branchUsage, hasUsage } = view; + const [toolsExpanded, setToolsExpanded] = useState(false); + const [insightsOpen, setInsightsOpen] = useState(false); + const insightsId = useId(); + const toolBreakdownId = useId(); + const usedTokens = normalizeTokenCount(view.usedTokens); + const maxTokens = view.maxTokens != null ? normalizeTokenCount(view.maxTokens) : undefined; + let percent = + maxTokens != null && maxTokens > 0 ? Math.min((usedTokens / maxTokens) * 100, 100) : 0; + if (typeof view.percent === 'number' && Number.isFinite(view.percent)) { + percent = Math.min(Math.max(view.percent, 0), 100); + } + const { snapshot, snapshotActive, branchUsage, hasUsage } = view; /** Show the all-branches total only when it (a) exceeds the active branch โ€” * epsilon guards against float summation order surfacing a spurious row in an * unbranched conversation โ€” and (b) has COMPLETE cost coverage, so a sibling * branch saved without cost can't render an under-reported total. */ - const showTotal = view.totalUsage.costKnown && view.totalCost - view.branchCost > 1e-9; + const showTotal = + view.totalUsage.costKnown && + Number.isFinite(view.totalCost) && + Number.isFinite(view.branchCost) && + view.totalCost - view.branchCost > 1e-9; + + /** Every normalized bucket of the subagent calls: a cached call reports its + * prompt under `cacheRead`/`cacheWrite`, so summing input+output alone + * under-reports it โ€” and hides a fully cached call with no output entirely, + * while the Totals rows above still count its cache traffic. */ + const subagentTokens = + normalizeTokenCount(view.subagentUsage?.input) + + normalizeTokenCount(view.subagentUsage?.output) + + normalizeTokenCount(view.subagentUsage?.cacheRead) + + normalizeTokenCount(view.subagentUsage?.cacheWrite); const breakdown = snapshotActive ? snapshot?.breakdown : undefined; - const instructionTokens = - snapshot?.effectiveInstructionTokens ?? breakdown?.instructionTokens ?? 0; - const systemTokens = - (breakdown?.systemMessageTokens ?? 0) + (breakdown?.dynamicInstructionTokens ?? 0); - /** Summary has its own row, so exclude it (it's part of `usedTokens`) to avoid - * double-counting it inside the Messages row on a summarized turn. */ - const messageTokens = Math.max( - 0, - usedTokens - instructionTokens - (breakdown?.summaryTokens ?? 0), + const instructionTokens = normalizeTokenCount( + snapshot?.effectiveInstructionTokens ?? breakdown?.instructionTokens, ); + const systemTokens = + normalizeTokenCount(breakdown?.systemMessageTokens) + + normalizeTokenCount(breakdown?.dynamicInstructionTokens); + /** Summary and tool calls have their own rows, so exclude both (they're part + * of `usedTokens`) to avoid double-counting them inside the Messages row. + * `toolMessageTokens` is absent on older SDK snapshots, keeping the row + * unsplit. */ + const summaryTokens = normalizeTokenCount(breakdown?.summaryTokens); + const messageBudget = Math.max(0, usedTokens - instructionTokens - summaryTokens); + /** `toolMessageTokens` is a subset of messages. Bound it by both the + * persisted message total and the current used-token remainder so malformed + * or stale snapshots cannot make the rows exceed the meter. */ + const rawToolCallTokens = + breakdown?.toolMessageTokens != null + ? normalizeTokenCount(breakdown.toolMessageTokens) + : undefined; + const toolCallTokens = + rawToolCallTokens != null + ? Math.min(rawToolCallTokens, normalizeTokenCount(breakdown?.messageTokens), messageBudget) + : undefined; + const messageTokens = Math.max(0, messageBudget - (toolCallTokens ?? 0)); const freeTokens = maxTokens != null ? Math.max(0, maxTokens - usedTokens) : null; + /** Context pressure: the gauge percent against soft (80%) and hard (95%) + * thresholds, driving the warning line's tint and the offender/runway hints. */ + let pressure: 'danger' | 'warn' | 'none' = 'none'; + if (percent >= 95) { + pressure = 'danger'; + } else if (percent >= 80) { + pressure = 'warn'; + } + const dynamicInstructionTokens = normalizeTokenCount(breakdown?.dynamicInstructionTokens); + /** Keep the largest tool for the pressure insight, but retain every named + * result in the disclosure. Result counts are a subset of the tool-call + * share; any invocation overhead in that share remains unassigned. */ + const { largestTool, toolBreakdown } = useMemo(() => { + const counts = view.toolMessageTokenCounts; + if (counts == null) { + return { + largestTool: null, + toolBreakdown: [] as Array<{ name: string; tokens: number }>, + }; + } + const candidates: Array<{ name: string; tokens: number }> = []; + for (const name in counts) { + if (!Object.prototype.hasOwnProperty.call(counts, name)) { + continue; + } + candidates.push({ name, tokens: normalizeTokenCount(counts[name]) }); + } + candidates.sort((a, b) => b.tokens - a.tokens); + + /** Keep displayed results within the reported tool-call share even when a + * malformed snapshot overstates one or more per-tool counts. Do not drop + * names after the share is exhausted: a known zero is different from an + * unavailable result count. */ + let remaining = toolCallTokens ?? Number.MAX_SAFE_INTEGER; + for (const candidate of candidates) { + candidate.tokens = Math.min(candidate.tokens, remaining); + remaining -= candidate.tokens; + } + return { + largestTool: candidates[0]?.tokens > 0 ? candidates[0] : null, + toolBreakdown: candidates, + }; + }, [toolCallTokens, view.toolMessageTokenCounts]); + const hasToolCounts = toolBreakdown.length > 0; + /** Insights shown on hovering the โ“˜ button: largest tool, runway, and what a + * summarization could reclaim. */ + const insights: string[] = []; + if (largestTool != null) { + insights.push( + localize('com_ui_context_largest_tool', { + 0: largestTool.name, + 1: formatTokens(largestTool.tokens), + }), + ); + } + const runwayTurns = + typeof view.runwayTurns === 'number' && Number.isFinite(view.runwayTurns) + ? Math.max(0, Math.floor(view.runwayTurns)) + : null; + if (runwayTurns != null && runwayTurns <= 25) { + insights.push(localize('com_ui_context_runway', { 0: String(runwayTurns) })); + } + const compactionReclaim = normalizeTokenCount(view.compactionReclaim); + if (compactionAvailable && compactionReclaim > 0) { + insights.push(localize('com_ui_context_compaction', { 0: formatTokens(compactionReclaim) })); + } const groups = breakdown?.toolTokenCounts != null @@ -114,14 +262,20 @@ export default function Breakdown({ value: messageTokens, slot: 1, }, - { id: 'system', label: localize('com_ui_context_system'), value: systemTokens, slot: 2 }, + { + id: 'tool-calls', + label: localize('com_ui_context_tool_calls'), + value: toolCallTokens ?? 0, + slot: 2, + }, + { id: 'system', label: localize('com_ui_context_system'), value: systemTokens, slot: 3 }, ...(groups == null ? [ { id: 'tools', label: localize('com_ui_context_tools'), - value: breakdown.toolSchemaTokens, - slot: 3, + value: normalizeTokenCount(breakdown.toolSchemaTokens), + slot: 4, }, ] : [ @@ -129,44 +283,43 @@ export default function Breakdown({ id: 'tools-system', label: localize('com_ui_context_tools_system'), value: groups.system, - slot: 3, + slot: 4, }, { id: 'tools-system-deferred', label: localize('com_ui_context_tools_system_deferred'), value: groups.systemDeferred, - slot: 3, + slot: 4, hatched: true, }, { id: 'tools-mcp', label: localize('com_ui_context_tools_mcp'), value: groups.mcp, - slot: 4, + slot: 5, }, { id: 'tools-mcp-deferred', label: localize('com_ui_context_tools_mcp_deferred'), value: groups.mcpDeferred, - slot: 4, + slot: 5, hatched: true, }, - { id: 'skills', label: localize('com_ui_skills'), value: groups.skills, slot: 5 }, + { id: 'skills', label: localize('com_ui_skills'), value: groups.skills, slot: 6 }, { id: 'subagents', label: localize('com_ui_context_subagents'), value: groups.subagents, - slot: 6, + slot: 7, }, ]), { id: 'summary', label: localize('com_ui_context_summary'), - value: breakdown.summaryTokens, - slot: 7, + value: normalizeTokenCount(breakdown.summaryTokens), + slot: 8, }, ]; - /** The estimate path knows the total but not the composition, so it keeps a * single unsegmented fill and its rows carry no swatches. */ const meterSegments: MeterSegment[] = @@ -192,7 +345,7 @@ export default function Breakdown({
@@ -225,27 +378,144 @@ export default function Breakdown({ so it collapses with the height. On the parent it would be a margin outside the animation, and unmounting the content would drop it in a single 12px jump after the height reached zero. */} - +
+ {(pressure !== 'none' || insights.length > 0) && ( +
+ {pressure !== 'none' ? ( +

+

+ ) : ( + + )} + {insights.length > 0 && ( + + + + + + + {insights.map((insight, index) => ( +

+ {insight} +

+ ))} +
+
+
+ )} +
+ )}
{breakdown ? ( <> - {segments.map( - ({ id, label, value, ...segment }) => - value > 0 && ( + {segments.map(({ id, label, value, ...segment }) => { + const shouldRender = + value > 0 || (id === 'tool-calls' && toolCallTokens != null); + if (!shouldRender) { + return null; + } + return ( + + {id === 'tool-calls' && hasToolCounts ? ( + setToolsExpanded((open) => !open)} + expanded={toolsExpanded} + controls={toolsExpanded ? toolBreakdownId : undefined} + /> + ) : ( + + )} + {id === 'tool-calls' && toolsExpanded && hasToolCounts && ( +
+

+ {localize('com_ui_context_tool_breakdown')} +

+ {toolBreakdown.map((tool) => ( + + ))} +
+ )} + {id === 'system' && dynamicInstructionTokens > 0 && ( +
+ +
+ )} +
+ ); + })} + {/** The reconciling call's cached prompt share is already inside + * the segments above (reconciliation counted it in `usedTokens`), + * so it renders indented, like the estimate path's tool-call + * share โ€” as peer rows a fully cached prompt would show its + * tokens twice and the visible rows would sum past the meter. */} + {(normalizeTokenCount(view.cacheRead) > 0 || + normalizeTokenCount(view.cacheWrite) > 0) && ( +
+ {normalizeTokenCount(view.cacheRead) > 0 && ( - ), + )} + {normalizeTokenCount(view.cacheWrite) > 0 && ( + + )} +
)} {freeTokens != null && ( ) : ( <> - {view.branchTotals.summaryBaseline > 0 && ( + {normalizeTokenCount(view.branchTotals.summaryBaseline) > 0 && ( - {view.estimatedTokens > 0 && ( + {normalizeTokenCount(view.estimatedTokens) > 0 && ( )} + {/** Retained tool traffic is already inside the message rows above + * (their clamped tool-content share), so it renders as an indented + * subtotal โ€” as a peer row the visible rows would sum past + * `usedTokens`. The snapshot path instead subtracts it from + * Messages, which is exact there; here the counted Input/Output + * rows are provider counts and this share is a char estimate, so + * subtracting it would corrupt them. */} + {normalizeTokenCount(view.toolCallTokens) > 0 && ( +
+ +
+ )} {view.overheadTokens > 0 && ( )} @@ -314,12 +599,19 @@ export default function Breakdown({ - {branchUsage.cacheRead > 0 && ( + {normalizeTokenCount(branchUsage.cacheRead) > 0 && ( )} - {branchUsage.cacheWrite > 0 && ( + {normalizeTokenCount(branchUsage.cacheWrite) > 0 && ( )} + {/** Subagent calls are accumulated per conversation for the session + * and are not attributed to a response, so they cannot be scoped to + * the viewed branch like the rows above; label them all-branches + * rather than imply branch scope. */} + {subagentTokens > 0 && ( + + )}
)} diff --git a/client/src/components/Chat/Input/TokenUsage/index.tsx b/client/src/components/Chat/Input/TokenUsage/index.tsx index c755694b81..41b3ad8ce0 100644 --- a/client/src/components/Chat/Input/TokenUsage/index.tsx +++ b/client/src/components/Chat/Input/TokenUsage/index.tsx @@ -43,6 +43,7 @@ function TokenUsageIndicator({ /** Owned here, not in the popover: `unmountOnHide` would otherwise lose the * in-flight state the moment the pointer leaves. */ const compaction = useCompactConversation(); + const compactionAvailable = compactionEnabled && supportsCompaction(conversation?.endpoint); const popover = Ariakit.usePopoverStore({ placement: 'top' }); const popoverOpen = Ariakit.useStoreState(popover, 'open'); const disclosureRef = useRef(null); @@ -243,10 +244,11 @@ function TokenUsageIndicator({ - {compactionEnabled && supportsCompaction(conversation?.endpoint) && ( + {compactionAvailable && ( <>
({ + useLatestMessageId: () => 'a2', +})); + +jest.mock('~/hooks/Chat/useTokenLimits', () => ({ + __esModule: true, + default: () => ({ maxContextTokens: 200000, endpoint: 'agents', model: 'test-model' }), +})); + +const convo = 'convo-post-snapshot'; + +/** Two completed turns: the tail response `a2` answers `u2`. */ +const messages = [ + { + messageId: 'u1', + parentMessageId: null, + conversationId: convo, + isCreatedByUser: true, + tokenCount: 400, + text: 'first ask', + }, + { + messageId: 'a1', + parentMessageId: 'u1', + conversationId: convo, + isCreatedByUser: false, + tokenCount: 1500, + text: 'first answer', + }, + { + messageId: 'u2', + parentMessageId: 'a1', + conversationId: convo, + isCreatedByUser: true, + tokenCount: 500, + text: 'second ask', + }, + { + messageId: 'a2', + parentMessageId: 'u2', + conversationId: convo, + isCreatedByUser: false, + tokenCount: 2000, + text: 'second answer', + }, +] as unknown as TMessage[]; + +/** Persisted per-turn snapshots: used grows 1000 tokens per call + * (196000 โ†’ 195000 remaining of a 200000 budget). */ +const anchorSnapshot = (remaining: number): ContextSnapshot => + ({ + anchorMessageId: null, + model: 'test-model', + provider: 'openAI', + contextBudget: 200000, + remainingContextTokens: remaining, + breakdown: { maxContextTokens: 200000, instructionTokens: 4000 }, + }) as unknown as ContextSnapshot; + +/** The live snapshot for the tail call: pre-invoke, so its remaining headroom + * and message total both predate `a2`'s 2000 output tokens. */ +const tailSnapshot: ContextSnapshot = { + anchorMessageId: 'u2', + model: 'test-model', + provider: 'openAI', + contextBudget: 200000, + remainingContextTokens: 5000, + completedOutputTokens: 2000, + breakdown: { + maxContextTokens: 200000, + instructionTokens: 4000, + messageTokens: 10000, + }, +} as unknown as ContextSnapshot; + +const renderTokenUsage = ( + anchors: Map = new Map([ + ['a1', anchorSnapshot(196000)], + ['a2', anchorSnapshot(195000)], + ]), + overrides: { messages?: TMessage[]; snapshot?: ContextSnapshot } = {}, +) => { + const queryClient = new QueryClient({ + defaultOptions: { queries: { retry: false } }, + }); + queryClient.setQueryData([QueryKeys.messages, convo], overrides.messages ?? messages); + const store = getDefaultStore(); + store.set(contextSnapshotFamily(convo), overrides.snapshot ?? tailSnapshot); + store.set(snapshotsByAnchorFamily(convo), anchors); + + return renderHook( + () => + useTokenUsage({ + index: 0, + conversation: { conversationId: convo, endpoint: 'agents' } as TConversation, + isSubmitting: false, + }), + { + wrapper: ({ children }) => ( + {children} + ), + }, + ); +}; + +describe('useTokenUsage โ€” post-snapshot output', () => { + it('charges the finalized output against the runway projection', () => { + const { result } = renderTokenUsage(); + + /** 5000 pre-invoke remaining โˆ’ 2000 already generated = 3000 of real + * headroom at 1000 tokens per call. Using the stale remaining would + * promise 5 more turns. */ + expect(result.current.runwayTurns).toBe(3); + }); + + it('counts the finalized output in what a summarization could reclaim', () => { + const { result } = renderTokenUsage(); + + /** 10000 pre-invoke message tokens + 2000 finalized output โˆ’ the kept + * latest exchange (500 + 2000) = 9500. Omitting the output would + * under-report the reclaim by the size of the final answer. */ + expect(result.current.compactionReclaim).toBe(9500); + }); + + /** The older persisted shape: a budget and a breakdown, no remaining count. */ + const legacySnapshot = (messageTokens: number): ContextSnapshot => + ({ + anchorMessageId: null, + model: 'test-model', + provider: 'openAI', + contextBudget: 200000, + breakdown: { maxContextTokens: 200000, instructionTokens: 4000, messageTokens }, + }) as unknown as ContextSnapshot; + + it('projects a branch whose snapshots predate the remaining-token field', () => { + /** Both readings are instruction+messages sums โ€” 13000 then 14000, so the + * call grew 1000. Reading the absent remaining as zero would score both + * turns as having spent the entire 200000 window and hide the projection. */ + const { result } = renderTokenUsage( + new Map([ + ['a1', legacySnapshot(9000)], + ['a2', legacySnapshot(10000)], + ]), + ); + + expect(result.current.runwayTurns).toBe(3); + }); + + it('withholds the projection when the two readings were measured differently', () => { + /** `a1` reports remaining headroom (4000 used of 200000) and `a2` only a + * breakdown (14000). Their difference is the content a breakdown omits, + * not what the last call added, so no per-call growth can be read from + * the pair โ€” a 10000-token "growth" would report zero turns left. */ + const { result } = renderTokenUsage( + new Map([ + ['a1', anchorSnapshot(196000)], + ['a2', legacySnapshot(10000)], + ]), + ); + + expect(result.current.runwayTurns).toBeUndefined(); + }); + + it.each([ + { model: 'other-model' }, + { provider: 'anthropic' }, + { agentId: 'other-agent' }, + { contextBudget: 100000 }, + { breakdown: { maxContextTokens: 200000, instructionTokens: 8000 } }, + { breakdown: { maxContextTokens: 200000, toolSchemaTokens: 8000 } }, + { model: undefined }, + { provider: undefined }, + ])('withholds runway across a configuration change: %j', (change) => { + const { result } = renderTokenUsage( + new Map([ + [ + 'a1', + { + ...anchorSnapshot(196000), + ...change, + breakdown: { ...anchorSnapshot(196000).breakdown, ...change.breakdown }, + }, + ], + ['a2', anchorSnapshot(195000)], + ]), + ); + expect(result.current.runwayTurns).toBeUndefined(); + }); + + it('withholds old runway history during a newly configured run', () => { + const { result } = renderTokenUsage(undefined, { + snapshot: { ...tailSnapshot, model: 'new-model' }, + }); + expect(result.current.runwayTurns).toBeUndefined(); + }); + + it('restores cache rows and call identity from persisted snapshots', () => { + const persisted = { + ...anchorSnapshot(195000), + cacheRead: 800, + cacheWrite: 200, + completedOutputTokens: 50, + }; + const persistedById: Record = { + a1: anchorSnapshot(196000), + a2: persisted, + }; + const { result } = renderTokenUsage(new Map(), { + snapshot: { ...tailSnapshot, anchorMessageId: 'unrelated-branch' }, + messages: messages.map((message) => ({ + ...message, + metadata: { + contextUsage: persistedById[message.messageId], + }, + })), + }); + expect(result.current.cacheRead).toBe(800); + expect(result.current.cacheWrite).toBe(200); + expect(result.current.snapshot?.model).toBe('test-model'); + expect(result.current.runwayTurns).toBe(194); + }); + + it('excludes the retained latest tool result from compaction savings', () => { + const { result } = renderTokenUsage(undefined, { + messages: messages.map((message) => + message.messageId === 'a2' + ? ({ + ...message, + content: [ + { + type: 'tool_call', + tool_call: { + name: 'read_file', + args: 'path', + output: 'r'.repeat(20000), + }, + }, + ], + } as TMessage) + : message, + ), + }); + expect(result.current.compactionReclaim).toBe(4500); + }); + + it('leaves a summarizing turnโ€™s summary completion out of the reclaim estimate', () => { + /** The tail turn compacted: its `tokenCount` carries the 700-token + * summarization completion the backend folded in, while the snapshot holds + * those tokens in `summaryTokens` rather than `messageTokens`. Subtracting + * the whole response would understate the reclaim by that summary. + * 10000 messages + 2000 finalized output โˆ’ (500 user + 2000 answer โˆ’ 700 + * summary) = 10200. */ + const summarizedMessages = messages.map((message) => + (message as TMessage).messageId === 'a2' + ? ({ ...message, metadata: { summaryUsedTokens: 8000 } } as TMessage) + : message, + ); + + /** After finalize the live snapshot is re-anchored to the response id, so + * the compacting turn's own snapshot still describes the viewed branch. */ + const summarizedSnapshot = { + ...tailSnapshot, + anchorMessageId: 'a2', + breakdown: { ...tailSnapshot.breakdown, summaryTokens: 700 }, + } as unknown as ContextSnapshot; + + const { result } = renderTokenUsage(undefined, { + messages: summarizedMessages, + snapshot: summarizedSnapshot, + }); + + expect(result.current.compactionReclaim).toBe(10200); + }); +}); diff --git a/client/src/hooks/Chat/useTokenUsage.ts b/client/src/hooks/Chat/useTokenUsage.ts index 1f4a474d3f..8cffbf407d 100644 --- a/client/src/hooks/Chat/useTokenUsage.ts +++ b/client/src/hooks/Chat/useTokenUsage.ts @@ -13,6 +13,8 @@ import { removeUsageAtoms, hydrateSnapshots, pendingUsageFamily, + subagentUsageFamily, + pendingSubagentUsageFamily, branchTotalsFamily, contextSnapshotFamily, snapshotsByAnchorFamily, @@ -24,7 +26,11 @@ import { mergeUsage, sumTotalUsage, prunedBranchTokens, + collectAnchorSeries, + snapshotConfiguration, + latestExchangeTokens, findBranchSnapshotAnchor, + normalizeTokenCount, } from '~/utils'; import { useLatestMessageId } from '~/hooks/Messages/useLatestMessage'; import useTokenLimits from './useTokenLimits'; @@ -68,6 +74,23 @@ export interface TokenUsageView { /** True when over-window pruning replaced the raw message sum, so the breakdown * shows a single pruned Messages row instead of input/output/estimated. */ messagesPruned: boolean; + /** Tool-call share of the message tokens: `breakdown.toolMessageTokens` on + * the snapshot path, the clamped branch walk share on the estimate path. + * Undefined when the producing SDK doesn't report it (older snapshots) or + * the estimate is zero โ€” the row stays hidden. A SUBSET of messages. */ + toolCallTokens?: number; + /** Cache split of the reconciling call's prompt (live or persisted) โ€” a + * share of the used context, not an addition. */ + cacheRead?: number; + cacheWrite?: number; + /** Per-tool result-token counts, when provided by the usage snapshot */ + toolMessageTokenCounts?: Record; + /** Turns of headroom at the current per-call growth (undefined when unknown) */ + runwayTurns?: number; + /** Tokens a summarization could reclaim โ‰ˆ context โˆ’ latest exchange */ + compactionReclaim?: number; + /** Live subagent model-call usage โ€” a subset of Totals */ + subagentUsage?: BranchUsage; rates?: TModelTokenomics; } @@ -90,6 +113,8 @@ export default function useTokenUsage({ const totalUsageBase = useAtomValue(totalUsageFamily(conversationKey)); const branchTotals = useAtomValue(branchTotalsFamily(conversationKey)); const liveTokens = useAtomValue(liveTokensFamily(conversationKey)); + const committedSubagentUsage = useAtomValue(subagentUsageFamily(conversationKey)); + const pendingSubagentUsage = useAtomValue(pendingSubagentUsageFamily(conversationKey)); const setBranchTotals = useSetAtom(branchTotalsFamily(conversationKey)); const setTotalUsage = useSetAtom(totalUsageFamily(conversationKey)); const limits = useTokenLimits(conversation); @@ -115,12 +140,17 @@ export default function useTokenUsage({ * call (premium tiers, cache rates), so cost sums authoritatively. */ const pendingAsUsage: BranchUsage = useMemo( () => ({ - input: pendingUsage.input, - output: pendingUsage.output, - cacheWrite: pendingUsage.cacheWrite, - cacheRead: pendingUsage.cacheRead, - cost: pendingUsage.costUSD, - costKnown: pendingUsage.costKnown, + input: normalizeTokenCount(pendingUsage.input), + output: normalizeTokenCount(pendingUsage.output), + cacheWrite: normalizeTokenCount(pendingUsage.cacheWrite), + cacheRead: normalizeTokenCount(pendingUsage.cacheRead), + cost: + typeof pendingUsage.costUSD === 'number' && + Number.isFinite(pendingUsage.costUSD) && + pendingUsage.costUSD > 0 + ? pendingUsage.costUSD + : 0, + costKnown: pendingUsage.costKnown === true, }), [pendingUsage], ); @@ -132,6 +162,13 @@ export default function useTokenUsage({ () => mergeUsage(totalUsageBase, pendingAsUsage), [totalUsageBase, pendingAsUsage], ); + /** The subagent figure mirrors that split: what earlier runs committed plus + * the in-flight run's share, which settles or is discarded with the pending + * usage it sits inside. */ + const subagentUsage = useMemo( + () => mergeUsage(committedSubagentUsage, pendingSubagentUsage), + [committedSubagentUsage, pendingSubagentUsage], + ); const hasUsage = branchUsage.input + branchUsage.output + branchUsage.cacheRead + branchUsage.cacheWrite > 0; @@ -222,19 +259,62 @@ export default function useTokenUsage({ * history, imports, never-generated branches) entirely client-side. */ const effective: ContextSnapshot | null = currentActive ? snapshot : branchSnapshot; + /** Branch analysis: the snapshot series (oldest โ†’ newest) drives the runway + * projection. Walks use the branch tail, capped at summary markers, + * mirroring `sumBranch`. */ + const tailId = branchTotals.tailId; + const anchorSeries = + effective != null ? collectAnchorSeries(conversationKey, tailId, snapshotsByAnchor) : []; + /** The snapshot is pre-invoke, so its `remainingContextTokens` predates the + * last call's finalized output and any in-flight output. Both already ate + * that headroom (`usedTokens` adds them), so the projection must too. */ + const completedOutput = normalizeTokenCount(effective?.completedOutputTokens); + const liveOutput = normalizeTokenCount(liveTokens); + const remainingForRunway = + effective?.remainingContextTokens != null + ? Math.max( + 0, + normalizeTokenCount(effective.remainingContextTokens) - completedOutput - liveOutput, + ) + : null; + let runwayTurns: number | undefined; + if (anchorSeries.length >= 2 && remainingForRunway != null) { + const latest = anchorSeries[anchorSeries.length - 1]; + const previous = anchorSeries[anchorSeries.length - 2]; + /** Growth is only a per-call delta when both readings were measured the + * same way; a remaining-based reading next to a breakdown-derived one + * (a snapshot saved before `remainingContextTokens` existed) differs by + * the content the breakdown omits, not by what the last call added. The + * projection stays unavailable rather than reporting that difference. */ + const comparable = + latest.basis === previous.basis && + latest.configuration != null && + latest.configuration === previous.configuration && + effective != null && + latest.configuration === snapshotConfiguration(effective); + const perCallGrowth = comparable ? latest.used - previous.used : 0; + if (perCallGrowth > 0) { + runwayTurns = Math.max(0, Math.floor(remainingForRunway / perCallGrowth)); + } + } if (effective != null) { const breakdown = effective.breakdown; - const maxTokens = effective.contextBudget ?? breakdown.maxContextTokens; - const instructionTokens = effective.effectiveInstructionTokens ?? breakdown.instructionTokens; - const baseUsed = + const maxTokens = normalizeTokenCount(effective.contextBudget ?? breakdown.maxContextTokens); + const instructionTokens = normalizeTokenCount( + effective.effectiveInstructionTokens ?? breakdown.instructionTokens, + ); + const remainingContextTokens = effective.remainingContextTokens != null - ? maxTokens - effective.remainingContextTokens - : instructionTokens + breakdown.messageTokens; + ? normalizeTokenCount(effective.remainingContextTokens) + : null; + const baseUsed = + remainingContextTokens != null + ? maxTokens - remainingContextTokens + : instructionTokens + normalizeTokenCount(breakdown.messageTokens); /** The snapshot is pre-invoke: in-flight output rides on `liveTokens` (0 * unless streaming this branch), the last call's finalized output on * `completedOutputTokens`. */ - const usedTokens = - Math.max(0, baseUsed) + liveTokens + (effective.completedOutputTokens ?? 0); + const usedTokens = normalizeTokenCount(Math.max(0, baseUsed) + liveOutput + completedOutput); return { usedTokens, maxTokens, @@ -248,11 +328,42 @@ export default function useTokenUsage({ hasUsage, branchCost: branchUsage.cost, totalCost: totalUsage.cost, - liveTokens, + liveTokens: liveOutput, estimatedTokens: 0, overheadTokens: 0, messageTokens: 0, messagesPruned: false, + toolCallTokens: + breakdown.toolMessageTokens != null + ? Math.min( + normalizeTokenCount(breakdown.toolMessageTokens), + normalizeTokenCount(breakdown.messageTokens), + ) + : undefined, + cacheRead: normalizeTokenCount(effective.cacheRead), + cacheWrite: normalizeTokenCount(effective.cacheWrite), + toolMessageTokenCounts: breakdown.toolMessageTokenCounts, + runwayTurns, + /** Both sides are measured after the tail call: `breakdown.messageTokens` + * is pre-invoke, so add the finalized output that `latestExchangeTokens` + * already counts in the exchange a summarization would keep, and hand it + * the summary block's size so a compacting turn's summarization output โ€” + * folded into the response's `tokenCount` but held outside + * `messageTokens` โ€” is not subtracted from a total that never carried + * it. While a response streams, `completedOutput` is 0 and the in-flight + * tail is excluded from both sides (it rides on `liveTokens`). */ + compactionReclaim: Math.max( + 0, + normalizeTokenCount(breakdown.messageTokens) + + completedOutput - + latestExchangeTokens( + conversationKey, + tailId, + liveTokens > 0, + normalizeTokenCount(breakdown.summaryTokens), + ), + ), + subagentUsage, rates: limits.rates, }; } @@ -266,8 +377,9 @@ export default function useTokenUsage({ * there, so input/output are post-summary only โ€” adding it keeps the estimate * from re-summing the discarded pre-summary history (which otherwise pins the * gauge at 100% after a compaction). */ - const maxTokens = limits.maxContextTokens; - const liveOnTail = liveTokens > 0; + const maxTokens = + limits.maxContextTokens != null ? normalizeTokenCount(limits.maxContextTokens) : undefined; + const liveOnTail = normalizeTokenCount(liveTokens) > 0; /** Fixed instruction + tool-schema overhead for this agent/model (the latter is * already folded into `instructionTokens`), cached from live usage events. The * client can't otherwise know it for a snapshot-less branch, so reserve it from @@ -279,11 +391,13 @@ export default function useTokenUsage({ const overheadTokens = branchTotals.summaryBaseline > 0 ? 0 - : getModelOverhead( - overheadKey( - limits.endpoint ?? conversation?.endpoint, - limits.model ?? conversation?.model, - conversation?.agent_id, + : normalizeTokenCount( + getModelOverhead( + overheadKey( + limits.endpoint ?? conversation?.endpoint, + limits.model ?? conversation?.model, + conversation?.agent_id, + ), ), ); /** When a stream is live the tail is the in-flight response, already counted @@ -291,10 +405,23 @@ export default function useTokenUsage({ * isn't double-counted on the estimate path. */ const estimatedTokens = Math.max( 0, - branchTotals.estTokens - (liveOnTail ? branchTotals.tailEstTokens : 0), + normalizeTokenCount(branchTotals.estTokens) - + (liveOnTail ? normalizeTokenCount(branchTotals.tailEstTokens) : 0), ); - const rawMessageTokens = branchTotals.input + branchTotals.output + estimatedTokens; + const rawMessageTokens = + normalizeTokenCount(branchTotals.input) + + normalizeTokenCount(branchTotals.output) + + estimatedTokens; let messageTokens = rawMessageTokens; + /** Tool-call share of the same walk (a subset of input/output/estimated). + * Mirrors the tail exclusion so a live in-flight response's tool tokens + * aren't counted before its content lands. */ + const estimatedToolTokens = Math.max( + 0, + normalizeTokenCount(branchTotals.estToolTokens) - + (liveOnTail ? normalizeTokenCount(branchTotals.tailEstToolTokens) : 0), + ); + let toolTokens = Math.min(estimatedToolTokens, messageTokens); /** The send path prunes an over-window branch oldest-first before calling the * model, so the next call can sit well under the window even when the full * branch exceeds it. Mirror that: when the raw sum overflows the message window @@ -302,21 +429,35 @@ export default function useTokenUsage({ * the newest messages that actually fit instead of clamping the whole branch to * 100%. */ if (maxTokens != null && maxTokens > 0) { - const messageBudget = Math.max(0, maxTokens - branchTotals.summaryBaseline - overheadTokens); + const messageBudget = Math.max( + 0, + maxTokens - normalizeTokenCount(branchTotals.summaryBaseline) - overheadTokens, + ); if (messageTokens > messageBudget) { - messageTokens = prunedBranchTokens( + const pruned = prunedBranchTokens( conversationKey, branchTotals.tailId, messageBudget, liveOnTail, ); + messageTokens = normalizeTokenCount(pruned.tokens); + toolTokens = Math.min(normalizeTokenCount(pruned.toolTokens), messageTokens); } } /** When pruning replaced the raw sum, the per-category input/output/estimated - * rows no longer describe what's sent, so the breakdown collapses them into a - * single pruned Messages row to stay consistent with the gauge. */ + * rows no longer describe what's sent, so the breakdown collapses them into + * a single pruned Messages row to stay consistent with the gauge. */ const messagesPruned = messageTokens < rawMessageTokens; - const usedTokens = overheadTokens + branchTotals.summaryBaseline + messageTokens + liveTokens; + const compactionReclaim = Math.max( + 0, + messageTokens - latestExchangeTokens(conversationKey, branchTotals.tailId, liveOnTail), + ); + const usedTokens = normalizeTokenCount( + overheadTokens + + normalizeTokenCount(branchTotals.summaryBaseline) + + messageTokens + + normalizeTokenCount(liveTokens), + ); return { usedTokens, maxTokens, @@ -331,11 +472,14 @@ export default function useTokenUsage({ hasUsage, branchCost: branchUsage.cost, totalCost: totalUsage.cost, - liveTokens, + liveTokens: normalizeTokenCount(liveTokens), estimatedTokens, overheadTokens, messageTokens, messagesPruned, + toolCallTokens: toolTokens > 0 ? toolTokens : undefined, + compactionReclaim, + subagentUsage, rates: limits.rates, }; }, [ @@ -348,6 +492,8 @@ export default function useTokenUsage({ liveTokens, limits, branchSnapshot, + snapshotsByAnchor, + subagentUsage, conversationKey, conversation, ]); diff --git a/client/src/hooks/SSE/__tests__/useUsageHandler.spec.tsx b/client/src/hooks/SSE/__tests__/useUsageHandler.spec.tsx index 41ba9e3511..d571c4d676 100644 --- a/client/src/hooks/SSE/__tests__/useUsageHandler.spec.tsx +++ b/client/src/hooks/SSE/__tests__/useUsageHandler.spec.tsx @@ -1,8 +1,14 @@ import { getDefaultStore } from 'jotai'; import { renderHook } from '@testing-library/react'; +import { Constants, reconcileContextUsageFromEvent } from 'librechat-data-provider'; import type { TContextUsageEvent, TTokenUsageEvent } from 'librechat-data-provider'; +import { + contextSnapshotFamily, + liveTokensFamily, + subagentUsageFamily, + pendingSubagentUsageFamily, +} from '~/store/usage'; import useUsageHandler from '~/hooks/SSE/useUsageHandler'; -import { contextSnapshotFamily } from '~/store/usage'; /** Mirrors a real web-search + summarization turn: calibration pinned at 5 * inflated messageTokens to 187471 (used 213375), while the call's true prompt @@ -33,6 +39,7 @@ const primaryUsage = (over?: Partial): TTokenUsageEvent => ({ total_tokens: 59553, input_token_details: { cache_read: 2071, cache_creation: 0 }, provider: 'anthropic', + model: 'primary-model', runId: 'run-1', seq: 1, ...over, @@ -60,6 +67,47 @@ describe('useUsageHandler โ€” live snapshot reconciliation', () => { /** instructions/summary stay raw; the anchor is preserved */ expect(snap?.breakdown.instructionTokens).toBe(4205); expect(snap?.anchorMessageId).toBe('u1'); + expect(snap).toMatchObject({ + cacheRead: 2071, + cacheWrite: 0, + model: 'primary-model', + provider: 'anthropic', + }); + }); + + it('keeps resumed completed output once through text seeding, replay, and finalize', () => { + const convo = 'convo-resume-completed'; + const submission = { + userMessage: { messageId: 'u-completed', conversationId: convo }, + conversation: { conversationId: convo }, + }; + const { result } = renderHook(() => useUsageHandler()); + const store = getDefaultStore(); + const usage = primaryUsage(); + const { completedOutputTokens, ...reconciled } = reconcileContextUsageFromEvent( + inflatedSnapshot(), + usage, + ); + const snapshot = { ...reconciled, resumedOutputTokens: completedOutputTokens }; + result.current.backfillUsage([usage], submission); + result.current.contextHandler(snapshot, submission); + result.current.seedLive(12000, submission); + result.current.usageHandler(usage, submission); + expect(store.get(liveTokensFamily(convo))).toBe(0); + expect(store.get(contextSnapshotFamily(convo))?.completedOutputTokens).toBe(3780); + result.current.finalizeUsage( + { + conversation: { conversationId: convo }, + responseMessage: { messageId: 'a-completed', conversationId: convo }, + }, + submission, + ); + expect(store.get(contextSnapshotFamily(convo))?.completedOutputTokens).toBe(3780); + + result.current.contextHandler(inflatedSnapshot(), submission); + result.current.seedLive(400, submission); + expect(store.get(contextSnapshotFamily(convo))?.completedOutputTokens).toBeUndefined(); + expect(store.get(liveTokensFamily(convo))).toBe(500); }); it('does not reconcile a replayed (already-folded) primary usage', () => { @@ -113,4 +161,85 @@ describe('useUsageHandler โ€” live snapshot reconciliation', () => { expect(store.get(contextSnapshotFamily(convo))?.breakdown.messageTokens).toBe(187471); }); + + it('carries subagent totals through the new-conversation id handoff', () => { + /** The first turn of a new chat accumulates under `new`; `finalizeUsage` + * migrates the usage atoms to the persisted id and drops the temporary + * ones, so a subagent total left behind would vanish on completion. */ + const newConvo: string = Constants.NEW_CONVO; + const submission = { + userMessage: { messageId: 'u-sub', conversationId: newConvo }, + conversation: { conversationId: newConvo }, + }; + const { result } = renderHook(() => useUsageHandler()); + const store = getDefaultStore(); + + result.current.usageHandler( + primaryUsage({ usage_type: 'subagent', runId: 'run-sub', seq: 11 }), + submission, + ); + expect(store.get(pendingSubagentUsageFamily(newConvo)).output).toBe(3780); + + result.current.finalizeUsage( + { + conversation: { conversationId: 'convo-sub-real' }, + responseMessage: { messageId: 'r-sub', conversationId: 'convo-sub-real' }, + }, + submission, + ); + + const migrated = store.get(subagentUsageFamily('convo-sub-real')); + expect(migrated.input).toBe(53702); + expect(migrated.output).toBe(3780); + expect(store.get(pendingSubagentUsageFamily('convo-sub-real')).output).toBe(0); + }); + + it('drops a failed runโ€™s subagent share with the usage it belonged to', () => { + /** Terminal error with no salvageable response: the pending usage is + * discarded, so the Totals row must not keep advertising tokens that no + * longer appear in the branch or conversation rollups. */ + const convo = 'convo-sub-failed'; + const submission = { + userMessage: { messageId: 'u-fail', conversationId: convo }, + conversation: { conversationId: convo }, + }; + const { result } = renderHook(() => useUsageHandler()); + const store = getDefaultStore(); + + result.current.usageHandler( + primaryUsage({ usage_type: 'subagent', runId: 'run-fail', seq: 21 }), + submission, + ); + result.current.resetLive(submission); + + expect(store.get(pendingSubagentUsageFamily(convo)).output).toBe(0); + expect(store.get(subagentUsageFamily(convo)).output).toBe(0); + }); + + it('counts a resumed runโ€™s subagent events once', () => { + /** `resetLive` forgets the folded identities so a resume can rebuild + * pending; re-folding the same subagent event must not add its tokens a + * second time. */ + const convo = 'convo-sub-resume'; + const submission = { + userMessage: { messageId: 'u-resume', conversationId: convo }, + conversation: { conversationId: convo }, + }; + const { result } = renderHook(() => useUsageHandler()); + const store = getDefaultStore(); + const event = primaryUsage({ usage_type: 'subagent', runId: 'run-resume', seq: 31 }); + + result.current.usageHandler(event, submission); + result.current.resetLive(submission); + result.current.backfillUsage([event], submission); + result.current.finalizeUsage( + { + conversation: { conversationId: convo }, + responseMessage: { messageId: 'r-resume', conversationId: convo }, + }, + submission, + ); + + expect(store.get(subagentUsageFamily(convo)).output).toBe(3780); + }); }); diff --git a/client/src/hooks/SSE/useUsageHandler.ts b/client/src/hooks/SSE/useUsageHandler.ts index 268d0153c1..ab6bea2c4b 100644 --- a/client/src/hooks/SSE/useUsageHandler.ts +++ b/client/src/hooks/SSE/useUsageHandler.ts @@ -1,6 +1,6 @@ import { useRef, useMemo } from 'react'; import { getDefaultStore } from 'jotai'; -import { Constants, reconcileContextUsage, promptTokensFromUsage } from 'librechat-data-provider'; +import { Constants, reconcileContextUsageFromEvent } from 'librechat-data-provider'; import type { TMessage, TConversation, @@ -11,6 +11,7 @@ import type { ContextSnapshot } from '~/store/usage'; import { overheadKey, markUsageFolded, + migrateUsageFolded, liveTokensFamily, totalUsageFamily, removeUsageAtoms, @@ -19,13 +20,16 @@ import { calibrationFamily, pendingUsageFamily, branchTotalsFamily, - migrateUsageFolded, + subagentUsageFamily, + pendingSubagentUsageFamily, EMPTY_USAGE_TOTALS, contextSnapshotFamily, snapshotsByAnchorFamily, } from '~/store/usage'; import { sumBranch, + mergeUsage, + EMPTY_USAGE, setEntryUsage, upsertEntries, migrateIndex, @@ -130,7 +134,8 @@ export default function useUsageHandler(): UsageHandlers { const flushPendingInto = (convoKey: string, responseId: string | null) => { const pendingAtom = pendingUsageFamily(convoKey); const pending = jotai.get(pendingAtom); - if (responseId != null && pending.eventCount > 0) { + const keep = responseId != null && pending.eventCount > 0; + if (keep) { setEntryUsage(convoKey, responseId, { input: pending.input, output: pending.output, @@ -140,6 +145,16 @@ export default function useUsageHandler(): UsageHandlers { costKnown: pending.costKnown, }); } + /** The run's subagent share settles with the usage it is a subset of: + * committed to the conversation figure when the response keeps that + * usage, dropped with it otherwise. */ + const pendingSubAtom = pendingSubagentUsageFamily(convoKey); + const pendingSub = jotai.get(pendingSubAtom); + if (keep) { + const committedAtom = subagentUsageFamily(convoKey); + jotai.set(committedAtom, mergeUsage(jotai.get(committedAtom), pendingSub)); + } + jotai.set(pendingSubAtom, EMPTY_USAGE); jotai.set(pendingAtom, EMPTY_USAGE_TOTALS); }; @@ -147,6 +162,7 @@ export default function useUsageHandler(): UsageHandlers { const convoKey = getConvoKey(submission); jotai.set(contextSnapshotFamily(convoKey), { ...data, + completedOutputTokens: data.resumedOutputTokens ?? data.completedOutputTokens, anchorMessageId: submission.userMessage?.messageId ?? null, }); if (data.calibrationRatio != null && data.calibrationRatio > 0) { @@ -187,6 +203,9 @@ export default function useUsageHandler(): UsageHandlers { /** Displayed counts use the same normalized units billing does: input is * the uncached portion, output includes repaired completion tokens */ const units = normalizeUsageUnits(data); + const rawCost = data.cost; + const costKnown = typeof rawCost === 'number' && Number.isFinite(rawCost) && rawCost >= 0; + const cost = costKnown ? rawCost : 0; const pendingAtom = pendingUsageFamily(convoKey); const prev = jotai.get(pendingAtom); @@ -198,10 +217,27 @@ export default function useUsageHandler(): UsageHandlers { eventCount: prev.eventCount + 1, /** Authoritative per-event cost from the backend (premium tiers, cache * rates); absent when contextCost is disabled โ€” sums to 0 then */ - costUSD: prev.costUSD + (data.cost ?? 0), + costUSD: prev.costUSD + cost, /** Coverage is complete only if EVERY folded event carried a cost */ - costKnown: prev.costKnown && data.cost != null, + costKnown: prev.costKnown && costKnown, }); + /** Subagent calls are inside the rollup above (the backend's persisted + * rollup includes them too) โ€” track this run's share beside the pending + * usage it is a subset of, so the Totals row settles with that usage + * instead of surviving a discarded run or being folded twice by a + * resume. */ + if (data.usage_type === 'subagent') { + const subAtom = pendingSubagentUsageFamily(convoKey); + const subPrev = jotai.get(subAtom); + jotai.set(subAtom, { + input: subPrev.input + units.input, + output: subPrev.output + units.output, + cacheWrite: subPrev.cacheWrite + units.cacheWrite, + cacheRead: subPrev.cacheRead + units.cacheRead, + cost: subPrev.cost + cost, + costKnown: subPrev.costKnown && costKnown, + }); + } return true; }; @@ -222,8 +258,12 @@ export default function useUsageHandler(): UsageHandlers { if (snapshot.runId != null && data.runId != null && snapshot.runId !== data.runId) { return; } - const reconciled = reconcileContextUsage(snapshot, promptTokensFromUsage(data)); - jotai.set(snapshotAtom, { ...reconciled, anchorMessageId: snapshot.anchorMessageId }); + jotai.set(snapshotAtom, { + ...reconcileContextUsageFromEvent(snapshot, data), + /** Live output is held in confirmedRef until finalization. */ + completedOutputTokens: undefined, + anchorMessageId: snapshot.anchorMessageId, + }); }; const usageHandler: UsageHandlers['usageHandler'] = (data, submission) => { @@ -286,12 +326,15 @@ export default function useUsageHandler(): UsageHandlers { const convoKey = getConvoKey(submission); setLive(convoKey, 0); /** Terminal path with no salvageable response (stream error / intentional - * close): discard the in-flight pending usage so it can't merge into the - * next response. The user-stop path uses `attributePending` to keep it on - * the partial reply. Also forget the folded-event identities so a resume's - * `backfillUsage` can rebuild pending โ€” otherwise it sees them as already - * folded and the response's usage stays missing until a full reload. */ + * close): discard the in-flight pending usage โ€” and the subagent share + * inside it โ€” so neither can merge into the next response nor outlive the + * rollups it belongs to. The user-stop path uses `attributePending` to + * keep it on the partial reply. Also forget the folded-event identities + * so a resume's `backfillUsage` can rebuild pending โ€” otherwise it sees + * them as already folded and the response's usage stays missing until a + * full reload. */ jotai.set(pendingUsageFamily(convoKey), EMPTY_USAGE_TOTALS); + jotai.set(pendingSubagentUsageFamily(convoKey), EMPTY_USAGE); clearUsageFolded(convoKey); }; @@ -320,10 +363,12 @@ export default function useUsageHandler(): UsageHandlers { }; const seedLive: UsageHandlers['seedLive'] = (chars, submission) => { - if (chars <= 0) { + const convoKey = getConvoKey(submission); + /** A completed resumed call already carries exact output in its snapshot. + * Trailing text is the same output, not a new streaming delta. */ + if (chars <= 0 || jotai.get(contextSnapshotFamily(convoKey))?.completedOutputTokens != null) { return; } - const convoKey = getConvoKey(submission); streamCharsRef.current = chars; confirmedRef.current = 0; setLive(convoKey, estimateTokens(chars, jotai.get(calibrationFamily(convoKey)))); @@ -346,6 +391,11 @@ export default function useUsageHandler(): UsageHandlers { jotai.set(snapshotsByAnchorFamily(realId), jotai.get(snapshotsByAnchorFamily(fromKey))); jotai.set(pendingUsageFamily(realId), jotai.get(pendingUsageFamily(fromKey))); jotai.set(calibrationFamily(realId), jotai.get(calibrationFamily(fromKey))); + jotai.set(subagentUsageFamily(realId), jotai.get(subagentUsageFamily(fromKey))); + jotai.set( + pendingSubagentUsageFamily(realId), + jotai.get(pendingSubagentUsageFamily(fromKey)), + ); removeUsageAtoms(fromKey); } diff --git a/client/src/locales/en/translation.json b/client/src/locales/en/translation.json index 99516df07c..63b2b06f53 100644 --- a/client/src/locales/en/translation.json +++ b/client/src/locales/en/translation.json @@ -1116,8 +1116,20 @@ "com_ui_context_estimated": "Estimated", "com_ui_context_filter_sort": "Filter and Sort by Context", "com_ui_context_free": "Free space", + "com_ui_context_cached": "Cached", + "com_ui_context_cache_write": "Cache write", + "com_ui_context_dynamic_instructions": "Agent instructions", + "com_ui_context_pressure_warn": "Context window is filling up", + "com_ui_context_pressure_danger": "Context window is almost full", + "com_ui_context_largest_tool": "Largest tool: {{0}} ({{1}})", + "com_ui_context_runway": "โ‰ˆ{{0}} more turns at this growth", + "com_ui_context_compaction": "Summarizing now could reclaim โ‰ˆ{{0}} tokens", + "com_ui_context_insights": "Context insights", + "com_ui_context_tool_breakdown": "By tool", "com_ui_context_messages": "Messages", "com_ui_context_subagents": "Subagents", + "com_ui_context_subagents_all": "Subagents (all branches)", + "com_ui_context_tool_calls": "Tool calls", "com_ui_context_summary": "Summary", "com_ui_context_system": "System prompt", "com_ui_context_tools": "Tool schemas", diff --git a/client/src/store/usage.ts b/client/src/store/usage.ts index 15eeaffd89..157c9cb82a 100644 --- a/client/src/store/usage.ts +++ b/client/src/store/usage.ts @@ -86,6 +86,30 @@ export const totalUsageFamily = atomFamily((_conversationId: string) => /** Throttled in-flight output token estimate for the current model call */ export const liveTokensFamily = atomFamily((_conversationId: string) => atom(0)); +/** + * Subagent model calls COMMITTED to the conversation, accumulated from + * `usage_type: 'subagent'` events once their run settles. Deliberately excluded + * from branch/total provider usage (they bill separately and would double-count + * a turn), so they surface as their own Totals row. Session-scoped: not + * persisted, cleared on convo switch. + */ +export const subagentUsageFamily = atomFamily((_conversationId: string) => + atom(EMPTY_USAGE), +); + +/** + * The in-flight run's subagent share, held beside `pendingUsageFamily` and + * settled with it: committed into `subagentUsageFamily` when the response is + * finalized or a stop is attributed, discarded when the run ends with no + * salvageable response. Committing on arrival instead would leave a failed + * run's subagent tokens in the Totals after its usage was discarded from every + * rollup they are a subset of, and would count them twice when a resume + * re-folds the same events. + */ +export const pendingSubagentUsageFamily = atomFamily((_conversationId: string) => + atom(EMPTY_USAGE), +); + /** Last known provider-vs-estimate calibration ratio for the conversation */ export const calibrationFamily = atomFamily((_conversationId: string) => atom(1)); @@ -180,6 +204,8 @@ export function removeUsageAtoms(conversationId: string): void { pendingUsageFamily.remove(conversationId); totalUsageFamily.remove(conversationId); liveTokensFamily.remove(conversationId); + subagentUsageFamily.remove(conversationId); + pendingSubagentUsageFamily.remove(conversationId); calibrationFamily.remove(conversationId); foldedUsageKeys.delete(conversationId); } diff --git a/client/src/style.css b/client/src/style.css index 281578894b..0acb2f2e59 100644 --- a/client/src/style.css +++ b/client/src/style.css @@ -229,6 +229,7 @@ html { --series-5: 216 90 142; --series-6: 126 35 205; --series-7: 1 131 1; + --series-8: 63 81 181; /* Code syntax highlighting */ --syntax-text: var(--gray-800); @@ -347,6 +348,7 @@ html { --series-5: 213 82 130; --series-6: 171 104 254; --series-7: 80 167 49; + --series-8: 120 130 190; /* Code syntax highlighting. Comment and meta are the flattened equivalents of the 50% and 60% alpha whites this palette used before it was tokenized. */ diff --git a/client/src/utils/tokens.spec.ts b/client/src/utils/tokens.spec.ts index ed4c865a4a..6f5dcef76d 100644 --- a/client/src/utils/tokens.spec.ts +++ b/client/src/utils/tokens.spec.ts @@ -11,9 +11,12 @@ import { setEntryUsage, sumTotalUsage, prunedBranchTokens, + collectAnchorSeries, + latestExchangeTokens, findBranchSnapshotAnchor, estimateTokens, normalizeUsageUnits, + formatTokens, formatCost, groupToolTokens, countTrailingOutputChars, @@ -219,6 +222,69 @@ describe('token index', () => { expect(totals.tailEstTokens).toBe(5); }); + it('splits the tool-call share out of counted and count-less messages', () => { + buildIndex(CONVO, [ + /** Counted assistant turn: text 80 + tool payload (3+2+38=43 chars). + * Tool share 43/4 = 11, clamped to the entry's 30-token count โ†’ 11. */ + { + messageId: 'a1', + parentMessageId: Constants.NO_PARENT, + isCreatedByUser: false, + conversationId: CONVO, + tokenCount: 30, + content: [ + { type: 'text', text: 't'.repeat(80) }, + { type: 'tool_call', tool_call: { name: 'run', args: 'aa', output: 'o'.repeat(38) } }, + ], + } as unknown as TMessage, + /** Counted user turn with no tool parts: contributes nothing. */ + msg('u1', 'a1', true, 12), + /** Count-less assistant turn: tool chars 3+2+7=12 โ†’ 3 (โ‰ค its est 8). */ + { + messageId: 'a2', + parentMessageId: 'u1', + isCreatedByUser: false, + conversationId: CONVO, + text: 'z'.repeat(20), + content: [ + { type: 'tool_call', tool_call: { name: 'sub', args: 'aa', output: 'o'.repeat(7) } }, + ], + } as unknown as TMessage, + ]); + + const totals = sumBranch(CONVO, 'a2'); + expect(totals.estToolTokens).toBe(14); + /** a2 is the tail */ + expect(totals.tailEstToolTokens).toBe(3); + }); + + it('keeps a counted tail out of both tail estimates', () => { + /** A resumed partial response that already carries a `tokenCount` stays in + * `output`, so the estimate path may drop neither its estimate nor its tool + * share โ€” dropping the share alone left the message total holding tokens + * whose tool traffic had been removed from the split. */ + buildIndex(CONVO, [ + msg('u1', Constants.NO_PARENT, true, 12), + { + messageId: 'a1', + parentMessageId: 'u1', + isCreatedByUser: false, + conversationId: CONVO, + tokenCount: 40, + content: [ + { type: 'text', text: 't'.repeat(60) }, + { type: 'tool_call', tool_call: { name: 'run', args: 'aa', output: 'o'.repeat(38) } }, + ], + } as unknown as TMessage, + ]); + + const totals = sumBranch(CONVO, 'a1'); + expect(totals.output).toBe(40); + expect(totals.estToolTokens).toBe(11); + expect(totals.tailEstTokens).toBe(0); + expect(totals.tailEstToolTokens).toBe(0); + }); + describe('prunedBranchTokens (over-window mirror of getMessagesWithinTokenLimit)', () => { /** u1 โ† a1(huge, old) โ† u2 โ† a2(tail). */ const buildChain = () => @@ -232,18 +298,47 @@ describe('token index', () => { it('keeps the newest messages that fit and stops at the first overflow', () => { buildChain(); /** Budget 8: a2(2)+u2(2)=4 fit; a1(10) would overflow โ†’ pruned. */ - expect(prunedBranchTokens(CONVO, 'a2', 8, false)).toBe(4); + expect(prunedBranchTokens(CONVO, 'a2', 8, false)).toEqual({ tokens: 4, toolTokens: 0 }); }); it('returns the full branch sum when it fits the budget', () => { buildChain(); - expect(prunedBranchTokens(CONVO, 'a2', 100, false)).toBe(16); + expect(prunedBranchTokens(CONVO, 'a2', 100, false)).toEqual({ + tokens: 16, + toolTokens: 0, + }); }); it('skips the in-flight tail when excludeTail is set', () => { buildChain(); /** Skip a2; a1(10)+u2(2)+u1(2)=14 all fit under 100. */ - expect(prunedBranchTokens(CONVO, 'a2', 100, true)).toBe(14); + expect(prunedBranchTokens(CONVO, 'a2', 100, true)).toEqual({ + tokens: 14, + toolTokens: 0, + }); + }); + + it('returns the tool-call share of the kept messages', () => { + /** u1(counted, 4) โ† a2(tail): content-only estimate 11/4 = 3, tool chars + * 3+2+6=11 โ†’ 3. Kept total 7, tool share 3. */ + buildIndex(CONVO, [ + msg('u1', Constants.NO_PARENT, true, 4), + { + messageId: 'a2', + parentMessageId: 'u1', + isCreatedByUser: false, + conversationId: CONVO, + text: 'z'.repeat(4), + content: [ + { type: 'tool_call', tool_call: { name: 'sub', args: 'aa', output: 'o'.repeat(6) } }, + ], + } as unknown as TMessage, + ]); + + expect(prunedBranchTokens(CONVO, 'a2', 100, false)).toEqual({ + tokens: 7, + toolTokens: 3, + }); }); }); @@ -438,6 +533,12 @@ describe('estimateTokens', () => { expect(estimateTokens(0)).toBe(0); expect(estimateTokens(100, 0)).toBe(25); }); + + it('renders malformed token counts as a bounded zero', () => { + expect(formatTokens(Number.NaN)).toBe('0'); + expect(formatTokens(Number.POSITIVE_INFINITY)).toBe('0'); + expect(formatTokens(-10)).toBe('0'); + }); }); describe('normalizeUsageUnits', () => { @@ -615,6 +716,19 @@ describe('groupToolTokens', () => { expect(groupToolTokens(undefined)).toBe(EMPTY_TOOL_GROUPS); expect(groupToolTokens({ execute_code: 0 })).toEqual(EMPTY_TOOL_GROUPS); }); + + it('handles prototype-sensitive names and malformed tool counts safely', () => { + const counts = Object.fromEntries([ + ['__proto__', 7], + ['constructor', 5], + ['web_search', Number.NaN], + ]); + + expect(groupToolTokens(counts)).toEqual({ + ...EMPTY_TOOL_GROUPS, + system: 12, + }); + }); }); describe('countTrailingOutputChars', () => { @@ -702,6 +816,28 @@ describe('per-message usage index (branch + total)', () => { expect(usage.input).toBe(100); }); + it('ignores malformed persisted usage values instead of poisoning totals', () => { + buildIndex(CONVO, [ + msg('u1', Constants.NO_PARENT, true, 10), + responseMsg('a1', 'u1', 50, { + input: Number.NaN, + output: Number.POSITIVE_INFINITY, + cacheWrite: -2, + cacheRead: 4.8, + cost: Number.NaN, + }), + ]); + + expect(sumBranch(CONVO, 'a1').usage).toEqual({ + input: 0, + output: 0, + cacheWrite: 0, + cacheRead: 4, + cost: 0, + costKnown: false, + }); + }); + it('messages without metadata.usage contribute zero (backward compat)', () => { buildIndex(CONVO, [msg('u1', Constants.NO_PARENT, true, 10), msg('a1', 'u1', false, 50)]); expect(sumBranch(CONVO, 'a1').usage).toEqual(EMPTY_USAGE); @@ -835,4 +971,119 @@ describe('per-message usage index (branch + total)', () => { expect(sumBranch('missing-convo', 'x')).toBe(EMPTY_BRANCH); expect(EMPTY_BRANCH.usage).toEqual(EMPTY_USAGE); }); + + it('collectAnchorSeries reads used tokens per anchor, oldest โ†’ newest', () => { + buildIndex(CONVO, [ + msg('u1', Constants.NO_PARENT, true, 4), + msg('a1', 'u1', false, 10), + msg('u2', 'a1', true, 4), + msg('a2', 'u2', false, 20), + msg('u3', 'a2', true, 4), + msg('a3', 'u3', false, 30), + ]); + const snap = (remaining: number) => ({ + contextBudget: 1000, + remainingContextTokens: remaining, + breakdown: { maxContextTokens: 1000 }, + }); + const anchors = new Map([ + ['a1', snap(960)], + ['a2', snap(920)], + ['a3', snap(860)], + ]); + + const series = collectAnchorSeries(CONVO, 'a3', anchors); + /** used = budget โˆ’ remaining: 40 โ†’ 80 โ†’ 140, oldest first */ + expect(series).toEqual([ + { used: 40, basis: 'remaining' }, + { used: 80, basis: 'remaining' }, + { used: 140, basis: 'remaining' }, + ]); + expect(collectAnchorSeries(CONVO, 'a3', new Map())).toEqual([]); + }); + + it('collectAnchorSeries derives a snapshot saved without remaining headroom', () => { + buildIndex(CONVO, [ + msg('u1', Constants.NO_PARENT, true, 4), + msg('a1', 'u1', false, 10), + msg('u2', 'a1', true, 4), + msg('a2', 'u2', false, 20), + ]); + const anchors = new Map([ + /** The older shape: a budget and a breakdown, no remaining count. Read as + * zero remaining it would claim the whole 1000-token window was spent. */ + [ + 'a1', + { + contextBudget: 1000, + breakdown: { maxContextTokens: 1000, instructionTokens: 100, messageTokens: 60 }, + }, + ], + /** Nothing to read at all โ€” skipped rather than counted as a full window. */ + ['a2', { contextBudget: 1000, breakdown: { maxContextTokens: 1000 } }], + ]); + + expect(collectAnchorSeries(CONVO, 'a2', anchors)).toEqual([{ used: 160, basis: 'breakdown' }]); + }); + + it('latestExchangeTokens sums the tail response and its user turn', () => { + buildIndex(CONVO, [ + msg('u1', Constants.NO_PARENT, true, 12), + msg('a1', 'u1', false, 40), + msg('u2', 'a1', true, 9), + msg('a2', 'u2', false, 50), + ]); + + expect(latestExchangeTokens(CONVO, 'a2', false)).toBe(59); + /** In-flight tail rides liveTokens โ€” only the user turn remains */ + expect(latestExchangeTokens(CONVO, 'a2', true)).toBe(9); + }); + + it('keeps tool results outside model completion counts in the latest exchange', () => { + const response = { + messageId: 'a1', + parentMessageId: 'u1', + conversationId: CONVO, + isCreatedByUser: false, + tokenCount: 40, + content: [ + { + type: 'tool_call', + tool_call: { + name: 'read_file', + args: 'a'.repeat(80), + output: 'r'.repeat(2000), + }, + }, + ], + } as TMessage; + buildIndex(CONVO, [msg('u1', Constants.NO_PARENT, true, 10), response]); + expect(latestExchangeTokens(CONVO, 'a1', false)).toBe(550); + expect(latestExchangeTokens(CONVO, 'a1', true)).toBe(10); + buildIndex(CONVO, [msg('u1', Constants.NO_PARENT, true, 10), { ...response, tokenCount: 0 }]); + expect(latestExchangeTokens(CONVO, 'a1', false)).toBe(532); + }); + + it('latestExchangeTokens drops a summarizing turnโ€™s summary completion', () => { + /** The backend folds the summarization pass into the response's + * `tokenCount`, while the snapshot holds those tokens in `summaryTokens` + * instead of `messageTokens`. Subtracting the whole tail would remove a + * summary the message total never carried. */ + buildIndex(CONVO, [ + msg('u1', Constants.NO_PARENT, true, 9), + { + messageId: 'a1', + parentMessageId: 'u1', + isCreatedByUser: false, + conversationId: CONVO, + tokenCount: 500, + text: 'answer', + metadata: { summaryUsedTokens: 4000 }, + } as unknown as TMessage, + ]); + + expect(latestExchangeTokens(CONVO, 'a1', false, 300)).toBe(209); + /** A turn that did not summarize keeps its whole response. */ + expect(latestExchangeTokens(CONVO, 'a1', false)).toBe(509); + }); }); diff --git a/client/src/utils/tokens.ts b/client/src/utils/tokens.ts index 8ee65cc3dc..c7205e15d1 100644 --- a/client/src/utils/tokens.ts +++ b/client/src/utils/tokens.ts @@ -1,5 +1,11 @@ import { Tools, Constants, inputTokensIncludesCache } from 'librechat-data-provider'; -import type { TMessage, TResponseUsage, TTokenUsageEvent } from 'librechat-data-provider'; +import type { + TMessage, + TResponseUsage, + TTokenUsageEvent, + TContextUsageEvent, + TTokenBudgetBreakdown, +} from 'librechat-data-provider'; /** Provider-reported usage of one response, in display units (post-normalize). */ export interface BranchUsage { @@ -24,6 +30,17 @@ export const EMPTY_USAGE: BranchUsage = { cost: 0, costKnown: true, }; +/** Token counts come from persisted/provider payloads. Keep malformed values + * out of every aggregate so one bad event cannot turn the meter into NaN. */ +export function normalizeTokenCount(value: unknown): number { + return typeof value === 'number' && Number.isFinite(value) && value > 0 + ? Math.min(Math.floor(value), Number.MAX_SAFE_INTEGER) + : 0; +} + +function hasKnownCost(value: unknown): value is number { + return typeof value === 'number' && Number.isFinite(value) && value >= 0; +} export interface TokenEntry { tokenCount: number; @@ -32,6 +49,12 @@ export interface TokenEntry { * text+quotes, since the send path ignores their stored count). Includes * tool-call name/args/output. Mutually exclusive with a counted `tokenCount`. */ estTokens: number; + /** Char/4 estimate of the tool-call share of the body (tool_call parts' + * name/args/output), for the estimate path's "Tool calls" split. Raw โ€” sum + * sites clamp it to the entry's actual contribution. */ + estToolTokens: number; + /** Tool results are outside a counted responseโ€™s model-completion total. */ + estToolResultTokens: number; isCreatedByUser: boolean; parentMessageId: string | null; /** Per-response provider usage from `metadata.usage` (response messages only) */ @@ -54,10 +77,18 @@ export interface BranchTotals { /** Uncalibrated estimate sum for count-less branch messages (imports / * pre-feature). Kept separate so known counts aren't re-estimated. */ estTokens: number; - /** The tail (latest) message's own `estTokens`. When a stream is live the tail - * is the in-flight response, already covered by `liveTokens`, so the estimate - * path excludes this to avoid double-counting a resumed/partial response. */ + /** The tail (latest) message's UNCOUNTED estimate โ€” its own `estTokens` when + * it has no `tokenCount`, else 0. A live tail rides on `liveTokens`, so the + * estimate path drops this; a resumed partial that already carries a count + * stays in `output`, so nothing may be dropped for it. */ tailEstTokens: number; + /** Tool-call share of the branch estimate (`estToolTokens`, clamped per + * message to its contribution) โ€” the estimate path's "Tool calls" split. */ + estToolTokens: number; + /** The tail's clamped tool share on the same basis as `tailEstTokens`: 0 for a + * counted tail, so the split never drops tool traffic whose tokens the + * message total keeps. */ + tailEstToolTokens: number; tailId: string | null; /** Whether the latest run's anchor message is on this branch */ containsAnchor: boolean; @@ -77,6 +108,8 @@ export const EMPTY_BRANCH: BranchTotals = { total: 0, estTokens: 0, tailEstTokens: 0, + estToolTokens: 0, + tailEstToolTokens: 0, tailId: null, containsAnchor: false, usage: EMPTY_USAGE, @@ -112,13 +145,14 @@ function readPersistedUsage(message: Partial): BranchUsage | undefined } const persisted = usage as TResponseUsage; return { - input: persisted.input ?? 0, - output: persisted.output ?? 0, - cacheWrite: persisted.cacheWrite ?? 0, - cacheRead: persisted.cacheRead ?? 0, - cost: persisted.cost ?? 0, - /** Cost is omitted when saved with `contextCost` off โ€” don't render $0.00 */ - costKnown: typeof persisted.cost === 'number', + input: normalizeTokenCount(persisted.input), + output: normalizeTokenCount(persisted.output), + cacheWrite: normalizeTokenCount(persisted.cacheWrite), + cacheRead: normalizeTokenCount(persisted.cacheRead), + cost: hasKnownCost(persisted.cost) ? persisted.cost : 0, + /** Cost is omitted (or malformed) when saved with `contextCost` off โ€” don't + * render a misleading $0.00. */ + costKnown: hasKnownCost(persisted.cost), }; } @@ -152,8 +186,8 @@ function addUsage(target: BranchUsage, usage?: BranchUsage): void { } /** Chars of a content part's text, handling both the string and `{ value }` forms. - * Reasoning (`think`) and error parts are excluded โ€” the send path strips them - * before counting, so they aren't part of the next call's context. */ + * Reasoning (`think`) and error parts are excluded โ€” the send path strips them + * before counting, so they aren't part of the next call's context. */ function partTextChars(part: unknown): number { if (part == null || typeof part !== 'object') { return 0; @@ -172,7 +206,10 @@ function partTextChars(part: unknown): number { if (typeof call.args === 'string') { chars += call.args.length; } else if (call.args != null) { - chars += JSON.stringify(call.args).length; + const serialized = JSON.stringify(call.args); + if (typeof serialized === 'string') { + chars += serialized.length; + } } if (typeof call.output === 'string') { chars += call.output.length; @@ -193,6 +230,46 @@ function partTextChars(part: unknown): number { return 0; } +/** Char length of a message's tool-call payload โ€” tool_call parts' name, args + * and output โ€” the estimate path's split of message body into conversation vs + * tool-call usage. Zero for plain-text bodies. */ +function messageToolChars(message: Partial): { chars: number; resultChars: number } { + if (!Array.isArray(message.content)) { + return { chars: 0, resultChars: 0 }; + } + let chars = 0; + let resultChars = 0; + for (const part of message.content) { + if (part == null || typeof part !== 'object') { + continue; + } + if ('type' in part && part.type === 'tool_call' && 'tool_call' in part) { + const call = part.tool_call; + if (call == null || typeof call !== 'object') { + continue; + } + if ('name' in call && typeof call.name === 'string') { + chars += call.name.length; + } + if ('args' in call) { + if (typeof call.args === 'string') { + chars += call.args.length; + } else { + const serialized = JSON.stringify(call.args ?? null); + if (typeof serialized === 'string') { + chars += serialized.length; + } + } + } + if ('output' in call && typeof call.output === 'string') { + chars += call.output.length; + resultChars += call.output.length; + } + } + } + return { chars, resultChars }; +} + /** Char length of a message's rendered text, for estimating count-less messages. * Prefer structured `content` when present โ€” the send path formats from it (incl. * tool calls), so a message carrying both `text` and `content` (e.g. a stopped @@ -226,8 +303,8 @@ function quoteChars(message: Partial): number { } function toEntry(message: Partial): TokenEntry { - const summaryUsedTokens = message.metadata?.summaryUsedTokens; - const tokenCount = typeof message.tokenCount === 'number' ? message.tokenCount : 0; + const summaryUsedTokens = normalizeTokenCount(message.metadata?.summaryUsedTokens); + const tokenCount = normalizeTokenCount(message.tokenCount); const isCreatedByUser = message.isCreatedByUser === true; const quoted = isCreatedByUser && Array.isArray(message.quotes) && message.quotes.length > 0; /** A quoted user turn's stored `tokenCount` is unreliable: a text-only Save edit @@ -242,16 +319,16 @@ function toEntry(message: Partial): TokenEntry { } else if (tokenCount === 0) { estTokens = Math.round(messageChars(message) / 4); } + const toolChars = messageToolChars(message); return { tokenCount: quoted ? 0 : tokenCount, estTokens, + estToolTokens: Math.round(toolChars.chars / 4), + estToolResultTokens: Math.round(toolChars.resultChars / 4), isCreatedByUser, parentMessageId: message.parentMessageId ?? null, usage: readPersistedUsage(message), - summaryUsedTokens: - typeof summaryUsedTokens === 'number' && summaryUsedTokens > 0 - ? summaryUsedTokens - : undefined, + summaryUsedTokens: summaryUsedTokens > 0 ? summaryUsedTokens : undefined, }; } @@ -335,12 +412,30 @@ export function sumBranch( return EMPTY_BRANCH; } - const totals = { input: 0, output: 0, counted: 0, total: 0, estTokens: 0, containsAnchor: false }; - /** The in-flight response, when streaming, is the branch tail and is covered by - * `liveTokens`; expose its estimate so the estimate path can drop it. */ - const tailEstTokens = index.get(tailId)?.estTokens ?? 0; - const usage: BranchUsage = { ...EMPTY_USAGE }; + const totals = { + input: 0, + output: 0, + counted: 0, + total: 0, + estTokens: 0, + estToolTokens: 0, + containsAnchor: false, + }; + /** The in-flight response, when streaming, is the branch tail and is covered + * by `liveTokens`; expose its estimate so the estimate path can drop it. A + * tail that already carries a `tokenCount` (a resumed partial response) is + * counted in `input`/`output` instead, and its tool share is inside + * `estToolTokens` clamped to that count โ€” dropping either would leave the + * message total holding tokens whose tool share had been removed, so both + * tail figures are zero for a counted tail. */ + const tailEntry = index.get(tailId); + const tailCounted = tailEntry != null && tailEntry.tokenCount > 0; + const tailEstTokens = tailCounted ? 0 : (tailEntry?.estTokens ?? 0); + const tailEstToolTokens = tailCounted + ? 0 + : Math.min(tailEntry?.estToolTokens ?? 0, tailEntry?.estTokens ?? 0); let summaryBaseline = 0; + const usage: BranchUsage = { ...EMPTY_USAGE }; /** Once a summary marker is crossed, older turns are out of the CONTEXT WINDOW * (subsumed by the baseline) โ€” but their provider spend still happened, so the * usage/cost walk continues to the root while context counting stops. */ @@ -372,6 +467,12 @@ export function sumBranch( } else if (!contextCapped && entry.estTokens > 0) { totals.estTokens += entry.estTokens; } + /** Tool-call share of this entry's contribution โ€” a subset of the + * input/output/estimated rows, so it never adds to the gauge. */ + if (!contextCapped) { + const contribution = entry.tokenCount > 0 ? entry.tokenCount : entry.estTokens; + totals.estToolTokens += Math.min(entry.estToolTokens, contribution); + } /** Cost/usage is cumulative spend โ€” never truncated at the summary boundary. */ addUsage(usage, entry.usage); /** This response's turn compacted the history: its own output is counted @@ -384,7 +485,7 @@ export function sumBranch( currentId = entry.parentMessageId; } - return { ...totals, tailEstTokens, tailId, usage, summaryBaseline }; + return { ...totals, tailEstTokens, tailEstToolTokens, tailId, usage, summaryBaseline }; } /** @@ -398,20 +499,22 @@ export function sumBranch( * `budget` is the message window (max minus the always-sent summary baseline); * when `excludeTail`, the in-flight tail response is skipped (it rides on * `liveTokens`). Per-message contribution matches `sumBranch`: stored `tokenCount` - * when counted, else the char-based `estTokens`. + * when counted, else the char-based `estTokens`. Returns the kept total plus the + * tool-call share of what was kept, so the estimate can split the Messages row. */ export function prunedBranchTokens( conversationId: string, tailId: string | null | undefined, budget: number, excludeTail: boolean, -): number { +): { tokens: number; toolTokens: number } { const index = registry.get(conversationId); if (!index || !tailId || budget <= 0) { - return 0; + return { tokens: 0, toolTokens: 0 }; } let total = 0; + let toolTotal = 0; let currentId: string | null = tailId; let guard = index.size; let isTail = true; @@ -429,6 +532,7 @@ export function prunedBranchTokens( break; } total += contribution; + toolTotal += Math.min(entry.estToolTokens, contribution); } /** Pre-summary turns are subsumed by the baseline the caller already reserved, * so stop after counting the summarizing turn โ€” mirrors `sumBranch`. */ @@ -437,7 +541,182 @@ export function prunedBranchTokens( } currentId = entry.parentMessageId; } - return total; + return { tokens: total, toolTokens: toolTotal }; +} + +/** + * One persisted snapshot's used-context reading, with the basis it was measured + * on. `remaining` is the authoritative pre-invoke figure (`budget โˆ’ remaining`); + * `breakdown` is the instruction+messages sum a snapshot saved before + * `remainingContextTokens` existed still supports. The two measure different + * quantities (the backend's remaining covers content the breakdown does not), + * so a growth delta may only compare readings of the same basis. + */ +export interface AnchorReading { + used: number; + basis: 'remaining' | 'breakdown'; + configuration?: string; +} + +/** Compare the measured call configuration, including fixed prompt overhead. + * Older snapshots without call identity cannot establish comparability. */ +export function snapshotConfiguration( + snapshot: Pick & { + breakdown?: Partial; + }, +): string | undefined { + if (!snapshot.model || !snapshot.provider) { + return undefined; + } + return JSON.stringify([ + snapshot.model, + snapshot.provider, + snapshot.agentId, + snapshot.contextBudget ?? snapshot.breakdown?.maxContextTokens, + snapshot.breakdown?.instructionTokens, + snapshot.breakdown?.toolSchemaTokens, + ]); +} + +/** + * Used-token readings for every persisted snapshot on the viewed branch, oldest + * โ†’ newest (the walk is tailโ†’root, so results are reversed). Drives the growth + * sparkline and the runway projection in the breakdown. Stops after a summarized + * response โ€” older snapshots describe discarded history, mirroring `sumBranch`. + * A snapshot that carries neither basis is skipped rather than reported as + * having consumed its entire budget. + */ +export function collectAnchorSeries( + conversationId: string, + tailId: string | null | undefined, + anchors: ReadonlyMap, +): AnchorReading[] { + const index = registry.get(conversationId); + if (!index || !tailId || anchors.size === 0) { + return []; + } + + const series: AnchorReading[] = []; + let currentId: string | null = tailId; + let guard = index.size; + + while (currentId && currentId !== Constants.NO_PARENT && guard-- > 0) { + const entry: TokenEntry | undefined = index.get(currentId); + if (!entry) { + break; + } + const snapshot = anchors.get(currentId) as + | (Partial> & { + breakdown?: Partial; + }) + | undefined; + if (snapshot != null) { + const budget = normalizeTokenCount( + snapshot.contextBudget ?? snapshot.breakdown?.maxContextTokens, + ); + const configuration = snapshotConfiguration(snapshot); + /** Same precedence as the render path's `baseUsed`: the backend's + * remaining headroom when it was saved, else the breakdown sum. */ + if (snapshot.remainingContextTokens != null && budget > 0) { + const remaining = normalizeTokenCount(snapshot.remainingContextTokens); + series.push({ used: Math.max(0, budget - remaining), basis: 'remaining', configuration }); + } else { + const used = + normalizeTokenCount( + snapshot.effectiveInstructionTokens ?? snapshot.breakdown?.instructionTokens, + ) + normalizeTokenCount(snapshot.breakdown?.messageTokens); + if (used > 0) { + series.push({ used, basis: 'breakdown', configuration }); + } + } + } + if (entry.summaryUsedTokens != null && entry.summaryUsedTokens > 0) { + break; + } + currentId = entry.parentMessageId; + } + return series.reverse(); +} + +/** + * The branch's largest messages (context contributions), largest first โ€” the + * "what is eating my window" list. Same contribution rule as `sumBranch`: + * stored `tokenCount` when counted, else the char-based `estTokens`. + */ +export function topBranchMessages( + conversationId: string, + tailId: string | null | undefined, + limit = 3, +): Array<{ role: 'user' | 'assistant'; tokens: number }> { + const index = registry.get(conversationId); + if (!index || !tailId) { + return []; + } + + const items: Array<{ role: 'user' | 'assistant'; tokens: number }> = []; + let currentId: string | null = tailId; + let guard = index.size; + let contextCapped = false; + + while (currentId && currentId !== Constants.NO_PARENT && guard-- > 0) { + const entry: TokenEntry | undefined = index.get(currentId); + if (!entry) { + break; + } + if (!contextCapped) { + const contribution = entry.tokenCount > 0 ? entry.tokenCount : entry.estTokens; + if (contribution > 0) { + items.push({ role: entry.isCreatedByUser ? 'user' : 'assistant', tokens: contribution }); + } + } + if (!contextCapped && entry.summaryUsedTokens != null && entry.summaryUsedTokens > 0) { + contextCapped = true; + } + currentId = entry.parentMessageId; + } + return items.sort((a, b) => b.tokens - a.tokens).slice(0, limit); +} + +/** + * Tokens of the most recent exchange (the tail response plus the user turn it + * answers) โ€” what a summarization would KEEP. `excludeTail` skips an in-flight + * tail (it rides on `liveTokens`), leaving the previous complete exchange. + * The compaction preview is context tokens minus this. + * + * `summaryOutputTokens` removes an internal summarization's completion from a + * tail whose turn compacted: the backend folds that pass into the response's + * `tokenCount`, while the snapshot keeps it in `breakdown.summaryTokens` rather + * than `messageTokens`. Leaving it in would subtract a summary the message + * total never carried, understating the reclaim by the size of the summary. + */ +export function latestExchangeTokens( + conversationId: string, + tailId: string | null | undefined, + excludeTail: boolean, + summaryOutputTokens = 0, +): number { + const index = registry.get(conversationId); + if (!index || !tailId) { + return 0; + } + const contribution = (entry: TokenEntry | undefined): number => { + if (entry == null) { + return 0; + } + /** Completion counts include call arguments, but tool results are separate. + * Count-less content already estimates both. */ + return entry.tokenCount > 0 ? entry.tokenCount + entry.estToolResultTokens : entry.estTokens; + }; + + const tailEntry = index.get(tailId); + const parentEntry = index.get(tailEntry?.parentMessageId ?? ''); + let tail = excludeTail ? 0 : contribution(tailEntry); + /** Only a turn that actually summarized carries that completion; the marker + * persists on the response whose turn compacted. */ + if (tail > 0 && tailEntry?.summaryUsedTokens != null && tailEntry.summaryUsedTokens > 0) { + tail = Math.max(0, tail - normalizeTokenCount(summaryOutputTokens)); + } + return tail + contribution(parentEntry); } /** @@ -465,6 +744,19 @@ export function sumTotalUsage(conversationId: string): BranchUsage { * that, including a sibling transiently dropped from the cache on regenerate). */ export function setEntryUsage(conversationId: string, messageId: string, usage: BranchUsage): void { + const costKnown = + usage.costKnown === true && + typeof usage.cost === 'number' && + Number.isFinite(usage.cost) && + usage.cost >= 0; + const safeUsage: BranchUsage = { + input: normalizeTokenCount(usage.input), + output: normalizeTokenCount(usage.output), + cacheWrite: normalizeTokenCount(usage.cacheWrite), + cacheRead: normalizeTokenCount(usage.cacheRead), + cost: costKnown ? usage.cost : 0, + costKnown, + }; /** Remember it durably first so a later rebuild โ€” or a transient cache drop * during regenerate โ€” can restore it even when the entry isn't present yet. */ let history = usageHistory.get(conversationId); @@ -472,10 +764,10 @@ export function setEntryUsage(conversationId: string, messageId: string, usage: history = new Map(); usageHistory.set(conversationId, history); } - history.set(messageId, usage); + history.set(messageId, safeUsage); const entry = registry.get(conversationId)?.get(messageId); if (entry) { - entry.usage = usage; + entry.usage = safeUsage; } } @@ -550,8 +842,9 @@ export function groupToolTokens( } const deferred = new Set(deferredToolNames ?? []); const groups = { ...EMPTY_TOOL_GROUPS }; - for (const [name, tokens] of Object.entries(toolTokenCounts)) { - if (tokens <= 0) { + for (const [name, rawTokens] of Object.entries(toolTokenCounts)) { + const tokens = normalizeTokenCount(rawTokens); + if (tokens === 0) { continue; } if (name === Tools.skill) { @@ -614,11 +907,14 @@ export function countTrailingOutputChars(content?: unknown[] | null): number { /** Rough live estimate for streaming text, calibrated by the last known provider ratio */ export function estimateTokens(charCount: number, calibrationRatio = 1): number { - if (charCount <= 0) { - return 0; - } - const ratio = calibrationRatio > 0 ? calibrationRatio : 1; - return Math.round((charCount / 4) * ratio); + const chars = normalizeTokenCount(charCount); + const ratio = + typeof calibrationRatio === 'number' && + Number.isFinite(calibrationRatio) && + calibrationRatio > 0 + ? calibrationRatio + : 1; + return normalizeTokenCount(Math.round((chars / 4) * ratio)); } /** Billable token quantities of one or more model calls, normalized for pricing */ @@ -633,20 +929,20 @@ export interface CostUnits { * Normalizes one call's usage into billable units, mirroring the backend's * authoritative `splitUsage`/`resolveCompletionTokens` * (packages/api/src/agents/usage.ts): - * - cache classification is by provider, not magnitude โ€” Anthropic/Bedrock - * keep cache additive (input is uncached-only); subset providers fold - * cache into `input_tokens`. Falls back to a magnitude heuristic only when + * - cache classification is by provider, not magnitude โ€” Bedrock keeps cache + * additive (input is uncached-only); Anthropic and OpenAI fold cache into + * `input_tokens`. Falls back to a magnitude heuristic only when * the provider is unknown. * - completion is repaired for providers (e.g. Vertex) that under-report * `output_tokens` but carry the gap in `total_tokens`. * Applied per event so units stay correct when summed across calls. */ export function normalizeUsageUnits(usage: TTokenUsageEvent): CostUnits { - const rawInput = usage.input_tokens ?? 0; - const rawOutput = usage.output_tokens ?? 0; - const total = usage.total_tokens ?? 0; - const cacheWrite = usage.input_token_details?.cache_creation ?? 0; - const cacheRead = usage.input_token_details?.cache_read ?? 0; + const rawInput = normalizeTokenCount(usage.input_tokens); + const rawOutput = normalizeTokenCount(usage.output_tokens); + const total = normalizeTokenCount(usage.total_tokens); + const cacheWrite = normalizeTokenCount(usage.input_token_details?.cache_creation); + const cacheRead = normalizeTokenCount(usage.input_token_details?.cache_read); const includesCache = usage.provider != null @@ -665,11 +961,13 @@ export function normalizeUsageUnits(usage: TTokenUsageEvent): CostUnits { }; } +const tokenFormatter = new Intl.NumberFormat(undefined, { + notation: 'compact', + maximumFractionDigits: 1, +}); + export function formatTokens(count: number): string { - const formatted = new Intl.NumberFormat(undefined, { - notation: 'compact', - maximumFractionDigits: 1, - }).format(count); + const formatted = tokenFormatter.format(normalizeTokenCount(count)); return formatted.replace(/\.0(?=[A-Za-z]|$)/, ''); } @@ -753,7 +1051,7 @@ export function formatCost(usd: number, currency: CurrencyConfig = DEFAULT_CURRE } const base = maxFractionDigits(code); - const amount = usd * rate; + const amount = (Number.isFinite(usd) && usd > 0 ? usd : 0) * rate; /** The currency's own minor unit โ€” USD/EURโ†’0.01, JPYโ†’1, KWDโ†’0.001. */ const smallest = Math.pow(10, -base); diff --git a/e2e/specs/mock/scenarios/persisted-context-usage.spec.ts b/e2e/specs/mock/scenarios/persisted-context-usage.spec.ts new file mode 100644 index 0000000000..9b81a5a9ca --- /dev/null +++ b/e2e/specs/mock/scenarios/persisted-context-usage.spec.ts @@ -0,0 +1,261 @@ +import { randomUUID } from 'crypto'; +import { expect, test } from '@playwright/test'; +import { getE2EUser } from '../../../setup/user'; +import { deleteConversations, deleteMessagesByConversation, withMongo } from '../db'; + +const NO_PARENT = '00000000-0000-0000-0000-000000000000'; +const userEmail = getE2EUser().email; +const cleanupConversationIds: string[] = []; + +async function insertConversation( + conversationId: string, + title: string, + messages: Record[], +): Promise { + await withMongo(async (db) => { + const user = await db.collection('users').findOne({ email: userEmail }); + if (!user) { + throw new Error(`E2E seed: user "${userEmail}" not found`); + } + + const now = new Date(); + await db.collection('conversations').insertOne({ + conversationId, + title, + user: user._id.toString(), + endpoint: 'Mock Provider A', + model: 'mock-model-a', + maxContextTokens: 300, + isArchived: false, + createdAt: now, + updatedAt: now, + __v: 0, + }); + + await db.collection('messages').insertMany( + messages.map((message, index) => ({ + ...message, + conversationId, + user: user._id.toString(), + endpoint: 'Mock Provider A', + model: 'mock-model-a', + error: false, + unfinished: false, + isTemporary: false, + createdAt: new Date(now.getTime() + index * 1000), + updatedAt: new Date(now.getTime() + index * 1000), + __v: 0, + })), + ); + }); +} + +test.afterEach(async () => { + const conversationIds = cleanupConversationIds.splice(0); + if (conversationIds.length === 0) { + return; + } + + try { + await deleteMessagesByConversation(conversationIds); + } finally { + await deleteConversations(conversationIds); + } +}); + +test.describe('persisted context usage', () => { + /** A snapshot persisted before `remainingContextTokens` existed carries only a + * budget and a breakdown. Reading the absent remaining count as zero scored + * such a snapshot as having spent its whole window, so an old conversation + * opened to a full meter. The used figure has to come from the breakdown + * instead: instructions + messages of the deepest snapshot on the branch. */ + test('reads used context from a legacy snapshotโ€™s breakdown @scenario:legacy-snapshot-shows-derived-used-context', async ({ + page, + }) => { + test.setTimeout(120000); + const conversationId = randomUUID(); + cleanupConversationIds.push(conversationId); + + await insertConversation(conversationId, 'Legacy context runway fixture', [ + { + messageId: 'legacy-user-1', + parentMessageId: NO_PARENT, + text: 'First legacy prompt', + isCreatedByUser: true, + sender: 'User', + tokenCount: 8, + }, + { + messageId: 'legacy-assistant-1', + parentMessageId: 'legacy-user-1', + text: 'First legacy reply', + isCreatedByUser: false, + sender: 'Assistant', + tokenCount: 4, + metadata: { + usage: { input: 10, output: 4, cacheRead: 0, cacheWrite: 0 }, + contextUsage: { + runId: 'legacy-run-1', + contextBudget: 300, + breakdown: { + maxContextTokens: 300, + instructionTokens: 20, + systemMessageTokens: 0, + dynamicInstructionTokens: 0, + toolSchemaTokens: 0, + summaryTokens: 0, + toolCount: 0, + messageCount: 2, + messageTokens: 100, + availableForMessages: 280, + }, + }, + }, + }, + { + messageId: 'legacy-user-2', + parentMessageId: 'legacy-assistant-1', + text: 'Second legacy prompt', + isCreatedByUser: true, + sender: 'User', + tokenCount: 8, + }, + { + messageId: 'legacy-assistant-2', + parentMessageId: 'legacy-user-2', + text: 'Second legacy reply', + isCreatedByUser: false, + sender: 'Assistant', + tokenCount: 4, + metadata: { + usage: { input: 14, output: 5, cacheRead: 0, cacheWrite: 0 }, + contextUsage: { + runId: 'legacy-run-2', + contextBudget: 300, + breakdown: { + maxContextTokens: 300, + instructionTokens: 20, + systemMessageTokens: 0, + dynamicInstructionTokens: 0, + toolSchemaTokens: 0, + summaryTokens: 0, + toolCount: 0, + messageCount: 4, + messageTokens: 160, + availableForMessages: 280, + }, + }, + }, + }, + ]); + + await page.goto(`/c/${conversationId}`, { timeout: 30000 }); + await expect(page.getByText('Second legacy reply')).toBeVisible({ timeout: 30000 }); + + const gauge = page.getByTestId('token-usage'); + await expect(gauge).toBeVisible({ timeout: 30000 }); + await gauge.click(); + const popover = page.getByRole('region', { name: 'Context usage' }); + await expect(popover).toBeVisible({ timeout: 10000 }); + + const breakdownToggle = popover.getByTestId('context-breakdown-toggle'); + if ((await breakdownToggle.getAttribute('aria-expanded')) === 'false') { + await breakdownToggle.click(); + } + await expect(breakdownToggle).toHaveAttribute('aria-expanded', 'true'); + + /** 20 instruction + 160 message tokens of the 300-token budget: the deepest + * legacy snapshot on the branch, derived from its breakdown. Scoring the + * missing remaining count as zero would read 300 / 300 (100%) here. */ + await expect(breakdownToggle).toContainText('180 / 300 (60%)'); + await expect(popover.getByRole('progressbar')).toHaveAttribute('aria-valuenow', '60'); + + /** The snapshot path renders, so the reading really is the persisted + * snapshot and not the client-side estimate, and the untouched remainder + * is the budget minus that derived total. */ + const breakdown = popover.getByTestId('context-breakdown'); + await expect(breakdown).toBeVisible({ timeout: 10000 }); + await expect(popover.getByTestId('context-estimate')).toHaveCount(0); + const freeSpace = breakdown.locator('div.flex', { hasText: 'Free space' }).first(); + await expect(freeSpace).toContainText('120'); + }); + + test('nests estimated tool traffic beneath message totals @scenario:estimate-path-nests-the-tool-share', async ({ + page, + }) => { + test.setTimeout(120000); + const conversationId = randomUUID(); + cleanupConversationIds.push(conversationId); + + await insertConversation(conversationId, 'Estimated tool share fixture', [ + { + messageId: 'estimate-user-1', + parentMessageId: NO_PARENT, + text: 'Estimate prompt', + isCreatedByUser: true, + sender: 'User', + tokenCount: 8, + }, + { + messageId: 'estimate-assistant-1', + parentMessageId: 'estimate-user-1', + text: 'Tool result', + isCreatedByUser: false, + sender: 'Assistant', + tokenCount: 20, + content: [ + { + type: 'tool_call', + tool_call: { + id: 'estimate-call-1', + name: 'lookup', + args: '{"q":"x"}', + output: 'result value', + }, + }, + ], + metadata: { + usage: { input: 7, output: 5, cacheRead: 0, cacheWrite: 0 }, + }, + }, + ]); + + await page.goto(`/c/${conversationId}`, { timeout: 30000 }); + await expect(page.getByText('Estimate prompt')).toBeVisible({ timeout: 30000 }); + + const gauge = page.getByTestId('token-usage'); + await expect(gauge).toBeVisible({ timeout: 30000 }); + await gauge.click(); + const popover = page.getByRole('region', { name: 'Context usage' }); + await expect(popover).toBeVisible({ timeout: 10000 }); + + const breakdownToggle = popover.getByTestId('context-breakdown-toggle'); + if ((await breakdownToggle.getAttribute('aria-expanded')) === 'false') { + await breakdownToggle.click(); + } + await expect(breakdownToggle).toHaveAttribute('aria-expanded', 'true'); + + const estimate = popover.getByTestId('context-estimate'); + await expect(estimate).toBeVisible({ timeout: 10000 }); + const toolRow = estimate.locator('div.pl-6').getByText('Tool calls', { exact: true }); + await expect(toolRow).toBeVisible(); + + // Tool traffic is already included in the counted assistant total, so the Tool calls subtotal must be nested rather than a peer that would double-count the meter. + const peerRows = estimate.locator(':scope > div.flex'); + await expect(peerRows).toHaveCount(2); + const peerText = await peerRows.allTextContents(); + expect(peerText.some((text) => text.includes('Tool calls'))).toBe(false); + + // All fixture values stay below 1000, so the compact formatter leaves each peer value parseable and their sum can be compared with the gauge readout. + const peerSum = await peerRows.evaluateAll((rows) => + rows.reduce((sum, row) => { + const value = row.lastElementChild?.textContent?.match(/\d+/)?.[0]; + return sum + (value == null ? 0 : Number(value)); + }, 0), + ); + const readout = await breakdownToggle.textContent(); + const used = readout?.match(/Context window\s*(\d+)/)?.[1]; + expect(used).toBeDefined(); + expect(peerSum).toBe(Number(used)); + }); +}); diff --git a/e2e/specs/mock/scenarios/series-eight-theme.spec.ts b/e2e/specs/mock/scenarios/series-eight-theme.spec.ts new file mode 100644 index 0000000000..e0301e0bb8 --- /dev/null +++ b/e2e/specs/mock/scenarios/series-eight-theme.spec.ts @@ -0,0 +1,134 @@ +import { expect, test } from '@playwright/test'; +import type { Page } from '@playwright/test'; +import { + MOCK_ENDPOINTS, + mockReply, + NEW_CHAT_PATH, + selectMockEndpoint, + sendMessage, +} from '../helpers'; +import { deleteConversations, deleteMessagesByConversation } from '../db'; + +/** + * Pin the stored mode to light in both tests. This keeps the desktop-dark + * project's browser colorScheme from selecting a different mode, while the + * payload still follows the reference theme's intent: it owns its seven-slot + * scale, its surfaces, and its secondary text. + */ +const OWNED_SCALE_COLORS = { + 'rgb-text-primary': '250 250 250', + 'rgb-text-secondary': '215 215 215', + 'rgb-surface-secondary': '18 18 24', + 'rgb-surface-tertiary': '30 30 38', + 'rgb-series-1': '120 200 255', + 'rgb-series-2': '255 160 90', + 'rgb-series-3': '110 230 210', + 'rgb-series-4': '240 200 100', + 'rgb-series-5': '250 150 200', + 'rgb-series-6': '190 160 255', + 'rgb-series-7': '130 220 120', +} as const; + +const OWNED_SCALE_THEME = { + version: 1, + name: 'e2e-owned-scale', + modes: { + light: { + colors: OWNED_SCALE_COLORS, + }, + }, +} as const; + +const NAMED_SCALE_THEME = { + ...OWNED_SCALE_THEME, + name: 'e2e-named-scale', + modes: { + light: { + colors: { + ...OWNED_SCALE_COLORS, + 'rgb-series-8': '10 20 30', + }, + }, + }, +} as const; + +async function installTheme(page: Page, theme: unknown) { + await page.addInitScript((definition) => { + localStorage.setItem('color-theme', 'light'); + localStorage.setItem('theme-definition', JSON.stringify(definition)); + localStorage.setItem('theme-source', 'definition'); + localStorage.removeItem('theme-colors'); + localStorage.removeItem('theme-name'); + }, theme); +} + +async function resolvedSeriesEight(page: Page) { + return page.evaluate(() => + getComputedStyle(document.documentElement).getPropertyValue('--series-8').trim(), + ); +} + +test.afterEach(async ({ page }) => { + const match = new URL(page.url()).pathname.match(/^\/c\/([^/]+)\/?$/); + if (!match || match[1] === 'new') { + return; + } + const conversationId = decodeURIComponent(match[1]); + // Remove the real turn so this theme-focused spec does not leak user data + // into later projects or reruns. + await deleteMessagesByConversation([conversationId]); + await deleteConversations([conversationId]); +}); + +test.describe('series-eight theme resolution', () => { + test('owned theme keeps summary stop visible @scenario:owned-theme-keeps-summary-stop-visible', async ({ + page, + }) => { + test.setTimeout(120000); + await installTheme(page, OWNED_SCALE_THEME); + await page.goto(NEW_CHAT_PATH, { timeout: 10000 }); + await expect(page.locator('html')).toHaveAttribute('data-theme', OWNED_SCALE_THEME.name); + + // A real mock turn supplies the live context snapshot that paints the gauge. + await selectMockEndpoint(page, MOCK_ENDPOINTS[0]); + const response = await sendMessage(page, 'hello'); + expect(response.ok()).toBeTruthy(); + await expect(mockReply(page)).toBeVisible({ timeout: 20000 }); + await expect(page).toHaveURL(/\/c\/(?!new)/, { timeout: 15000 }); + + await expect(page.getByTestId('token-usage')).toBeVisible({ timeout: 20000 }); + await page.getByTestId('token-usage').click(); + const popover = page.getByRole('region', { name: 'Context usage' }); + await expect(popover).toBeVisible({ timeout: 10000 }); + const toggle = popover.getByTestId('context-breakdown-toggle'); + await expect(toggle).toBeVisible({ timeout: 10000 }); + if ((await toggle.getAttribute('aria-expanded')) === 'false') { + await toggle.click(); + } + await expect(toggle).toHaveAttribute('aria-expanded', 'true'); + await expect(popover.getByTestId('context-breakdown')).toBeVisible({ timeout: 10000 }); + + await expect.poll(() => resolvedSeriesEight(page)).toBe('215 215 215'); + // The bundled light-theme indigo must not leak into a theme-owned scale. + await expect.poll(() => resolvedSeriesEight(page)).not.toBe('63 81 181'); + + // The ordinary mock snapshot has summaryTokens === 0, so its slot-8 Summary + // legend is absent. The rendered Messages swatch proves the custom scale is + // painted in the real popover; the --series-8 assertions above cover its + // summary-stop fallback even when no summary segment is present. + const messagesSwatch = popover.locator('span.bg-series-1'); + await expect(messagesSwatch).toHaveCount(1); + await expect(messagesSwatch).toHaveCSS('background-color', 'rgb(120, 200, 255)'); + }); + + test('named series eight overrides fallback @scenario:named-series-eight-overrides-fallback', async ({ + page, + }) => { + test.setTimeout(120000); + await installTheme(page, NAMED_SCALE_THEME); + await page.goto(NEW_CHAT_PATH, { timeout: 10000 }); + await expect(page.locator('html')).toHaveAttribute('data-theme', NAMED_SCALE_THEME.name); + + await expect.poll(() => resolvedSeriesEight(page)).toBe('10 20 30'); + }); +}); diff --git a/e2e/specs/mock/scenarios/subagent-context-total.spec.ts b/e2e/specs/mock/scenarios/subagent-context-total.spec.ts new file mode 100644 index 0000000000..1fab174ecc --- /dev/null +++ b/e2e/specs/mock/scenarios/subagent-context-total.spec.ts @@ -0,0 +1,118 @@ +import { expect, test } from '@playwright/test'; +import type { Page } from '@playwright/test'; +import type { AgentDetail } from '../agents.helpers'; +import { cleanupAgent, openAgentBuilder, uniqueAgentName } from '../agents.helpers'; +import { + MOCK_ENDPOINTS, + NEW_CHAT_PATH, + getAccessToken, + requestJson, + sendMessageAndWaitForCompletion, +} from '../helpers'; + +async function createAgent( + page: Page, + token: string, + name: string, + subagents?: AgentDetail['subagents'], +): Promise { + return requestJson(page, { + path: '/api/agents', + token, + method: 'POST', + body: { + name, + description: 'Playwright verification of subagent context usage totals.', + instructions: 'Follow the test request exactly.', + provider: MOCK_ENDPOINTS[0].label, + model: MOCK_ENDPOINTS[0].model, + subagents, + }, + }); +} + +async function selectAgent(page: Page, name: string): Promise { + const form = await openAgentBuilder(page); + await form.getByRole('combobox', { name: 'Agent', exact: true }).click(); + await page.getByRole('option', { name }).click(); + await expect(form.getByLabel('Agent name')).toHaveValue(name); + await form.getByRole('button', { name: 'Select Agent' }).click(); +} + +test.describe('subagent context usage totals', () => { + /** Selecting a parent agent needs the Agent Builder side panel, which the + * phone layout does not render, so the subagent run cannot be started from a + * touch viewport at all. */ + test.skip(({ isMobile }) => isMobile === true, 'Agent Builder panel is desktop-only'); + + test('shows subagent usage across all branches separately from branch totals @scenario:subagents-row-reads-all-branches', async ({ + page, + }) => { + test.setTimeout(180000); + const label = `context-total-${Date.now()}`; + const childName = uniqueAgentName('E2E Child'); + const parentName = uniqueAgentName('E2E Parent'); + let childId: string | undefined; + let parentId: string | undefined; + + try { + await page.goto(NEW_CHAT_PATH); + const token = await getAccessToken(page); + const child = await createAgent(page, token, childName); + childId = child.id; + const parent = await createAgent(page, token, parentName, { + enabled: true, + allowSelf: false, + agent_ids: [child.id], + }); + parentId = parent.id; + + await selectAgent(page, parentName); + const response = await sendMessageAndWaitForCompletion( + page, + `E2E_SUBAGENT_RESULT:${child.id}:${label}`, + ); + expect(response.ok()).toBeTruthy(); + + const gauge = page.getByTestId('token-usage'); + await expect(gauge).toBeVisible({ timeout: 30000 }); + await gauge.click(); + const popover = page.getByRole('region', { name: 'Context usage' }); + await expect(popover).toBeVisible({ timeout: 10000 }); + + const toggle = popover.getByTestId('context-breakdown-toggle'); + if ((await toggle.getAttribute('aria-expanded')) === 'false') { + await toggle.click(); + } + await expect(toggle).toHaveAttribute('aria-expanded', 'true'); + + // A real subagent emits a context snapshot, so this scope comparison is + // against the rendered context rows rather than an estimate fallback. + const contextRows = popover.getByTestId('context-breakdown'); + await expect(contextRows).toBeVisible({ timeout: 10000 }); + await expect(contextRows.getByText('Subagents (all branches)', { exact: true })).toHaveCount( + 0, + ); + + const totals = popover.getByTestId('token-usage-totals'); + await expect(totals).toBeVisible({ timeout: 10000 }); + await expect(totals.getByRole('heading', { name: 'Totals' })).toBeVisible(); + await expect(totals.getByText('Input', { exact: true })).toBeVisible(); + await expect(totals.getByText('Output', { exact: true })).toBeVisible(); + + const subagentRow = totals + .locator('div.flex.w-full.items-center.justify-between') + .filter({ hasText: 'Subagents (all branches)' }); + await expect(subagentRow).toHaveCount(1); + await expect( + subagentRow.locator(':scope > span').nth(1), + // The value is compact-formatted when large; a leading non-zero digit + // proves the all-branches usage emitted by the child is not zero. + ).toHaveText(/^[1-9]\d*(?:\.\d+)?[KMB]?$/); + await expect(totals.getByText('Subagents', { exact: true })).toHaveCount(0); + } finally { + await cleanupAgent(page, parentId); + await cleanupAgent(page, childId); + } + }); +}); diff --git a/e2e/specs/mock/scenarios/tool-context-split.spec.ts b/e2e/specs/mock/scenarios/tool-context-split.spec.ts new file mode 100644 index 0000000000..6264aceba1 --- /dev/null +++ b/e2e/specs/mock/scenarios/tool-context-split.spec.ts @@ -0,0 +1,236 @@ +import { expect, test } from '@playwright/test'; +import type { Locator, Page } from '@playwright/test'; +import { + MOCK_ENDPOINTS, + NEW_CHAT_PATH, + escapeRegExp, + messagesView, + selectMockEndpoint, + sendMessageAndWaitForCompletion, +} from '../helpers'; + +const MCP_SERVER_TITLE = 'E2E Memory'; +const TOOL_FINAL_TEXT = 'E2E steer tool reply done'; + +const uniqueLabel = (prefix: string) => + `${prefix}-${Date.now()}-${Math.floor(Math.random() * 1e4)}`; + +const gauge = (page: Page) => page.getByTestId('token-usage'); +const gaugeMeter = (page: Page) => gauge(page).getByRole('meter'); + +async function expectGaugeAboveZero(page: Page) { + await expect(gauge(page)).toBeVisible({ timeout: 20000 }); + await expect(gaugeMeter(page)).toHaveAttribute('aria-valuenow', /[1-9]/, { timeout: 20000 }); +} + +/** Select the ephemeral MCP server whose real remember_fact tool creates the + * tool boundary and causes the fake model to take its tool-response path. */ +async function selectEphemeralMCP(page: Page) { + await page.getByRole('button', { name: 'MCP Servers', exact: true }).click(); + const serverItem = page.getByRole('menuitemcheckbox', { name: new RegExp(MCP_SERVER_TITLE) }); + await expect(serverItem).toBeVisible(); + await serverItem.click(); + await expect(serverItem).toHaveAttribute('aria-checked', 'true'); + await page.keyboard.press('Escape'); + await expect(page.getByRole('button', { name: new RegExp(MCP_SERVER_TITLE) })).toBeVisible(); +} + +/** The popover opens with only the meter visible; its detail is a remembered + * disclosure, so this helper is deliberately idempotent across reloads. */ +async function expandBreakdown(popover: Locator) { + const toggle = popover.getByTestId('context-breakdown-toggle'); + await expect(toggle).toBeVisible({ timeout: 10000 }); + if ((await toggle.getAttribute('aria-expanded')) === 'false') { + await toggle.click(); + } + await expect(toggle).toHaveAttribute('aria-expanded', 'true'); +} + +/** Opens the click-pinned popover and expands its context detail. Hover is not + * used because it is unavailable on the touch project. */ +async function openBreakdown(page: Page) { + await expectGaugeAboveZero(page); + await gauge(page).click(); + const popover = page.getByRole('region', { name: 'Context usage' }); + await expect(popover).toBeVisible({ timeout: 10000 }); + await expandBreakdown(popover); + return popover; +} + +async function runToolTurn(page: Page, prefix: string) { + const label = uniqueLabel(prefix); + await page.goto(NEW_CHAT_PATH, { timeout: 10000 }); + // The endpoint must be committed before the ephemeral agent is selected. + await selectMockEndpoint(page, MOCK_ENDPOINTS[0]); + await selectEphemeralMCP(page); + const response = await sendMessageAndWaitForCompletion(page, `E2E_STEER_TOOL_REPLY:${label}`, { + timeout: 120000, + }); + expect(response.ok()).toBeTruthy(); + await expect( + messagesView(page).getByText(new RegExp(`${TOOL_FINAL_TEXT} ${escapeRegExp(label)}`)), + ).toBeVisible({ timeout: 30000 }); +} + +function directPeerRows(breakdown: Locator) { + return breakdown.locator(':scope > *:not(.pl-6)'); +} + +function rowValue(row: Locator) { + return row.locator(':scope > span').last(); +} + +test.describe('retained tool context split', () => { + /** Every test here needs a real tool boundary, and the only affordance that + * attaches one to an ephemeral conversation is the composer's MCP Servers + * picker, which the phone layout does not render (its narrow composer keeps + * the tool pickers behind a sidebar that overlays the input). The split + * itself is viewport-independent: it is computed by the hook, and the mobile + * project still exercises the same rows through the persisted fixtures in + * `persisted-context-usage.spec.ts`. */ + test.skip(({ isMobile }) => isMobile === true, 'composer MCP picker is desktop-only'); + + test('splits retained tool traffic out of the message total @scenario:tool-calls-split-from-messages', async ({ + page, + }) => { + test.setTimeout(180000); + await runToolTurn(page, 'context-split'); + + const popover = await openBreakdown(page); + const breakdown = popover.getByTestId('context-breakdown'); + await expect(breakdown).toBeVisible({ timeout: 10000 }); + await expect(popover.getByTestId('context-estimate')).toHaveCount(0); + + const peers = directPeerRows(breakdown); + const toolRow = peers.filter({ hasText: 'Tool calls' }); + const messageRow = peers.filter({ hasText: 'Messages' }); + await expect(toolRow).toHaveCount(1); + await expect(messageRow).toHaveCount(1); + // Tool calls and Messages are meter peers, not an indented subset row. + await expect(toolRow.locator('xpath=ancestor::*[contains(@class, "pl-6")]')).toHaveCount(0); + await expect(messageRow.locator('xpath=ancestor::*[contains(@class, "pl-6")]')).toHaveCount(0); + + const toolValue = (await rowValue(toolRow).innerText()).split('(')[0].trim(); + const messageValue = (await rowValue(messageRow).innerText()).split('(')[0].trim(); + // The real tool round-trip contributes a non-zero retained share, shown + // separately from Messages; values are compact-formatted by the UI. + expect(toolValue).not.toBe('0'); + expect(toolValue).not.toBe(messageValue); + await expect(toolRow.locator('span.bg-series-2')).toHaveCount(1); + }); + + test('discloses per-tool counts with keyboard controls @scenario:per-tool-counts-keyboard-disclosure', async ({ + page, + }) => { + test.setTimeout(180000); + await runToolTurn(page, 'keyboard-disclosure'); + + const popover = await openBreakdown(page); + const breakdown = popover.getByTestId('context-breakdown'); + const toolRow = directPeerRows(breakdown).filter({ hasText: 'Tool calls' }); + // Selecting the native button directly proves this row is keyboard-operable, + // rather than merely looking like a clickable div. + const toolButton = breakdown.locator(':scope > button').filter({ hasText: 'Tool calls' }); + await expect(toolButton).toHaveCount(1); + await expect(toolButton).toHaveAttribute('aria-expanded', 'false'); + await expect(toolRow).toHaveCount(1); + + await toolButton.focus(); + await expect(toolButton).toBeFocused(); + await page.keyboard.press('Enter'); + await expect(toolButton).toHaveAttribute('aria-expanded', 'true'); + await expect(toolButton).toBeFocused(); + + const panelId = await toolButton.getAttribute('aria-controls'); + expect(panelId).toBeTruthy(); + const toolPanel = popover.locator(`[id="${panelId}"]`); + await expect(toolPanel.getByText('By tool', { exact: true })).toBeVisible(); + const namedToolRow = toolPanel.locator(':scope > *').filter({ hasText: /remember_fact/ }); + await expect(namedToolRow).toHaveCount(1); + await expect(namedToolRow).toHaveText(/remember_fact[\s\S]*(?:[1-9]\d{0,2}|\d+(?:\.\d+)?K)/); + + await page.keyboard.press('Space'); + await expect(toolButton).toHaveAttribute('aria-expanded', 'false'); + await expect(toolButton).toBeFocused(); + await expect(toolPanel.getByText('By tool', { exact: true })).toHaveCount(0); + }); + + test('keeps visible context peers within the meter @scenario:context-rows-never-exceed-the-meter', async ({ + page, + }) => { + test.setTimeout(180000); + await runToolTurn(page, 'meter-bound'); + + const popover = await openBreakdown(page); + const breakdown = popover.getByTestId('context-breakdown'); + const meter = popover.getByRole('progressbar'); + await expect(breakdown).toBeVisible({ timeout: 10000 }); + const usedPercent = Number(await meter.getAttribute('aria-valuenow')); + expect(Number.isFinite(usedPercent)).toBeTruthy(); + expect(usedPercent).toBeGreaterThan(0); + + const peers = directPeerRows(breakdown); + await expect(peers.filter({ hasText: 'Messages' })).toHaveCount(1); + await expect(peers.filter({ hasText: 'Tool calls' })).toHaveCount(1); + + const toolButton = breakdown.locator(':scope > button').filter({ hasText: 'Tool calls' }); + await toolButton.focus(); + await page.keyboard.press('Enter'); + await expect(toolButton).toHaveAttribute('aria-expanded', 'true'); + // Since compact values can be 1.5K, use the UI's subset relation instead + // of parsing rounded values: only segment peers may count toward the meter. + // Per-tool, cached, and instruction rows are always indented subsets. + await expect(peers.filter({ hasText: /remember_fact/ })).toHaveCount(0); + await expect(peers.filter({ hasText: 'Agent instructions' })).toHaveCount(0); + await expect(peers.filter({ hasText: 'Cached' })).toHaveCount(0); + await expect(peers.filter({ hasText: 'Cache write' })).toHaveCount(0); + }); + + test('rehydrates the tool split after reload @scenario:tool-split-survives-reload', async ({ + page, + }) => { + test.setTimeout(180000); + await runToolTurn(page, 'reload-split'); + + const initialPopover = await openBreakdown(page); + const initialBreakdown = initialPopover.getByTestId('context-breakdown'); + const initialToolRow = directPeerRows(initialBreakdown).filter({ hasText: 'Tool calls' }); + await expect(initialToolRow).toHaveCount(1); + const initialToolButton = initialBreakdown + .locator(':scope > button') + .filter({ hasText: 'Tool calls' }); + await initialToolButton.click(); + const initialPanelId = await initialToolButton.getAttribute('aria-controls'); + expect(initialPanelId).toBeTruthy(); + const initialPanel = initialPopover.locator(`[id="${initialPanelId}"]`); + const initialNamedTool = initialPanel + .locator(':scope > *') + .filter({ hasText: /remember_fact/ }); + await expect(initialNamedTool).toHaveCount(1); + const initialToolName = (await initialNamedTool.innerText()).trim().split(/\s+/)[0]; + expect(initialToolName).toMatch(/remember_fact/); + + await page.reload({ timeout: 15000 }); + await expectGaugeAboveZero(page); + await expect(messagesView(page).getByText(/E2E steer tool reply done/)).toBeVisible({ + timeout: 20000, + }); + const reloadedPopover = await openBreakdown(page); + const reloadedBreakdown = reloadedPopover.getByTestId('context-breakdown'); + await expect(reloadedBreakdown).toBeVisible({ timeout: 10000 }); + await expect(reloadedPopover.getByTestId('context-estimate')).toHaveCount(0); + const reloadedToolButton = reloadedBreakdown + .locator(':scope > button') + .filter({ hasText: 'Tool calls' }); + await expect(reloadedToolButton).toHaveCount(1); + await reloadedToolButton.click(); + const reloadedPanelId = await reloadedToolButton.getAttribute('aria-controls'); + expect(reloadedPanelId).toBeTruthy(); + const reloadedPanel = reloadedPopover.locator(`[id="${reloadedPanelId}"]`); + await expect(reloadedPanel.getByText('By tool', { exact: true })).toBeVisible(); + await expect(reloadedPanel.getByText(initialToolName, { exact: true })).toBeVisible(); + await expect( + reloadedPanel.locator(':scope > *').filter({ hasText: /remember_fact/ }), + ).toHaveText(/remember_fact[\s\S]*(?:[1-9]\d{0,2}|\d+(?:\.\d+)?K)/); + }); +}); diff --git a/packages/api/src/agents/usage.spec.ts b/packages/api/src/agents/usage.spec.ts index b8b748c61e..d80c882a1b 100644 --- a/packages/api/src/agents/usage.spec.ts +++ b/packages/api/src/agents/usage.spec.ts @@ -1935,13 +1935,13 @@ describe('buildPersistedContextUsage', () => { contextBudget: 7800, }; - it('trims zero-valued per-tool counts', () => { + it('persists positive per-tool schema counts', () => { const result = buildPersistedContextUsage(baseSnapshot); expect(result.breakdown.toolTokenCounts).toEqual({ add: 15 }); expect(result.contextBudget).toBe(7800); }); - it('drops the tool counts object entirely when all are zero', () => { + it('omits a schema-count record with no positive counts', () => { const result = buildPersistedContextUsage({ ...baseSnapshot, breakdown: { ...baseSnapshot.breakdown, toolTokenCounts: { add: 0 } }, @@ -1956,6 +1956,88 @@ describe('buildPersistedContextUsage', () => { expect(result.breakdown.messageTokens).toBe(500); }); + it('passes a non-zero toolMessageTokens split through to the blob', () => { + const result = buildPersistedContextUsage({ + ...baseSnapshot, + breakdown: { ...baseSnapshot.breakdown, toolMessageTokens: 220 }, + }); + expect(result.breakdown.toolMessageTokens).toBe(220); + expect(result.breakdown.messageTokens).toBe(500); + }); + + it('keeps invocation-inclusive totals separate from per-tool result shares', () => { + const result = buildPersistedContextUsage({ + ...baseSnapshot, + breakdown: { + ...baseSnapshot.breakdown, + toolMessageTokens: 10, + toolMessageTokenCounts: { search: 4 }, + }, + }); + expect(result.breakdown.toolMessageTokens).toBe(10); + expect(result.breakdown.toolMessageTokenCounts).toEqual({ search: 4 }); + expect( + Object.values(result.breakdown.toolMessageTokenCounts ?? {}).reduce( + (total, count) => total + count, + 0, + ), + ).toBeLessThan(result.breakdown.toolMessageTokens ?? 0); + }); + + it('trims zero-valued result-message entries and clamps their sum', () => { + const result = buildPersistedContextUsage({ + ...baseSnapshot, + breakdown: { + ...baseSnapshot.breakdown, + toolMessageTokens: 2, + toolMessageTokenCounts: { grep: 180, read_file: 0 }, + }, + }); + expect(result.breakdown.toolMessageTokenCounts).toEqual({ grep: 2 }); + expect( + Object.values(result.breakdown.toolMessageTokenCounts ?? {}).reduce( + (total, count) => total + count, + 0, + ), + ).toBeLessThanOrEqual(result.breakdown.toolMessageTokens ?? 0); + }); + + it('preserves a known-zero tool-message total', () => { + const result = buildPersistedContextUsage({ + ...baseSnapshot, + breakdown: { ...baseSnapshot.breakdown, toolMessageTokens: 0 }, + }); + expect(result.breakdown.toolMessageTokens).toBe(0); + expect(Object.prototype.hasOwnProperty.call(result.breakdown, 'toolMessageTokens')).toBe(true); + }); + it('safely persists prototype-sensitive names and ignores malformed counts', () => { + const counts = JSON.parse('{"__proto__":3,"constructor":3,"invalid":"4"}') as Record< + string, + number + >; + const sdkExtension = { + nested: { source: 'retained-sdk-field', flags: ['opaque'] }, + invocation: { estimated: 4, providerOnly: true }, + }; + const snapshot = { + ...baseSnapshot, + breakdown: { + ...baseSnapshot.breakdown, + toolMessageTokens: 5, + toolMessageTokenCounts: counts, + sdkExtension, + }, + }; + const result = buildPersistedContextUsage(snapshot); + const persistedCounts = result.breakdown.toolMessageTokenCounts; + expect(persistedCounts?.['__proto__']).toBe(3); + expect(persistedCounts?.constructor).toBe(2); + expect(persistedCounts?.invalid).toBeUndefined(); + expect(Object.keys(persistedCounts ?? {})).toEqual(['__proto__', 'constructor']); + expect(result.breakdown).toMatchObject({ sdkExtension }); + expect(snapshot.breakdown.toolMessageTokenCounts).toBe(counts); + }); + it('records the final primary call output as completedOutputTokens', () => { /** The latest snapshot precedes the final call, so its post-snapshot delta * is that call's output โ€” not the full multi-call response tokenCount. */ @@ -1974,6 +2056,40 @@ describe('buildPersistedContextUsage', () => { expect(result.completedOutputTokens).toBe(25); }); + it.each(['openAI', 'bedrock'])('persists the final primary cache split for %s', (provider) => { + const events: TTokenUsageEvent[] = [ + { runId: 'run-1', input_tokens: 100, input_token_details: { cache_read: 10 } }, + { + runId: 'run-1', + provider, + model: 'primary-model', + input_tokens: 200, + output_tokens: 25, + input_token_details: { cache_read: 80, cache_creation: 40 }, + }, + { runId: 'run-2', input_tokens: 900, input_token_details: { cache_read: 900 } }, + { usage_type: 'summarization', input_tokens: 500, input_token_details: { cache_read: 500 } }, + { usage_type: 'subagent', input_tokens: 600, input_token_details: { cache_read: 600 } }, + ]; + const result = buildPersistedContextUsage(baseSnapshot, events); + expect(result).toMatchObject({ + cacheRead: 80, + cacheWrite: 40, + model: 'primary-model', + provider, + completedOutputTokens: 25, + }); + }); + + it('replaces an earlier cache split with a final uncached call', () => { + const result = buildPersistedContextUsage({ ...baseSnapshot, cacheRead: 80, cacheWrite: 40 }, [ + { input_tokens: 200, output_tokens: 25 }, + ]); + expect(result).toMatchObject({ cacheRead: 0, cacheWrite: 0 }); + expect(buildPersistedContextUsage(baseSnapshot).cacheRead).toBeUndefined(); + expect(buildPersistedContextUsage(baseSnapshot).model).toBeUndefined(); + }); + it('omits completedOutputTokens when there are no primary calls', () => { expect(buildPersistedContextUsage(baseSnapshot, []).completedOutputTokens).toBeUndefined(); }); diff --git a/packages/api/src/agents/usage.ts b/packages/api/src/agents/usage.ts index c1c6ac4111..d0614361c8 100644 --- a/packages/api/src/agents/usage.ts +++ b/packages/api/src/agents/usage.ts @@ -1,9 +1,5 @@ import { logger } from '@librechat/data-schemas'; -import { - inputTokensIncludesCache, - reconcileContextUsage, - promptTokensFromUsage, -} from 'librechat-data-provider'; +import { inputTokensIncludesCache, reconcileContextUsageFromEvent } from 'librechat-data-provider'; import type { TCustomConfig, TResponseUsage, @@ -352,41 +348,93 @@ function finalPrimaryCall( return undefined; } +const finiteNonNegativeInteger = (value: unknown): number | undefined => { + if (typeof value !== 'number' || !Number.isFinite(value)) { + return undefined; + } + return Math.min(Number.MAX_SAFE_INTEGER, Math.max(0, Math.floor(value))); +}; + +/** + * Sanitizes persisted per-tool counts and drops zero entries. Null-prototype + * records keep tool names as data keys; the cap bounds each result-message share. + */ +const normalizePersistedTokenRecord = ( + value: unknown, + maxTotal?: number, +): Record | undefined => { + if (value == null || typeof value !== 'object' || Array.isArray(value)) { + return undefined; + } + const normalized: Record = Object.create(null); + let remaining = maxTotal; + let found = false; + for (const [name, rawCount] of Object.entries(value)) { + const count = finiteNonNegativeInteger(rawCount); + if (count == null) { + continue; + } + const bounded = remaining == null ? count : Math.min(count, remaining); + if (bounded === 0) { + continue; + } + normalized[name] = bounded; + found = true; + if (remaining != null) { + remaining -= bounded; + } + } + return found ? normalized : undefined; +}; + /** * Projects the latest live context snapshot into the blob persisted on * `responseMessage.metadata.contextUsage`. Reconciles the calibrated estimate to * the final call's ACTUAL prompt tokens (the SDK multiplier over-inflates * `messageTokens`, badly so when a provider injects server-side content like web * search), so a reloaded turn shows the real context โ€” not a severalร—-too-high - * number. Trims zero-valued per-tool counts (privacy/size) and records the final - * call's output as `completedOutputTokens` so rehydration adds the same - * post-snapshot delta the live gauge did. The client re-anchors the blob to the - * response message id on load. + * number. Sanitizes malformed optional token fields, bounds each result-message + * share by the parent total, and records the final call's output as + * `completedOutputTokens` so rehydration adds the same post-snapshot delta the + * live gauge did. The client re-anchors the blob to the response message id on + * load. */ export function buildPersistedContextUsage( snapshot: TContextUsageEvent, usageEvents: ReadonlyArray = [], ): TContextUsageEvent { const finalCall = finalPrimaryCall(usageEvents, snapshot.runId); - const completedOutputTokens = finalCall ? normalizeEventUnits(finalCall).output : 0; - const reconciled = finalCall - ? reconcileContextUsage(snapshot, promptTokensFromUsage(finalCall)) - : snapshot; + const reconciled = finalCall ? reconcileContextUsageFromEvent(snapshot, finalCall) : snapshot; const { breakdown } = reconciled; - let toolTokenCounts = breakdown.toolTokenCounts; - if (toolTokenCounts != null) { - const trimmed: Record = {}; - for (const [name, count] of Object.entries(toolTokenCounts)) { - if (count > 0) { - trimmed[name] = count; - } + const messageTokens = finiteNonNegativeInteger(breakdown.messageTokens) ?? 0; + const toolTokenCounts = normalizePersistedTokenRecord(breakdown.toolTokenCounts); + const rawToolMessageTokens = finiteNonNegativeInteger(breakdown.toolMessageTokens); + const toolMessageTokens = + rawToolMessageTokens == null ? undefined : Math.min(rawToolMessageTokens, messageTokens); + const toolMessageTokenCounts = + toolMessageTokens != null + ? normalizePersistedTokenRecord(breakdown.toolMessageTokenCounts, toolMessageTokens) + : undefined; + const persistedBreakdown = { ...breakdown, messageTokens }; + if (toolTokenCounts == null) { + delete persistedBreakdown.toolTokenCounts; + } else { + persistedBreakdown.toolTokenCounts = toolTokenCounts; + } + if (toolMessageTokens == null) { + delete persistedBreakdown.toolMessageTokens; + delete persistedBreakdown.toolMessageTokenCounts; + } else { + persistedBreakdown.toolMessageTokens = toolMessageTokens; + if (toolMessageTokenCounts == null) { + delete persistedBreakdown.toolMessageTokenCounts; + } else { + persistedBreakdown.toolMessageTokenCounts = toolMessageTokenCounts; } - toolTokenCounts = Object.keys(trimmed).length > 0 ? trimmed : undefined; } return { ...reconciled, - breakdown: { ...breakdown, toolTokenCounts }, - ...(completedOutputTokens > 0 && { completedOutputTokens }), + breakdown: persistedBreakdown, }; } diff --git a/packages/api/src/stream/GenerationJobManager.ts b/packages/api/src/stream/GenerationJobManager.ts index 65873fd734..869b1571f2 100644 --- a/packages/api/src/stream/GenerationJobManager.ts +++ b/packages/api/src/stream/GenerationJobManager.ts @@ -8,8 +8,7 @@ import { ApprovalEvents, SteerEvents, parseTextParts, - reconcileContextUsage, - promptTokensFromUsage, + reconcileContextUsageFromEvent, } from 'librechat-data-provider'; import type { TMessageContentParts, @@ -7714,9 +7713,14 @@ class GenerationJobManagerClass { snapshot != null && (snapshot.runId == null || usage.runId == null || snapshot.runId === usage.runId) ) { - update.contextUsage = JSON.stringify( - reconcileContextUsage(snapshot, promptTokensFromUsage(usage)), + const { completedOutputTokens, ...reconciled } = reconcileContextUsageFromEvent( + snapshot, + usage, ); + update.contextUsage = JSON.stringify({ + ...reconciled, + resumedOutputTokens: completedOutputTokens, + }); } } catch { /* leave the stored snapshot as-is on parse failure */ diff --git a/packages/api/src/stream/__tests__/usageResume.spec.ts b/packages/api/src/stream/__tests__/usageResume.spec.ts index 59564730f9..978c9cbaf7 100644 --- a/packages/api/src/stream/__tests__/usageResume.spec.ts +++ b/packages/api/src/stream/__tests__/usageResume.spec.ts @@ -15,6 +15,28 @@ function createInMemoryManager(): GenerationJobManagerClass { return manager; } +const makeSnapshot = (messageTokens: number) => ({ + runId: 'run-1', + agentId: 'agent-1', + breakdown: { + maxContextTokens: 200000, + instructionTokens: 1500, + systemMessageTokens: 1000, + dynamicInstructionTokens: 0, + toolSchemaTokens: 500, + summaryTokens: 0, + toolCount: 3, + messageCount: 4, + messageTokens, + availableForMessages: 188500, + }, + contextBudget: 190000, + effectiveInstructionTokens: 1500, + prePruneContextTokens: messageTokens, + remainingContextTokens: 190000 - 1500 - messageTokens, + calibrationRatio: 1.05, +}); + describe('GenerationJobManager usage resume state', () => { let manager: GenerationJobManagerClass | undefined; @@ -60,33 +82,62 @@ describe('GenerationJobManager usage resume state', () => { expect(resumeState?.contextUsage).toBeUndefined(); }); + test('restores final primary call details without replaying usage', async () => { + manager = createInMemoryManager(); + const streamId = `context-call-resume-${Date.now()}`; + await manager.createJob(streamId, 'user-1', streamId); + await manager.emitChunk(streamId, { event: 'on_context_usage', data: makeSnapshot(9000) }); + const primary = { + runId: 'run-1', + input_tokens: 10000, + output_tokens: 300, + input_token_details: { cache_creation: 100, cache_read: 800 }, + model: 'primary-model', + provider: 'anthropic', + }; + await manager.emitChunk(streamId, { event: 'on_token_usage', data: primary }); + await manager.emitChunk(streamId, { + event: 'on_token_usage', + data: { ...primary, runId: 'other-run', model: 'other-model' }, + }); + await manager.emitChunk(streamId, { + event: 'on_token_usage', + data: { ...primary, usage_type: 'subagent', model: 'child-model' }, + }); + + const resumeState = await manager.getResumeState(streamId); + expect(resumeState?.contextUsage).toMatchObject({ + model: 'primary-model', + provider: 'anthropic', + resumedOutputTokens: 300, + cacheRead: 800, + cacheWrite: 100, + remainingContextTokens: 180000, + breakdown: { messageTokens: 8500 }, + }); + expect(resumeState?.contextUsage?.completedOutputTokens).toBeUndefined(); + expect(resumeState?.collectedUsage).toHaveLength(3); + + await manager.emitChunk(streamId, { event: 'on_context_usage', data: makeSnapshot(12000) }); + const midCall = await manager.getResumeState(streamId); + expect(midCall?.contextUsage).toEqual(makeSnapshot(12000)); + await manager.emitChunk(streamId, { + event: 'on_token_usage', + data: { ...primary, input_token_details: {}, model: 'next-model' }, + }); + const nextCall = await manager.getResumeState(streamId); + expect(nextCall?.contextUsage).toMatchObject({ + cacheRead: 0, + cacheWrite: 0, + model: 'next-model', + }); + }); + test('persists the latest context usage snapshot for resume', async () => { manager = createInMemoryManager(); const streamId = `context-resume-${Date.now()}`; await manager.createJob(streamId, 'user-1', streamId); - const makeSnapshot = (messageTokens: number) => ({ - runId: 'run-1', - agentId: 'agent-1', - breakdown: { - maxContextTokens: 200000, - instructionTokens: 1500, - systemMessageTokens: 1000, - dynamicInstructionTokens: 0, - toolSchemaTokens: 500, - summaryTokens: 0, - toolCount: 3, - messageCount: 4, - messageTokens, - availableForMessages: 188500, - }, - contextBudget: 190000, - effectiveInstructionTokens: 1500, - prePruneContextTokens: messageTokens, - remainingContextTokens: 190000 - 1500 - messageTokens, - calibrationRatio: 1.05, - }); - await manager.emitChunk(streamId, { event: 'on_context_usage', data: makeSnapshot(4000) }); await manager.emitChunk(streamId, { event: 'on_context_usage', data: makeSnapshot(9000) }); diff --git a/packages/client/src/components/SegmentedMeter.tsx b/packages/client/src/components/SegmentedMeter.tsx index 37445e5abf..047c0b1193 100644 --- a/packages/client/src/components/SegmentedMeter.tsx +++ b/packages/client/src/components/SegmentedMeter.tsx @@ -10,6 +10,7 @@ const SERIES_FILL = [ 'bg-series-5', 'bg-series-6', 'bg-series-7', + 'bg-series-8', ] as const; const SERIES_TINT = [ @@ -20,6 +21,7 @@ const SERIES_TINT = [ 'bg-series-5/25', 'bg-series-6/25', 'bg-series-7/25', + 'bg-series-8/25', ] as const; const SERIES_EDGE = [ @@ -30,6 +32,7 @@ const SERIES_EDGE = [ 'ring-series-5', 'ring-series-6', 'ring-series-7', + 'ring-series-8', ] as const; export const SERIES_SLOT_COUNT: number = SERIES_FILL.length; diff --git a/packages/client/src/theme/registry.spec.ts b/packages/client/src/theme/registry.spec.ts index a6216c02e0..5407c4e085 100644 --- a/packages/client/src/theme/registry.spec.ts +++ b/packages/client/src/theme/registry.spec.ts @@ -211,6 +211,89 @@ describe('theme registry', () => { expect(resolved.colors['rgb-chart-widget-stroke']).toBe('50 51 52'); }); + /** A deliberately different reference theme: it paints the whole seven-slot + * scale and its own surfaces, so it predates slot 8 and cannot name it. + * Falling back to the bundled indigo would paint a stop whose 3:1 mark + * contrast was only ever measured against LibreChat's surfaces. */ + const ownedScaleTheme: ThemeDefinition = { + version: 1, + name: 'owned-scale-reference', + modes: { + light: { + colors: { + 'rgb-text-primary': '250 250 250', + 'rgb-text-secondary': '215 215 215', + 'rgb-surface-secondary': '18 18 24', + 'rgb-surface-tertiary': '30 30 38', + 'rgb-series-1': '120 200 255', + 'rgb-series-2': '255 160 90', + 'rgb-series-3': '110 230 210', + 'rgb-series-4': '240 200 100', + 'rgb-series-5': '250 150 200', + 'rgb-series-6': '190 160 255', + 'rgb-series-7': '130 220 120', + }, + }, + }, + }; + + it('derives an omitted eighth series slot from an owned scaleโ€™s neutral text', () => { + const owned = resolveTheme(ownedScaleTheme, 'light'); + + expect(owned.colors['rgb-series-8']).toBe('215 215 215'); + expect(owned.colors['rgb-series-7']).toBe('130 220 120'); + }); + + it('derives it from the inherited text of an owned scale that names no text', () => { + /** The same reference theme minus its text overrides: it repaints the scale + * and its surfaces but reads body copy in LibreChat's own colours, so the + * stop tracks the text it will actually sit beside instead of reverting to + * the bundled indigo. */ + const inheritedText = resolveTheme( + { + ...ownedScaleTheme, + modes: { + light: { + colors: { + 'rgb-surface-secondary': '18 18 24', + 'rgb-surface-tertiary': '30 30 38', + 'rgb-series-1': '120 200 255', + 'rgb-series-7': '130 220 120', + }, + }, + }, + }, + 'light', + ); + + expect(inheritedText.colors['rgb-series-8']).toBe(defaultTheme['rgb-text-secondary']); + }); + + it('lets an owned scale name the eighth slot itself', () => { + const named = resolveTheme( + { + ...ownedScaleTheme, + modes: { + light: { + colors: { ...ownedScaleTheme.modes.light?.colors, 'rgb-series-8': '10 20 30' }, + }, + }, + }, + 'light', + ); + + expect(named.colors['rgb-series-8']).toBe('10 20 30'); + }); + + it('keeps the bundled eighth slot for a theme that paints no series colors', () => { + expect(resolveTheme(compactTheme, 'dark').colors['rgb-series-8']).toBe( + darkTheme['rgb-series-8'], + ); + expect(resolveTheme(compactTheme, 'light').colors['rgb-series-8']).toBe( + defaultTheme['rgb-series-8'], + ); + }); + it('resolves provider brand tokens and lets a theme override them', () => { const defaults = resolveTheme(libreChatTheme, 'light'); expect(defaults.brands['provider-anthropic']).toBe('#d09a74'); diff --git a/packages/client/src/theme/registry.ts b/packages/client/src/theme/registry.ts index 8220b2fd4d..b2fd084315 100644 --- a/packages/client/src/theme/registry.ts +++ b/packages/client/src/theme/registry.ts @@ -18,7 +18,7 @@ export const THEME_VERSION = 1 as const; * hand-maintained token maps, so a slot added to one and missed in another * fails the build rather than surfacing as a broken theme downstream. */ -type SeriesSlot = 1 | 2 | 3 | 4 | 5 | 6 | 7; +type SeriesSlot = 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8; type Assert = Declared; type DeclaredIn = [Keys] extends [keyof Tokens] ? true : false; @@ -391,6 +391,30 @@ export function resolveTheme(theme: ThemeDefinition, mode: ThemeMode): ResolvedT customColors?.['rgb-border-light'] !== undefined ? { 'rgb-chart-widget-stroke': customColors['rgb-border-light'] } : {}; + /** + * Slot 8 arrived after the seven-slot scale shipped, so a stored or + * environment theme that paints its own scale cannot name it. Filling the + * omission from the bundled base would drop LibreChat's indigo onto that + * theme's own surfaces โ€” the one pairing it never checked, since the stop's + * 3:1 mark contrast is a claim about the bundled surfaces only. The RESOLVED + * secondary text is the one colour that tracks whatever the theme reads its + * body copy against, whether it names its own or inherits ours, so slot 8 + * stays exactly as visible as that text; hue-neutral, it cannot collide with + * a custom slot 1โ€“7 under protanopia/deuteranopia either. A theme that wants + * a hue for slot 8 names it, the way `rgb-surface-composer-hover` opts out of + * its own fallback. + */ + const ownsSeriesScale = + customColors != null && + ([1, 2, 3, 4, 5, 6, 7] as const).some( + (slot) => customColors[`rgb-series-${slot}`] !== undefined, + ); + const seriesEightFallback = + customColors?.['rgb-series-8'] === undefined && ownsSeriesScale + ? { + 'rgb-series-8': customColors?.['rgb-text-secondary'] ?? baseColors['rgb-text-secondary'], + } + : {}; return { version: THEME_VERSION, @@ -405,6 +429,7 @@ export function resolveTheme(theme: ThemeDefinition, mode: ThemeMode): ResolvedT ...textMutedFallback, ...chartWidgetSurfaceFallback, ...chartWidgetStrokeFallback, + ...seriesEightFallback, } as Required, appearance: { ...defaultAppearance, ...definition?.appearance }, /** Mode last: a mode override is more specific than the theme-wide set. */ diff --git a/packages/client/src/theme/semanticTokens.spec.ts b/packages/client/src/theme/semanticTokens.spec.ts index 1073a0e923..ffe1459a94 100644 --- a/packages/client/src/theme/semanticTokens.spec.ts +++ b/packages/client/src/theme/semanticTokens.spec.ts @@ -264,7 +264,7 @@ describe.each([ * checked capability badges sit on `surface-chat`; dialog option toggles sit * on `surface-dialog`. All have to clear the 3:1 mark-contrast floor. */ const seriesTokens = Array.from( - { length: 7 }, + { length: 8 }, (_, index) => `rgb-series-${index + 1}` as keyof IThemeRGB, ); const seriesSurfaces: Array = [ diff --git a/packages/client/src/theme/themes/dark.ts b/packages/client/src/theme/themes/dark.ts index d16ecc3d61..65dcf7786e 100644 --- a/packages/client/src/theme/themes/dark.ts +++ b/packages/client/src/theme/themes/dark.ts @@ -112,8 +112,12 @@ export const darkTheme: IThemeRGB = { 'rgb-syntax-attr': '223 48 121', // #df3079 'rgb-syntax-title': '242 44 61', // #f22c3d - /** Categorical series scale โ€” the same seven hues stepped for the #212121 - * surface: worst adjacent CVD ฮ”E 13.0, normal-vision ฮ”E 19.0, all โ‰ฅ 3:1. */ + /** Categorical series scale โ€” the same eight hues stepped for the #212121 + * surface: worst adjacent CVD ฮ”E 13.0, normal-vision ฮ”E 19.0, all โ‰ฅ 3:1. + * Slot 8 is muted rather than the light mode's saturated indigo because a + * slot also fills a badge chip under `text-on-status` white: #8c98e6 carried + * that glyph at only 2.71:1, this reads 3.66:1 and still clears 3.66:1 on + * every series surface. */ 'rgb-series-1': '9 140 238', // #098cee (cerulean) 'rgb-series-2': '217 87 35', // #d95723 (orange) 'rgb-series-3': '6 158 152', // #069e98 (aqua) @@ -121,6 +125,7 @@ export const darkTheme: IThemeRGB = { 'rgb-series-5': '213 82 130', // #d55282 (magenta) 'rgb-series-6': '171 104 254', // #ab68fe (violet) 'rgb-series-7': '80 167 49', // #50a731 (green) + 'rgb-series-8': '120 130 190', // #8082be (indigo) /** Unchecked switch track. 3.38:1 against the page and the `surface-primary` * thumb, 5.74:1 against the checked `surface-inverted` track. */ diff --git a/packages/client/src/theme/themes/default.ts b/packages/client/src/theme/themes/default.ts index 9e688ff446..a21426d622 100644 --- a/packages/client/src/theme/themes/default.ts +++ b/packages/client/src/theme/themes/default.ts @@ -106,7 +106,9 @@ export const defaultTheme: IThemeRGB = { /** Categorical series scale. Steps clear 3:1 against BOTH the popover surface * and the #ececec meter track, with worst adjacent CVD ฮ”E 12.4 and worst - * adjacent normal-vision ฮ”E 19.0. Slot order is the CVD-safety mechanism. */ + * adjacent normal-vision ฮ”E 19.0. Slot order is the CVD-safety mechanism: + * indigo(8) sits beside green(7) because blue separates from green under + * protanopia/deuteranopia where red would not. */ 'rgb-series-1': '5 110 189', // #056ebd (cerulean) 'rgb-series-2': '233 86 13', // #e9560d (orange) 'rgb-series-3': '0 148 142', // #00948e (aqua) @@ -114,6 +116,7 @@ export const defaultTheme: IThemeRGB = { 'rgb-series-5': '216 90 142', // #d85a8e (magenta) 'rgb-series-6': '126 35 205', // #7e23cd (violet) 'rgb-series-7': '1 131 1', // #018301 (green) + 'rgb-series-8': '63 81 181', // #3f51b5 (indigo) /** Unchecked switch track. 3.03:1 against the white page and the * `surface-primary` thumb, 5.91:1 against the checked `surface-inverted` diff --git a/packages/client/src/theme/themes/highContrast.ts b/packages/client/src/theme/themes/highContrast.ts index fd07d37d54..5140c07b74 100644 --- a/packages/client/src/theme/themes/highContrast.ts +++ b/packages/client/src/theme/themes/highContrast.ts @@ -141,8 +141,12 @@ export const highContrastLightTheme: IThemeRGB = { 'rgb-syntax-title': '143 26 16', // #8f1a10 /** Categorical series scale. Every slot clears 7:1 on the white canvas and no - * two adjacent slots sit closer than CIE76 dE 50 under normal vision or dE 45 - * under simulated deuteranopia. */ + * two adjacent chromatic slots sit closer than CIE76 dE 50 under normal + * vision or dE 45 under simulated deuteranopia. Seven saturated hues is all + * this canvas affords at that separation, so slot 8 is the palette's own + * neutral text โ€” the stop `resolveTheme` also derives for a theme that + * paints its own scale โ€” separated by lightness (21:1 on the canvas, dE 22.9 + * from its neighbour under deuteranopia) rather than by hue. */ 'rgb-series-1': '11 79 160', // #0b4fa0 'rgb-series-2': '143 59 0', // #8f3b00 'rgb-series-3': '0 82 79', // #00524f @@ -150,6 +154,7 @@ export const highContrastLightTheme: IThemeRGB = { 'rgb-series-5': '148 0 92', // #94005c 'rgb-series-6': '77 26 153', // #4d1a99 'rgb-series-7': '15 92 15', // #0f5c0f + 'rgb-series-8': '0 0 0', // #000000 (neutral text stop) /** Unchecked switch track. The stock 58%/40% greys land at 2.9:1 and 2.2:1 * against these canvases. This clears 3:1 three ways at once: 5.74:1 against @@ -272,9 +277,12 @@ export const highContrastDarkTheme: IThemeRGB = { 'rgb-syntax-attr': '255 153 194', // #ff99c2 'rgb-syntax-title': '255 143 143', // #ff8f8f - /** Categorical series scale. Every slot clears 9.9:1 on the black canvas and no - * two adjacent slots sit closer than CIE76 dE 37 under normal vision or dE 42 - * under simulated deuteranopia. */ + /** Categorical series scale. Every slot clears 9.9:1 on the black canvas and + * no two adjacent chromatic slots sit closer than CIE76 dE 37 under normal + * vision or dE 42 under simulated deuteranopia. As in the light palette, + * slot 8 is the neutral text stop rather than an eighth pastel this canvas + * cannot separate: 21:1 on the canvas and dE 26.8 from the nearest slot + * under deuteranopia, distinguished by lightness rather than hue. */ 'rgb-series-1': '107 184 255', // #6bb8ff 'rgb-series-2': '255 179 102', // #ffb366 'rgb-series-3': '92 230 219', // #5ce6db @@ -282,6 +290,7 @@ export const highContrastDarkTheme: IThemeRGB = { 'rgb-series-5': '255 153 194', // #ff99c2 'rgb-series-6': '200 163 255', // #c8a3ff 'rgb-series-7': '140 230 140', // #8ce68c + 'rgb-series-8': '255 255 255', // #ffffff (neutral text stop) /** Unchecked switch track: 5.32:1 against the page and the `surface-primary` * thumb, 3.95:1 against the checked `surface-inverted` track. */ diff --git a/packages/client/src/theme/types/index.ts b/packages/client/src/theme/types/index.ts index 94fde9189a..7582c558c9 100644 --- a/packages/client/src/theme/types/index.ts +++ b/packages/client/src/theme/types/index.ts @@ -121,6 +121,7 @@ export interface IThemeRGB { 'rgb-series-5'?: string; 'rgb-series-6'?: string; 'rgb-series-7'?: string; + 'rgb-series-8'?: string; /** * Unchecked track of the shared `Switch`. A control state rather than a @@ -232,6 +233,7 @@ export interface IThemeVariables { '--series-5': string; '--series-6': string; '--series-7': string; + '--series-8': string; '--switch-unchecked': string; @@ -322,6 +324,7 @@ export interface IThemeColors { 'series-6'?: string; 'series-7'?: string; 'switch-unchecked'?: string; + 'series-8'?: string; presentation?: string; // Retained for excluded SidePanel/Agents + SidePanel/Builder (pending migration) diff --git a/packages/client/src/theme/utils/createTailwindColors.js b/packages/client/src/theme/utils/createTailwindColors.js index c4e31cfe25..050868fb56 100644 --- a/packages/client/src/theme/utils/createTailwindColors.js +++ b/packages/client/src/theme/utils/createTailwindColors.js @@ -149,6 +149,7 @@ function createTailwindColors() { 'series-5': cssVar('--series-5'), 'series-6': cssVar('--series-6'), 'series-7': cssVar('--series-7'), + 'series-8': cssVar('--series-8'), 'switch-unchecked': cssVar('--switch-unchecked'), }; diff --git a/packages/data-provider/src/types/runs.spec.ts b/packages/data-provider/src/types/runs.spec.ts index e35d1a03c2..cd00f9a40b 100644 --- a/packages/data-provider/src/types/runs.spec.ts +++ b/packages/data-provider/src/types/runs.spec.ts @@ -1,5 +1,5 @@ import type { TContextUsageEvent, TTokenUsageEvent } from './runs'; -import { promptTokensFromUsage, reconcileContextUsage } from './runs'; +import { promptTokensFromUsage, outputTokensFromUsage, reconcileContextUsage } from './runs'; describe('promptTokensFromUsage', () => { it('adds cache reads/writes for additive providers (Bedrock)', () => { @@ -32,6 +32,15 @@ describe('promptTokensFromUsage', () => { it('handles missing fields', () => { expect(promptTokensFromUsage({ provider: 'anthropic' })).toBe(0); }); + it('ignores malformed numeric usage fields', () => { + expect( + promptTokensFromUsage({ + input_tokens: Number.NaN, + input_token_details: { cache_read: Number.POSITIVE_INFINITY, cache_creation: '4' as never }, + provider: 'bedrock', + }), + ).toBe(0); + }); it('accepts the activity-label usage bucket emitted on the wire', () => { /** Type-level pin: the backend emits this literal for fast-model header @@ -76,6 +85,30 @@ describe('promptTokensFromUsage', () => { }); }); +describe('outputTokensFromUsage', () => { + it('repairs omitted reasoning while excluding additive prompt-cache tokens', () => { + expect( + outputTokensFromUsage({ + provider: 'bedrock', + input_tokens: 200, + output_tokens: 30, + total_tokens: 450, + input_token_details: { cache_read: 100, cache_creation: 50 }, + }), + ).toBe(100); + expect( + outputTokensFromUsage({ + provider: 'bedrock', + input_tokens: 200, + output_tokens: 30, + total_tokens: 380, + input_token_details: { cache_read: 100, cache_creation: 50 }, + }), + ).toBe(30); + expect(outputTokensFromUsage({ output_tokens: NaN, total_tokens: Infinity })).toBe(0); + }); +}); + describe('reconcileContextUsage', () => { /** The exact over-reporting case from a real web-search + summarization turn: * calibrationRatio pinned at 5 inflated messageTokens to 187471 โ†’ used 213375, @@ -121,6 +154,133 @@ describe('reconcileContextUsage', () => { expect(result.remainingContextTokens).toBe(237500 - 3000); }); + it('rescales toolMessageTokens proportionally to the corrected messageTokens', () => { + const snapshot: TContextUsageEvent = { + ...inflatedSnapshot, + breakdown: { ...inflatedSnapshot.breakdown, toolMessageTokens: 46868 }, + }; + const result = reconcileContextUsage(snapshot, 55773); + const messageTokens = 55773 - 4205 - 1938; + const toolMessageTokens = Math.round((46868 / 187471) * messageTokens); + expect(result.breakdown.messageTokens).toBe(messageTokens); + expect(result.breakdown.toolMessageTokens).toBe(toolMessageTokens); + /** the split stays a subset of the corrected total */ + expect(result.breakdown.toolMessageTokens).toBeLessThanOrEqual(result.breakdown.messageTokens); + /** rows still sum to the real total */ + expect( + result.breakdown.messageTokens + + result.breakdown.instructionTokens + + result.breakdown.summaryTokens, + ).toBe(55773); + }); + + it('rescales toolMessageTokenCounts with the reconciled total', () => { + const snapshot: TContextUsageEvent = { + ...inflatedSnapshot, + breakdown: { + ...inflatedSnapshot.breakdown, + toolMessageTokens: 46868, + toolMessageTokenCounts: { grep: 40000, read_file: 6868, zero: 0 }, + }, + }; + const result = reconcileContextUsage(snapshot, 55773); + const factor = result.breakdown.toolMessageTokens! / 46868; + expect(result.breakdown.toolMessageTokenCounts?.grep).toBe(Math.round(40000 * factor)); + expect(result.breakdown.toolMessageTokenCounts?.read_file).toBe(Math.round(6868 * factor)); + }); + + it('rescales toolMessageTokenCounts without rounding past the total', () => { + const snapshot: TContextUsageEvent = { + ...inflatedSnapshot, + breakdown: { + ...inflatedSnapshot.breakdown, + messageTokens: 10, + toolMessageTokens: 10, + toolMessageTokenCounts: { first: 5, second: 5 }, + }, + }; + const result = reconcileContextUsage(snapshot, 4205 + 1938 + 1); + expect(result.breakdown.toolMessageTokens).toBe(1); + expect( + Object.values(result.breakdown.toolMessageTokenCounts ?? {}).reduce( + (total, count) => total + count, + 0, + ), + ).toBeLessThanOrEqual(result.breakdown.toolMessageTokens ?? 0); + }); + + it('sanitizes malformed counts and preserves prototype-sensitive names', () => { + const counts = JSON.parse('{"__proto__":2,"constructor":2,"invalid":"3"}') as Record< + string, + number + >; + const snapshot: TContextUsageEvent = { + ...inflatedSnapshot, + breakdown: { + ...inflatedSnapshot.breakdown, + messageTokens: 10, + toolMessageTokens: 4, + toolMessageTokenCounts: counts, + }, + }; + const result = reconcileContextUsage(snapshot, 4205 + 1938 + 10); + const resultCounts = result.breakdown.toolMessageTokenCounts; + expect(resultCounts?.['__proto__']).toBe(2); + expect(resultCounts?.constructor).toBe(2); + expect(resultCounts?.invalid).toBeUndefined(); + expect(Object.keys(resultCounts ?? {})).toEqual(['__proto__', 'constructor']); + }); + + it('keeps a supplied zero tool split distinct from an absent split', () => { + const snapshot: TContextUsageEvent = { + ...inflatedSnapshot, + breakdown: { ...inflatedSnapshot.breakdown, toolMessageTokens: 0 }, + }; + const result = reconcileContextUsage(snapshot, 55773); + expect(result.breakdown.toolMessageTokens).toBe(0); + expect(Object.prototype.hasOwnProperty.call(result.breakdown, 'toolMessageTokens')).toBe(true); + }); + + it('keeps toolMessageTokenCounts absent when the tool total is absent', () => { + const snapshot: TContextUsageEvent = { + ...inflatedSnapshot, + breakdown: { ...inflatedSnapshot.breakdown, toolMessageTokenCounts: { grep: 100 } }, + }; + const result = reconcileContextUsage(snapshot, 55773); + expect(result.breakdown.toolMessageTokenCounts).toBeUndefined(); + }); + + it('keeps toolMessageTokens absent on snapshots without the field (older SDK)', () => { + const result = reconcileContextUsage(inflatedSnapshot, 55773); + expect(result.breakdown.toolMessageTokens).toBeUndefined(); + }); + + it('does not mutate the snapshot while sanitizing malformed values', () => { + const snapshot: TContextUsageEvent = { + ...inflatedSnapshot, + breakdown: { + ...inflatedSnapshot.breakdown, + toolMessageTokens: Number.NaN, + toolMessageTokenCounts: { grep: Number.POSITIVE_INFINITY }, + }, + }; + const result = reconcileContextUsage(snapshot, 55773); + expect(snapshot.breakdown.toolMessageTokens).toBeNaN(); + expect(snapshot.breakdown.toolMessageTokenCounts?.grep).toBe(Infinity); + expect(result.breakdown.toolMessageTokens).toBeUndefined(); + expect(result.breakdown.toolMessageTokenCounts).toBeUndefined(); + }); + + it('clamps an over-sized tool share to the corrected messageTokens', () => { + const snapshot: TContextUsageEvent = { + ...inflatedSnapshot, + breakdown: { ...inflatedSnapshot.breakdown, toolMessageTokens: 187471 }, + }; + const result = reconcileContextUsage(snapshot, 6144); + expect(result.breakdown.messageTokens).toBe(1); + expect(result.breakdown.toolMessageTokens).toBe(1); + }); + it('is a no-op for an unusable prompt count', () => { expect(reconcileContextUsage(inflatedSnapshot, 0)).toBe(inflatedSnapshot); expect(reconcileContextUsage(inflatedSnapshot, -5)).toBe(inflatedSnapshot); diff --git a/packages/data-provider/src/types/runs.ts b/packages/data-provider/src/types/runs.ts index e05f5cec8c..4ab53c8251 100644 --- a/packages/data-provider/src/types/runs.ts +++ b/packages/data-provider/src/types/runs.ts @@ -260,6 +260,10 @@ export type TTokenBudgetBreakdown = { toolTokenCounts?: Record; /** Names of counted tools that are deferred (`defer_loading`) and discovered */ deferredToolNames?: string[]; + /** Calibrated retained tool traffic: tool results plus assistant turns made only of tool calls, inline provider tool results and reasoning. A subset of messageTokens, not an additional budget category. */ + toolMessageTokens?: number; + /** Per-tool result-message share; excludes assistant invocation overhead. */ + toolMessageTokenCounts?: Record; }; /** Per-model-call context snapshot, dispatched after pruning and before the LLM call. */ @@ -276,12 +280,21 @@ export type TContextUsageEvent = { /** Tokens still free after instructions + pruned messages */ remainingContextTokens?: number; calibrationRatio?: number; + /** Model and provider of the reconciling primary call, retained across reloads. */ + model?: string; + provider?: string; + /** Cache portions of that call's prompt, in normalized display units. */ + cacheRead?: number; + cacheWrite?: number; /** Output tokens of the response's final model call (the call this pre-invoke - * snapshot precedes). Populated only on the persisted `metadata.contextUsage` - * blob so a reloaded multi-call turn adds the same post-snapshot delta the + * snapshot precedes). Saved in message metadata and used by the client so a + * reloaded multi-call turn adds the same post-snapshot delta the * live finalizer did โ€” not the full response `tokenCount`, which the snapshot * already includes for earlier steps. */ completedOutputTokens?: number; + /** Completed output in a resumable job. Separate from the saved-message field + * so older clients do not add it alongside their trailing-text estimate. */ + resumedOutputTokens?: number; }; /** @@ -332,6 +345,13 @@ export type TTokenUsageEvent = { cost?: number; }; +const finiteNonNegativeInteger = (value: unknown): number | undefined => { + if (typeof value !== 'number' || !Number.isFinite(value)) { + return undefined; + } + return Math.min(Number.MAX_SAFE_INTEGER, Math.max(0, Math.floor(value))); +}; + /** * Full prompt token count for one completed model call โ€” the EXACT context the * model saw, provider-aware: additive providers (Bedrock) report `input_tokens` @@ -343,26 +363,66 @@ export type TTokenUsageEvent = { * gauge reconciles its calibrated estimate to. */ export const promptTokensFromUsage = (event: TTokenUsageEvent): number => { - const input = event.input_tokens ?? 0; + const input = finiteNonNegativeInteger(event.input_tokens) ?? 0; const details = event.input_token_details ?? {}; - const cacheRead = details.cache_read ?? 0; - const cacheCreation = details.cache_creation ?? 0; + const cacheRead = finiteNonNegativeInteger(details.cache_read) ?? 0; + const cacheCreation = finiteNonNegativeInteger(details.cache_creation) ?? 0; const includesCache = event.provider != null ? inputTokensIncludesCache(event.provider) : cacheRead + cacheCreation <= input; - return includesCache ? input : input + cacheRead + cacheCreation; + return includesCache + ? input + : Math.min(Number.MAX_SAFE_INTEGER, input + cacheRead + cacheCreation); +}; + +/** + * Scales per-tool result-message counts while bounding both malformed input and + * rounding error. Null-prototype records keep tool names as ordinary data keys; + * cumulative apportionment preserves the scaled sum in one linear pass. + */ +const scaleToolMessageTokenCounts = ( + value: unknown, + oldTotal: number, + newTotal: number, +): Record | undefined => { + if (value == null || typeof value !== 'object' || Array.isArray(value)) { + return undefined; + } + const scaled: Record = Object.create(null); + let remaining = oldTotal; + let cumulative = 0; + let allocated = 0; + let found = false; + for (const [name, rawCount] of Object.entries(value)) { + const count = finiteNonNegativeInteger(rawCount); + if (count == null || count === 0) { + continue; + } + const bounded = Math.min(count, remaining); + remaining -= bounded; + cumulative += bounded; + const target = oldTotal > 0 ? Math.round((cumulative / oldTotal) * newTotal) : 0; + const apportioned = target - allocated; + allocated = target; + if (apportioned > 0) { + scaled[name] = apportioned; + found = true; + } + } + return found ? scaled : undefined; }; /** * Reconciles a pre-invoke context snapshot's CALIBRATED estimate to a call's * ACTUAL prompt tokens. The SDK's calibration multiplier scales only - * `messageTokens` (instructions/summary are raw tiktoken counts), and it can + * `messageTokens` (instructions/summary are raw tiktoken counts) and can * over-shoot badly when a provider injects server-side content the SDK never * counted (e.g. Anthropic web search) โ€” pinning the gauge severalร— too high and * persisting it. Trust the provider's own prompt count: keep the raw - * instruction/summary rows, set `messageTokens` to the remainder, and recompute - * the free space. No-op when `promptTokens` is unusable. + * instruction/summary rows, set `messageTokens` to the remainder, recompute the + * free space, and rescale the `toolMessageTokens` share to the new message + * total. No-op when `promptTokens` is unusable. */ export const reconcileContextUsage = ( snapshot: TContextUsageEvent, @@ -371,18 +431,100 @@ export const reconcileContextUsage = ( if (!Number.isFinite(promptTokens) || promptTokens <= 0) { return snapshot; } + const normalizedPromptTokens = Math.min(Number.MAX_SAFE_INTEGER, Math.floor(promptTokens)); + if (normalizedPromptTokens <= 0) { + return snapshot; + } const { breakdown } = snapshot; - const budget = snapshot.contextBudget ?? breakdown.maxContextTokens; - const nonMessageTokens = (breakdown.instructionTokens ?? 0) + (breakdown.summaryTokens ?? 0); - const messageTokens = Math.max(0, promptTokens - nonMessageTokens); - return { - ...snapshot, - breakdown: { ...breakdown, messageTokens }, - remainingContextTokens: - budget != null ? Math.max(0, budget - promptTokens) : snapshot.remainingContextTokens, + const instructionTokens = finiteNonNegativeInteger(breakdown.instructionTokens) ?? 0; + const summaryTokens = finiteNonNegativeInteger(breakdown.summaryTokens) ?? 0; + const budget = + finiteNonNegativeInteger(snapshot.contextBudget) ?? + finiteNonNegativeInteger(breakdown.maxContextTokens); + const nonMessageTokens = instructionTokens + summaryTokens; + const messageTokens = Math.max(0, normalizedPromptTokens - nonMessageTokens); + /** `toolMessageTokens` is a subset of the OLD (calibrated) `messageTokens`; + * rescale it by the same proportion so the split tracks the provider's real + * total. A supplied zero remains a known-zero split; absent fields stay absent + * for older snapshots. */ + const priorMessageTokens = finiteNonNegativeInteger(breakdown.messageTokens) ?? 0; + const priorToolMessageTokens = finiteNonNegativeInteger(breakdown.toolMessageTokens); + let toolMessageTokens: number | undefined; + if (priorToolMessageTokens != null) { + toolMessageTokens = 0; + if (priorMessageTokens > 0 && priorToolMessageTokens > 0) { + toolMessageTokens = Math.min( + messageTokens, + Math.round( + (Math.min(priorToolMessageTokens, priorMessageTokens) / priorMessageTokens) * + messageTokens, + ), + ); + } + } + const toolMessageTokenCounts = + toolMessageTokens != null + ? scaleToolMessageTokenCounts( + breakdown.toolMessageTokenCounts, + Math.min(priorToolMessageTokens ?? 0, priorMessageTokens), + toolMessageTokens, + ) + : undefined; + + const nextBreakdown = { + ...breakdown, + instructionTokens, + summaryTokens, + messageTokens, }; + if (toolMessageTokens == null) { + delete nextBreakdown.toolMessageTokens; + delete nextBreakdown.toolMessageTokenCounts; + } else { + nextBreakdown.toolMessageTokens = toolMessageTokens; + if (toolMessageTokenCounts == null) { + delete nextBreakdown.toolMessageTokenCounts; + } else { + nextBreakdown.toolMessageTokenCounts = toolMessageTokenCounts; + } + } + const result = { + ...snapshot, + breakdown: nextBreakdown, + }; + if (budget != null) { + result.remainingContextTokens = Math.max(0, budget - normalizedPromptTokens); + } else { + const remaining = finiteNonNegativeInteger(snapshot.remainingContextTokens); + if (remaining == null) { + delete result.remainingContextTokens; + } else { + result.remainingContextTokens = remaining; + } + } + return result; }; +/** Provider output, including reasoning omitted from an under-reported output count. */ +export const outputTokensFromUsage = (event: TTokenUsageEvent): number => { + const output = finiteNonNegativeInteger(event.output_tokens) ?? 0; + const total = finiteNonNegativeInteger(event.total_tokens) ?? 0; + return Math.max(output, total - promptTokensFromUsage(event)); +}; + +/** Reconcile and retain the primary call details on live, saved, and resumable snapshots. */ +export const reconcileContextUsageFromEvent = ( + snapshot: TContextUsageEvent, + event: TTokenUsageEvent, +): TContextUsageEvent => ({ + ...reconcileContextUsage(snapshot, promptTokensFromUsage(event)), + model: event.model, + provider: event.provider, + completedOutputTokens: outputTokensFromUsage(event), + cacheRead: finiteNonNegativeInteger(event.input_token_details?.cache_read) ?? 0, + cacheWrite: finiteNonNegativeInteger(event.input_token_details?.cache_creation) ?? 0, +}); + /** Lifecycle phase carried on subagent-progress envelopes (mirrors SDK SubagentUpdatePhase). */ export type SubagentUpdatePhase = | 'start' diff --git a/packages/data-schemas/src/methods/message.spec.ts b/packages/data-schemas/src/methods/message.spec.ts index 033c94bbc3..bf3c9d20a2 100644 --- a/packages/data-schemas/src/methods/message.spec.ts +++ b/packages/data-schemas/src/methods/message.spec.ts @@ -158,6 +158,85 @@ describe('Message Operations', () => { expect(savedMessage?.isUserSubmitted).toBeUndefined(); }); + it('round-trips context usage metadata without narrowing tool fields or unknown data', async () => { + const contextUsage = { + runId: 'run-context-rich', + breakdown: { + messageTokens: 120, + toolMessageTokens: 18, + toolMessageTokenCounts: { search: 11, read_file: 4 }, + sdkExtension: { + providerOnly: true, + nested: { invocationOverhead: 3, labels: ['retained'] }, + }, + }, + sdkExtension: { + source: 'retained-snapshot-field', + nested: { calibration: { mode: 'provider' } }, + }, + }; + const knownZero = { + ...contextUsage, + breakdown: { + ...contextUsage.breakdown, + toolMessageTokens: 0, + toolMessageTokenCounts: { search: 0 }, + }, + }; + const { + toolMessageTokens: _missingToolMessageTokens, + toolMessageTokenCounts: _missingToolMessageTokenCounts, + ...missingBreakdown + } = contextUsage.breakdown; + const missing = { ...contextUsage, breakdown: missingBreakdown }; + + await Promise.all([ + saveMessage(mockCtx, { + ...mockMessageData, + messageId: 'msg-context-rich', + metadata: { contextUsage }, + }), + saveMessage(mockCtx, { + ...mockMessageData, + messageId: 'msg-context-zero', + metadata: { contextUsage: knownZero }, + }), + saveMessage(mockCtx, { + ...mockMessageData, + messageId: 'msg-context-missing', + metadata: { contextUsage: missing }, + }), + ]); + + const persisted = await Message.find({ + user: 'user123', + messageId: { $in: ['msg-context-rich', 'msg-context-zero', 'msg-context-missing'] }, + }).lean(); + const byId = new Map(persisted.map((message) => [message.messageId, message])); + + expect(byId.get('msg-context-rich')?.metadata?.contextUsage).toEqual(contextUsage); + expect(byId.get('msg-context-zero')?.metadata?.contextUsage).toEqual(knownZero); + expect(byId.get('msg-context-missing')?.metadata?.contextUsage).toEqual(missing); + + const zeroBreakdown = ( + byId.get('msg-context-zero')?.metadata?.contextUsage as typeof knownZero + ).breakdown; + expect(zeroBreakdown.toolMessageTokens).toBe(0); + expect(Object.prototype.hasOwnProperty.call(zeroBreakdown, 'toolMessageTokens')).toBe(true); + + const missingBreakdownRecord = ( + byId.get('msg-context-missing')?.metadata?.contextUsage as typeof missing + ).breakdown; + expect( + Object.prototype.hasOwnProperty.call(missingBreakdownRecord, 'toolMessageTokens'), + ).toBe(false); + expect(missingBreakdownRecord.sdkExtension).toEqual(contextUsage.breakdown.sdkExtension); + expect( + (byId.get('msg-context-rich')?.metadata?.contextUsage as typeof contextUsage).sdkExtension + .nested, + ).toEqual({ calibration: { mode: 'provider' } }); + }); + it('unsets a previously stored context meta in the same update as the terminal save', async () => { await saveMessage(mockCtx, { ...mockMessageData,