mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-18 22:14:27 +00:00
* feat: surface retained tool-message context usage * fix: correct post-snapshot context accounting and lockfile registry Adjudicated the review findings on the retained tool-message context work: - package-lock.json resolved four entries (@librechat/agents, call-bind-apply-helpers, es-object-atoms, get-intrinsic) from a private mirror, breaking `npm ci` outside that network. Restored the public registry hosts; the integrity hashes already match the public tarballs. - `finalizeUsage` migrated every usage atom except `subagentUsageFamily` before dropping the temporary `new` key, so a first-turn subagent total vanished when the conversation id was assigned. - The runway projection divided the pre-invoke `remainingContextTokens` by per-call growth, ignoring the output generated after that snapshot, and the compaction hint subtracted a post-call exchange from a pre-call message total. Both now apply the post-snapshot output, so each side is measured at the same point in time. - The estimate path rendered its tool-call share as a peer row although it is a subset of Input/Output/Estimated, so the visible rows summed past `usedTokens`. It is now an indented subtotal; the counted rows stay provider-accurate. - The Totals block presented conversation-wide session subagent usage beside branch-scoped rows; it is now labeled all-branches. - Rewrote the truncated `rgb-series-8` rationale as one comment on the series block. * fix: keep subset accounting whole across cache, legacy snapshots, and custom themes Adjudicated the second review round on the retained tool-message context work. One cause per finding, each measured on the same basis as what it is compared to: - The all-branches subagent subtotal summed input+output only, so a cached subagent call was under-reported and a fully cached call with no output disappeared from the row while the Cache read/write rows above still counted its traffic. It now sums every normalized bucket. - `collectAnchorSeries` read an absent `remainingContextTokens` as zero and recorded that snapshot as having spent its entire budget, which distorted the growth delta feeding the runway projection. A snapshot saved in the older shape now derives its used total from the breakdown, exactly as the render path's `baseUsed` does, and one that carries neither basis is skipped. Each reading is tagged with its basis and the projection compares only readings of the same basis instead of reporting the difference between two measurements. - `resolveTheme` filled an omitted `rgb-series-8` from the bundled base, so a stored or environment theme that paints its own seven-slot scale got LibreChat's indigo on surfaces it never checked: 2.41:1 against that theme's meter track, under the 3:1 mark floor the scale claims. An owned scale now derives the slot from the theme's own neutral text — 11.5:1 on the same reference theme — and naming `rgb-series-8` still opts out. - Ordered the `--series-*` declarations in the light block by slot. * fix: keep cached shares out of the peer rows and track resolved text for slot 8 Third round on the retained tool-message context work, correcting the two variants of invariants this PR already fixed elsewhere rather than patching them locally: - The snapshot path rendered the reconciling call's Cached and Cache write shares as peer rows with window percentages, although reconciliation already counted those tokens inside the segments above. A fully cached prompt showed its tokens twice and the visible rows summed past the meter. They now render as an indented subtotal, the same treatment the estimate path's tool-call share received. The existing test claimed this invariant in its name but only asserted the rows exist, so it now asserts they are not peers. - The `rgb-series-8` fallback required the theme to name its own text, so a theme that repaints the scale and its surfaces while inheriting LibreChat's text colours still received the bundled indigo. It now derives from the RESOLVED secondary text, so the stop tracks whatever the theme reads its body copy against whether it names one or inherits ours. * fix: settle the subagent share with its run and measure both reclaim sides alike Fourth round on the retained tool-message context work: - The subagent share was committed on arrival while the usage it is a subset of stayed pending, so a run that ended with no salvageable response left its tokens in the Totals after `resetLive` discarded them from every rollup, and a resume re-folded the same events into the cumulative figure. It now accumulates in `pendingSubagentUsageFamily` beside the pending usage and settles with it: committed by `flushPendingInto` when the response keeps its usage, dropped by `resetLive` otherwise, migrated with the conversation id. The view merges committed and in-flight shares, so the live row is unchanged. - `latestExchangeTokens` subtracted a summarizing turn's whole response, including the summarization completion the backend folds into `tokenCount` while the snapshot holds it in `breakdown.summaryTokens` rather than `messageTokens`. The reclaim hint was understated by the summary and could reach zero after a long one; the summary block's size is now excluded from the tail's contribution when that turn compacted. - `sumBranch` exposed the tail's char estimate and tool share even when the tail carried a `tokenCount`, so a resumed partial response stayed in the message total while its tool traffic was dropped from the Tool calls split. Both tail figures are now zero for a counted tail, which also stops the estimate path subtracting an estimate that was never summed. * chore: drop the lockfile churn the base branch already carries The SDK bump this PR set out to make landed on dev as 3.8.5, so the branch no longer changes any package.json and its lockfile edits (the 3.8.4 entry plus the transitive call-bind-apply-helpers/es-object-atoms/get-intrinsic bumps that install pulled in) are churn against the base. * fix: give the eighth series slot a value every bundled theme can carry The base branch grew high-contrast palettes that must declare every registry token, and a test that a series slot can carry `text-on-status` at the 3:1 mark floor, because a slot also fills a badge chip under that glyph. Slot 8 failed both: the high-contrast palettes could not name a token that did not exist when they were written, and the dark indigo #8c98e6 carried white at 2.71:1. Dark slot 8 is muted to #8082be, which reads 3.66:1 under the status label and still clears 3.66:1 on every series surface. The high-contrast palettes take their own neutral text as slot 8 — the same stop `resolveTheme` derives for a third-party theme that paints its own scale — because seven saturated hues is all either canvas affords at their stated CVD separation; it is distinguished by lightness at 21:1 on the canvas instead of by hue. * test: exercise the context split through the mock harness Nine scenarios of the acceptance contract, one tagged Playwright test each, driving the real application: a tool round-trip through the e2e MCP server for the Messages/Tool calls split, its keyboard disclosure of per-tool counts, the peer rows staying inside the meter and surviving a reload; a real subagent run for the all-branches total; seeded conversations for the legacy snapshot whose used figure comes from its breakdown and for the estimate path's nested tool share; and a stored theme that owns the series scale for the eighth slot's fallback and its opt-out. * test: keep the tool and subagent scenarios on the viewports that expose them * 📌 feat: Unified Pinned Section With Sidebar Drag Interactions (#15057) * feat: unified pinned section with drag interactions and sidebar polish - Pinned chats and favorite models/agents/specs render as one interleaved, reorderable list persisted via users.pinnedOrder - Drag conversations onto project rows to file them, onto Chats to unfile, and onto the Pinned section to pin (empty section materializes as a drop zone during a conversation drag) - Row-level unpin button on pinned chats matching the favorites' button, and the row action menu shares the same hover/focus treatment - Agent Marketplace entry moved under the New Chat button in the side rail - Bookmarks trigger sized to the search bar, search focus no longer swaps its border color, tag filtering shows a loading state, and opening a project shows row-shaped skeletons * fix: add pinnedOrder to IUser type and sort imports * fix: pinned drag threshold, unpin focus target, and marketplace link * fix: surface failed pinned reorders and skip no-op order writes * fix: address review findings on unified pinned section Move the pinned-order endpoints into packages/api as TypeScript handlers and route their URLs through the data-provider endpoint registry. Raise the entry and key caps so a valid model favorite and an unbounded pin count can both be persisted. Keep the marketplace entry reachable from the mobile drawer, which is the only sidebar surface on small screens. Merge a reorder into the stored order while a bookmark filter hides part of the list, instead of dropping the hidden keys. Add Alt+Arrow keyboard reordering with a live-region announcement, restore the focus handoff when a favorite is removed, and hold the local arrangement until the stored order catches up. Serialize overlapping order writes and discard the optimistic value when the first one fails. Derive the tag-filter loading state from isPreviousData rather than a flag that could stick on. * fix: release drag while renaming and dedupe repeated favorite keys A draggable ancestor swallows drag-select inside the rename input, so both the row's own drag source and the pinned wrapper's release while a title is being edited. The favorites payload validator accepts a repeated entry, and two rows keyed the same would collide in React, point every drag at the first copy, and fail the order endpoint's uniqueness check. Build the pinned list keyed-unique. * fix: address PR review bot findings Make the model favorite key injective. Joining endpoint and model on '::' collided for components containing the delimiter, so two distinct favorites keyed the same and the dedupe hid one; the endpoint is now length-prefixed. Replace the pinned-order entry cap with a total size guard. Pinning has no membership cap and the sidebar query drains every cursor, so any count limit rejects a legitimate list rather than bounding anything; what needs bounding is the user document. Let only the newest order write touch the cache. An earlier write failing rolled back over a newer arrangement that was still in flight, leaving the UI on a stale order while the server held the current one. A confirmed write now takes the server's answer, and a failed newest write discards the value that was never accepted. Persist the favorites array only once the order write succeeds, so a rejected write cannot leave the two stores disagreeing with the rolled-back display. * fix: address second round of PR review bot findings Read favorites membership at write time rather than capturing it at drag end. A favorite added or removed while the order request was in flight was undone by writing the older list back; only the relative order now comes from the drag. Move the pinned-order queue to module scope. The sidebar unmounts whole sections while a search is active, so a component-local ref let a remounted hook start a second queue alongside a request that was still running. Serialize project assignment per conversation. Every drop target owns its own mutation instance and the write is an unconditional update, so two quick drops let whichever request reached the database last decide the project. Read the pinned-order request body null-safely: destructuring a nullish body threw past the handler into Express's generic error path instead of returning the intended 400. * fix: address third round of PR review bot findings Drop the secondary favorites write. No consumer of the favorites array reads its order, so mirroring the arrangement there bought nothing while racing the membership mutations that share that whole-array endpoint; pinnedOrder is the single source of ordering. Always merge the committed order into the stored one. The visible list is not always the whole list, since a bookmark filter hides rows and the pinned query publishes partial results while draining its cursor, and replacing wholesale dropped every key it could not see. Release the local arrangement when the write settles instead of when the stored order matches it key for key. A pin arriving mid-request made that equality unreachable, stranding the snapshot and hiding the new row until remount. Abandon a queued pinned-order write if the signed-in user changed, so a deferred request cannot write one account's order into another's document. * fix: address fourth round of PR review bot findings Guard the pinned-order completion callbacks on the signed-in account. The pre-send check only covers a write that has not left yet; a response arriving after a session change would publish one account's order into the next one's cache, which this query never refetches on its own. Commit a reorder only when the pinned rows handled the drop. Dragging a pin onto a project row or into Chats sweeps the pointer across sibling rows, whose hover handlers shift the order, so a filing action was also saving a move the user never asked for. * fix: address fifth round of PR review bot findings Deselect pinnedOrder at schema level. It is display-only and allowed to grow large, but every authentication request loads the user document, so it was adding its whole payload to paths that never read it. Refuse to reorder until the stored order has loaded. Merging against an absent order and then cancelling its fetch posted only the visible keys and discarded saved positions for good. Prune keys only once the visible list is known to be the whole list. Merging unconditionally grew the array until the size guard rejected every write, so completeness now gates compaction rather than the filter flag alone. Track every renaming row, not just the most recent. RenameForm has no blur cancellation, so a second form can open while the first is still mounted and the single key reconnected the first row's drag source under its open input. Judge drop eligibility by the queued destination. A drag item carries the project its row had when the drag began, so re-dropping a chat back where it started while an assignment was queued read as a no-op. Take mutation ownership from the user atom AuthContext fills on login rather than the user query, which is still empty in that window, and never treat an unknown owner as a match. Restore the pinned section's 30vh cap. Matching the Projects section's 42vh put 84vh of non-shrinking content above Chats and starved it on short viewports. * fix: address sixth round of PR review bot findings Gate reordering and pruning on successful fetches rather than attempted ones. A failed pinned-order GET still reports as fetched, and a failed later page of the pinned drain still publishes partial data and stops fetching, so both gates were opening on exactly the states they were meant to exclude. Scope the post-unpin focus handoff to this section by its own marker. The whole conversations pane is a labelled region too, and the project lists inside it render the same row, where unpinning only clears the flag and the row stays put, so focus was being moved off a row that never went away. Clear the account tracker when the mutation hook unmounts. The sidebar drops that section during a search, leaving a stale non-null id that also shadowed the cache fallback, so a deferred write could pass its owner check after a session change. * fix: gate pruning on a delivered favorites list A favorites fetch that exhausted its retries leaves isLoading false with the list empty, so an otherwise complete pinned load enabled pruning and dropped every favorite key from the stored order. useFavorites now exposes the query's success state and the pruning gate requires it, matching the pinned-order and pinned-membership sources. * fix: address eighth round of PR review bot findings Identify queued assignments by token rather than by destination. Comparing destination values cannot tell two drops onto the same project apart, so a B, A, B run let the first B's cleanup discard an entry the still-queued later drops owned, and the next drop was then judged against a stale project. Report the rename ending when a row unmounts. A pinned row removed mid-rename, for instance by unpinning the same chat from the project list that also renders it, left its key in the renaming set and came back permanently undraggable. * fix: address ninth round of PR review bot findings Stand the keyboard reorder down wherever the pointer drag does. Alt/Option plus an arrow moves the caret word-wise inside a text field, so the shortcut was moving and saving the whole row out from under an open rename input. Hold a settled assignment destination until a drag item shows the new project. The mutation's cache invalidations are not awaited, so a row still reported its old project for a moment after the write succeeded and a quick drag back to where it started read as a no-op. * fix: address tenth round of PR review bot findings Serialize project assignment inside the mutation instead of at one caller. The drag targets, the row menu and the project dialog each hold their own instance, so a queued drop could still overwrite a newer menu write; moving the queue down covers every surface and lets the drag helper drop its own. Tie the live snapshot's release to the arrangement that created it. An older write settling mid-drag cleared the newer arrangement, snapping the rows back to a stale order while the drag still held its own, so releasing saved an order the user could no longer see. * fix: address eleventh round of PR review bot findings Move assignment destination tracking next to the queue that owns it. Keeping it in the drag helper meant a menu or dialog move left the drag helper still reporting its own older destination, with no row left to observe that would ever clear it, so a later move back was rejected for good. Resolve a conversation's project from the write the mutation confirmed rather than from whichever list rendered the row. The same chat appears in the Pinned section and its expanded project, and those caches refresh independently, so observing one of them was never a sound way to decide the other had caught up. The conversation cache is written synchronously on success, which makes the lazy self-clearing and its settled flag unnecessary. * fix: cancel an in-flight conversation refetch before confirming an assignment Navigation invalidates the exact conversation query, so a refetch that read the old project can be in flight when an assignment resolves and land afterwards, reverting the move in the cache that now answers where a conversation lives. * fix: keep queued pinned-order writes attributable across an unmount Refuse a queued write only when a different account is positively established. Clearing the identity on unmount stopped stale attributions but stranded work that belonged to the same account, since the sidebar drops the section that owns this hook whenever a search is active, and it denied that work its rollback as well. Correct the cache from the request rather than from onError. The observer callbacks do not run once that section unmounts, so an order the server never accepted could survive on a query that refetches on nothing. * fix: publish the session identity from the user atom Queued pinned-order writes were attributed by a tracker that only the pinned section's hook updated, and the sidebar unmounts that section during a search. A retained value went stale across a sign-in, and login clears the user query before the next account's fetch resolves, so a write queued by the previous account could pass its own check and post on the new account's credentials. A Recoil atom effect publishes the identity instead: it runs wherever the atom is set, including sign-out, so attribution comes from the state itself rather than from a component that happens to be mounted. That makes a signed-out session foreign to every owner and lets the check be a strict comparison again, rather than a rule that had to guess when identity was merely unreadable. * fix: discard reorders that ended on a refused external target A drop the pinned rows did not handle usually means a reorder, but a project row or the Chats section refusing one reports the same, so dragging a chat onto the project it already belongs to was saving the incidental shift its pointer caused on the way out. The rows and those targets now report their hovers, so the last one under the pointer settles it, which also allows for a drag that strays over a project and returns to reorder after all. * fix: attribute queued writes to the credentials they will travel with Publishing the account from React state, whether an effect or a Recoil atom effect, always lagged the credential change: setUserContext installs the new Authorization header in one synchronous call, and a write queued by the previous account and sent in that gap read its own owner back and went out as the new one. A test against the atom effect showed onSet is deferred too, so the atom was no better than the effect it replaced. The header is the thing that decides who the server sees, so ownership is read from it. The id claim is compared rather than the token, since a silent refresh rotates the token for the same account and discarding that work would strand something the user is still entitled to. Leaves AuthContext and the user atom untouched. * fix: attribute a reorder to the account that started it onMutate awaits cancelQueries before the request is built, and the mutation function was reading the session again afterwards, so credentials turning over inside that gap sent one account's order as the next. The owner is now recorded in onMutate's first synchronous statement and carried on the order itself, and the optimistic write is skipped when the session moved on, since that cache is shared and showing an order to the wrong account is the same mistake as sending it. Record a pinned hover before the guards rather than after. Hovering the dragged row is a no-op for ordering but still means the pointer is inside the list, so a drag that wandered onto a project and returned to its own position was being discarded as a filing action. * fix: address a further round of PR review bot findings Mark a hover anywhere in the pinned section as internal. The header and the padding around the rows are part of the list, so a drag that wandered onto a project and came back to rest on one of them ran no row hover and was discarded as a filing action. Hold off pruning while a favorites write is unsettled. The earlier GET's success stands through an optimistic removal, so pruning then dropped the ordering key of a write that had not landed, and a failed one came back without its saved position. Declare the reorder shortcut on the elements that take focus. The section's written hint is skipped by anyone tabbing straight to a row, leaving the only non-pointer way to reorder undiscoverable. * fix: reach a real New Chat control, and let the shortcut prop through the memo The unpin fallback looked for a test id no component carries, so once the last pinned row unmounted focus fell to the document rather than to New Chat. It now tries the controls that exist, in sidebar order. The test that covered this had fabricated the missing id, so it proved nothing. Convo is memoized on conversation fields and isGenerating alone, so the shortcut appearing once the pinned order loaded never reached the element that announces it. The comparator now accounts for it. * fix: identify OpenID sessions, and see favorites writes from every hook With OpenID token reuse the bearer is the provider's own id_token, which names its subject as sub under an issuer and carries no LibreChat id, so every such session resolved to nobody and two different accounts compared equal. That is the case the ownership guard exists for, so it read as safe while being exactly wrong. Sessions are identified by whatever the credential offers now, and an owner that cannot be identified is foreign to everything including itself. Count favorites writes across the query client rather than from one observer. A favorite row runs its own useFavorites, so a removal started there was invisible to the pinned section and its ordering key could still be pruned. * fix: close the assignment cache gap, and let a failed drain be retried The assignment released its pending destination as soon as the request settled, while the cache write waited on an awaited cancellation in onSuccess. A drag started in that gap saw neither, fell back to the stale row and refused a valid drop. The write now happens in the request, before the entry is released, which also survives the owning component unmounting. Also clear the pagination guard when the Chats section collapses. A drain that exhausted its retries leaves the conversations array unchanged, so the guard barred every later attempt and the remaining chats stayed unreachable for the session. Reopening the section now tries once more. That guard came from #14860 rather than this branch, so it can be split out if preferred. * fix: compose the marketplace link, and keep focus on a surviving row The marketplace control was a raw link with restated appearance classes, and in restating them it left out the focus ring entirely, so keyboard focus on it was invisible. It composes through the shared button now, which carries the ring, the hover fill and the theme timing. Unpinning from a project list leaves the row in place but removes the pin badge that had focus, and the pinned-section successor search deliberately returns nothing there, so focus fell to the document. It moves to the row's own link. Reconcile the pinned order on window focus and reconnect. It is one shared per-user record, so a second tab would otherwise show a stale arrangement and its next reorder would post a full array undoing the other tab's. * fix: keep a rename from restoring a conversation's other fields A rename request carries only a title, but its response is the whole conversation as the server saw it, and writing all of that back restores the pre-request value of every other field. An assignment confirmed while the rename was in flight was therefore reverted in the cache that now answers which project a conversation belongs to, and a drag back to the original project read as a no-op. Only the title and its timestamp are taken from the response. Cancelling queries cannot help here, since this is another mutation's write rather than a fetch. * fix: compose the unpin badge through the shared row-action button The badge restated a reusable icon control's sizing, radius, hover and focus classes. The client package already had a row-action variant and an icon-xs size that match it exactly, so it uses those instead of its own copy. * fix: reconcile before reordering, and scope assignments to their session A reorder stayed enabled while the order's focus refetch was in flight, so returning to a tab could cancel that refetch and post the arrangement it was about to replace. Pruning had the matching gap: the pinned list keeps its data fresh for five minutes, so a tab whose order had reconciled could prune against membership that had not, dropping the position of a conversation another tab pinned. Both now wait for data that is settled and not stale. Assignments carry the account that started them, like the order writes already did. Conversation ids are per-user, so a queued one travelling with the next account's credentials would act on whatever that id names over there, and its answer would describe one account's conversation to another in a cache that is not scoped by user. The row's overflow trigger derives from the shared row-action variant too, so the two actions in a row stop keeping separate copies of the same recipe. * fix: reject foreign assignment results, and share the section header An assignment settling after a session change returned normally, so the success path still ran: the caller reported success and the response could be applied to the next account's conversation, since ids repeat across accounts. It rejects now, so nothing downstream treats it as a result. The collapsible section header existed as three identical copies, Chats, Projects and Pinned, and this branch added the third. It moves into the shared button as a section-header variant and all three compose from it. * fix: let favorites membership reconcile between tabs The favorites query disabled focus, reconnect and mount refetches, so a second tab kept its old list indefinitely. Its pruning gate waits for a fetch, and none was ever going to happen, so a reorder there could delete the saved position of a favorite the first tab had added. It reconciles on focus and reconnect now, like the order it is arranged alongside. * fix: hold reconciling refetches off while a write is in flight Enabling focus and reconnect refetches on the order and favorites queries let one start after onMutate's single cancellation, read the pre-write state and land after the write, putting the old value back. Both now decline those refetches while a write for that resource is outstanding, keeping the cross-tab reconciliation itself. The check asks the query client at the moment the event fires rather than reading useIsMutating: that hook only reaches the query's options on the next render, which is a race against the very event being gated. A test that dispatches focus mid-write fails without it. * fix: release abandoned assignments and gate pruning on current membership An assignment rejected before entering the try block kept its pending entry forever, so a later session read a destination that was never sent. The ownership check now sits inside the block that releases it, and it runs again after the conversation query cancellation resolves, since the session can turn over while that await is pending. Pruning the pinned order now requires membership at least as current as the order it is pruned against. The pinned query holds its data for five minutes while the order has no such window, so focusing a tab reconciled the order alone and the next reorder dropped the key of a conversation another tab had pinned in between. The assignment spec resets its mocks between tests and gives each test its own conversation id, so one test's unconsumed mock and leftover queue entries no longer decide the next test's result. * fix: address the sixth round of review findings on the pinned section The section header recipe asked for the shared button variant without a size, so CVA emitted the default `h-10 px-4` after it and the merge kept it, putting a 40px control in a 32px header row. The variant opts out of default sizing through a compound now, since a heading is sized by its own text. Restore the search field's focus border. Removing it left keyboard focus signalled only by a background change that hover already uses, so the field had no distinguishable focus state at all. An assignment can reach the database before a concurrent rename and still answer after it, carrying a conversation that predates the rename. Only the project field is merged into the cached conversation and published to the active one now, matching what the rename handler already does. An order refetch that starts mid drag no longer lets the drop post an array built from the order that refetch is about to replace: the arrangement is held and written once the fresh order lands. Reorder writes superseded by a newer arrangement are skipped rather than sent, so holding the shortcut costs two writes instead of one per intermediate position. The order and favorites queries reconcile on mount as well, since a sidebar search unmounts the section and a query with no observer never sees the focus event. * fix: preserve pinned order until favorites membership catches up * fix: reconcile pinned membership and preserve auth and focus state * fix: preserve reconciled keys during active pinned arrangements * style: sort pinned order regression imports * fix: reconcile favorite rollbacks before pinned-order pruning * 🧊 fix: Withhold Rerun Controls on Compaction Turns (#15797) * 🧊 fix: Withhold Rerun Controls on Compaction Turns A compaction turn's response parents onto the leaf it summarized, not onto a user message, so the hover Edit's Rerun replayed an assistant message in the user slot: the request minted a user message under the existing assistant leaf's id and ran the turn with empty text, which fails. Editing offered nothing either, since SUMMARY is not an editable part type. Edit, Regenerate and Continue are now withheld on a compaction turn, and both rerun paths refuse a parent that is not user-created, matching the guard `useChatFunctions.regenerate` already applied. * 🧭 fix: Gate Rerun Controls on the Replayed Parent Turn Withholding Edit, Regenerate and Continue on summary-only content also hit an ordinary turn that auto-summarized and was cancelled before its first answer token: that response is parented to the user's message and is exactly the turn a rerun is for. The row now resolves the message it would replay and withholds the rerun shapes only when a model turn hangs off another model turn, which is the manual compaction shape. An unresolvable parent withholds nothing. `isCompactionTurn` is gone with it, and `isCompactedLeaf` keeps its original body; its docstring now says per part what it always measured per part. * 🧮 fix: Identify a Compaction Turn by Content and Parent Gating the rerun controls on parent authorship alone withheld Edit, Regenerate and Continue from any model turn hanging off another model turn. An imported conversation chains messages sequentially, so a skipped empty human message leaves an ordinary reply parented to the previous reply; its stored content is editable and its controls must stay. It also made every model row resolve its parent out of the flat message list: 100 full-thread scans while mounting a 200-message conversation. A manual compaction is now both halves of its shape — summary-only content and a parent that is not the user's turn — so only such a row consults the thread. `isSummaryOnlyContent` is the content half, in data-provider next to `isCompactedLeaf`. * 🧷 test: Pin the Rerun Refusal on a Chained Model Turn A rerun replays its parent as the user turn, so a model turn parented onto another model turn must submit nothing rather than mint a user message under an existing response's id. Removing the guard makes the case call `ask` once. * 🧱 fix: Gate the Editor on Content and the Rerun on the Parent Identifying a compaction turn by content shape missed the ones the server persists with an error part (a skipped or failed compaction keeps its explanation instead of a summary), which left Edit, Regenerate and Continue on a turn where every rerun shape is inert. The two controls answer two different questions now. A rerun replays the message's parent as the turn's user message, so Regenerate and Continue go whenever a model turn hangs off a model turn. The editor goes only when that turn also has no part to type in: an imported chain's reply is saved directly and stays editable, while a compaction — summary or error part — would open an empty editor with one inert Rerun. `findMessageById` indexes the messages array once per identity, so resolving the parent no longer scans the thread per row: 100 full-array walks while mounting a 200-message conversation become one. * 🏷️ fix: Honor the Server's Compaction Marker on Any Leaf Compact runs on whatever leaf the branch ends with, so a compaction can hang off a user message rather than off the answer it summarized. Inferring the turn from its position alone left the rerun controls on that one: replaying the parent would answer the user's earlier message again instead of redoing the compaction. `isUserInitiatedCompaction` reads the marker the server already stamps — `markCompactionSummary` sets `initiatedBy: 'user'`, and nothing else writes it, so an automatic summary detour keeps its controls. A marked turn is excluded from the rerun shapes wherever it hangs. * 🎨 fix: Ask the Editor What It Can Edit The hover row had its own idea of an editable part and counted artifact-bearing text as one, while `EditContentParts` keeps an artifact in its read-only renderer. A chained model turn whose only content is an artifact therefore still offered Edit, and opening it gave a read-only artifact, a disabled Save and a Rerun the guards refuse. `editableParts.ts` now owns that rule for both: `isEditablePart` is the editor's own filter, `hasEditablePart` asks it per message. The row resolves it lazily — the artifact check parses markdown, and only a turn with nothing to replay needs the answer — so a streaming row never pays for it. * 🚫 fix: Withhold the Editor's Rerun Instead of Refusing It Both editors resolved the parent inside the handler and returned when it was not a user turn, so a chained model turn — an imported thread with no user message between two replies — kept an enabled Rerun that did nothing, with Ctrl/Cmd+Enter bound to it. Each editor resolves that parent at render and withholds the action instead: the button is gone, the shortcut is inert and `aria-keyshortcuts` no longer advertises it, while Save keeps writing the edit. An unresolvable parent keeps the action, matching the hover row. * 🧹 style: Sort the New Imports in Both Editors CI's import-sort check failed on the two files the previous commit touched. * 💬 fix: Report Unsaved Changes When No Rerun Is Offered A save-only editor still warned about the rerun. An answer's draft is discarded by the rerun, and multi-part edits must be saved before one, but neither is true where the action is withheld — a model turn chained onto another model turn — so the status slot claimed a precondition for something the footer does not offer. Both statuses are gated on the same `canRerun` the button is: the plain-text editor reports unsaved changes, and `rerunRequiresSave` cannot hold without a rerun to require it. * 🧪 test: Exercise the Compaction Controls in the Browser Eight mock-harness scenarios drive the real row and the real editors: a finished, failed, error-part and user-parented compaction withhold Edit, Regenerate and Continue while Copy stays; a turn that only auto-summarized keeps all three; a reply chained onto a reply keeps a save-only editor whose status reports unsaved changes; an artifact-only chained reply loses the editor; and the context indicator still offers Compact after a failed compaction. `SeedMessage` carries `content`, `finish_reason` and `model` so a branch can be seeded in the shapes the mock model cannot produce — a persisted summary, an error part, an artifact. * 🩻 feat: improve agent filter controls for insights (#15807) * feat(insights): improve agent filter controls * feat(insights): add search and fixed agent selection controls * fix(insights): show search before all agents control * test(e2e): allow steer consumption before arm response * 🔠 fix: Name Artifact Downloads After Their Titles (#15836) * fix: Use artifact titles for download filenames * fix: Match artifact filenames to downloaded content * fix: Preserve raw artifact extensions and GFM headings * fix: Download complete artifact files and retain filename provenance * fix: Distinguish cached artifact exports from originals * fix: Name unnamed artifact originals by their media type * fix: Recognize legacy CSV MIME in artifact filenames * fix: preserve context insights across reloads and configuration changes * fix: retain primary call details when resuming context snapshots * fix: preserve resumed output and retained tool-result estimates * fix: keep resumed output compatible with older clients --------- Co-authored-by: Ravi Kumar L <upman@users.noreply.github.com> Co-authored-by: Danny Avila <danny@librechat.ai>
2506 lines
83 KiB
TypeScript
2506 lines
83 KiB
TypeScript
import type { TContextUsageEvent, TTokenUsageEvent } from 'librechat-data-provider';
|
||
import type { RecordUsageDeps, RecordUsageParams, SubagentUsageEvent } from './usage';
|
||
import type { UsageMetadata } from '../stream/interfaces/IJobStore';
|
||
import type { BulkWriteDeps, PricingFns } from './transactions';
|
||
import {
|
||
computeUsageCostUSD,
|
||
aggregateEmittedUsage,
|
||
createDetachedSubagentUsageRecorder,
|
||
createSubagentUsageSink,
|
||
aggregateCollectedUsage,
|
||
recordCollectedUsage,
|
||
resolveAgentTokenConfig,
|
||
buildPersistedContextUsage,
|
||
buildAbortedResponseMetadata,
|
||
computeSummaryUsedTokens,
|
||
priorRunOutputTokens,
|
||
} from './usage';
|
||
import { runWithDetachedSubagentUsage } from './subagentTaskContext';
|
||
|
||
describe('aggregateCollectedUsage', () => {
|
||
it('preserves the no-child baseline and ignores absent entries', () => {
|
||
expect(
|
||
aggregateCollectedUsage([{ input_tokens: 100, output_tokens: 40, provider: 'openai' }, null]),
|
||
).toEqual({
|
||
total: {
|
||
inputTokens: 100,
|
||
outputTokens: 40,
|
||
totalTokens: 140,
|
||
cacheReadTokens: 0,
|
||
reasoningTokens: 0,
|
||
},
|
||
primary: {
|
||
inputTokens: 100,
|
||
outputTokens: 40,
|
||
totalTokens: 140,
|
||
cacheReadTokens: 0,
|
||
reasoningTokens: 0,
|
||
},
|
||
subagent: {
|
||
inputTokens: 0,
|
||
outputTokens: 0,
|
||
totalTokens: 0,
|
||
cacheReadTokens: 0,
|
||
reasoningTokens: 0,
|
||
},
|
||
});
|
||
});
|
||
|
||
it('includes multiple child calls once in the combined and subagent totals', () => {
|
||
const result = aggregateCollectedUsage([
|
||
{ input_tokens: 100, output_tokens: 40, provider: 'openai' },
|
||
{
|
||
input_tokens: 25,
|
||
output_tokens: 10,
|
||
provider: 'openai',
|
||
usage_type: 'subagent',
|
||
},
|
||
{
|
||
input_tokens: 35,
|
||
output_tokens: 15,
|
||
provider: 'openai',
|
||
usage_type: 'subagent',
|
||
},
|
||
]);
|
||
|
||
expect(result.total).toEqual(
|
||
expect.objectContaining({ inputTokens: 160, outputTokens: 65, totalTokens: 225 }),
|
||
);
|
||
expect(result.subagent).toEqual(
|
||
expect.objectContaining({ inputTokens: 60, outputTokens: 25, totalTokens: 85 }),
|
||
);
|
||
});
|
||
|
||
it('uses provider-aware cache normalization for primary and child calls', () => {
|
||
const result = aggregateCollectedUsage([
|
||
{
|
||
input_tokens: 200,
|
||
output_tokens: 80,
|
||
provider: 'anthropic',
|
||
input_token_details: { cache_creation: 60, cache_read: 30 },
|
||
},
|
||
{
|
||
input_tokens: 100,
|
||
output_tokens: 50,
|
||
provider: 'bedrock',
|
||
usage_type: 'subagent',
|
||
input_token_details: { cache_creation: 20, cache_read: 10 },
|
||
},
|
||
]);
|
||
|
||
expect(result.primary.inputTokens).toBe(200);
|
||
expect(result.subagent.inputTokens).toBe(130);
|
||
expect(result.total.cacheReadTokens).toBe(40);
|
||
});
|
||
|
||
it('repairs provider output undercounts and aggregates reasoning details', () => {
|
||
const result = aggregateCollectedUsage([
|
||
{
|
||
input_tokens: 64,
|
||
output_tokens: 2674,
|
||
total_tokens: 3379,
|
||
provider: 'vertexai',
|
||
output_token_details: { reasoning: 641 },
|
||
},
|
||
{
|
||
input_tokens: 20,
|
||
output_tokens: 10,
|
||
provider: 'openai',
|
||
usage_type: 'subagent',
|
||
output_token_details: { reasoning_tokens: 3 },
|
||
},
|
||
]);
|
||
|
||
expect(result.total.outputTokens).toBe(3325);
|
||
expect(result.total.reasoningTokens).toBe(644);
|
||
});
|
||
});
|
||
|
||
describe('recordCollectedUsage', () => {
|
||
let mockSpendTokens: jest.Mock;
|
||
let mockSpendStructuredTokens: jest.Mock;
|
||
let deps: RecordUsageDeps;
|
||
|
||
const baseParams: Omit<RecordUsageParams, 'collectedUsage'> = {
|
||
user: 'user-123',
|
||
conversationId: 'convo-123',
|
||
model: 'gpt-4',
|
||
context: 'message',
|
||
balance: { enabled: true },
|
||
transactions: { enabled: true },
|
||
};
|
||
|
||
beforeEach(() => {
|
||
jest.clearAllMocks();
|
||
mockSpendTokens = jest.fn().mockResolvedValue(undefined);
|
||
mockSpendStructuredTokens = jest.fn().mockResolvedValue(undefined);
|
||
deps = {
|
||
spendTokens: mockSpendTokens,
|
||
spendStructuredTokens: mockSpendStructuredTokens,
|
||
};
|
||
});
|
||
|
||
describe('basic functionality', () => {
|
||
it('should return undefined if collectedUsage is empty', async () => {
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage: [],
|
||
});
|
||
|
||
expect(result).toBeUndefined();
|
||
expect(mockSpendTokens).not.toHaveBeenCalled();
|
||
expect(mockSpendStructuredTokens).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it('should return undefined if collectedUsage is null-ish', async () => {
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage: null as unknown as UsageMetadata[],
|
||
});
|
||
|
||
expect(result).toBeUndefined();
|
||
expect(mockSpendTokens).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it('should handle single usage entry correctly', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledTimes(1);
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({
|
||
user: 'user-123',
|
||
conversationId: 'convo-123',
|
||
model: 'gpt-4',
|
||
context: 'message',
|
||
}),
|
||
{ promptTokens: 100, completionTokens: 50 },
|
||
);
|
||
expect(result).toEqual({ input_tokens: 100, output_tokens: 50 });
|
||
});
|
||
|
||
it('should skip null entries in collectedUsage', async () => {
|
||
const collectedUsage = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
null,
|
||
{ input_tokens: 200, output_tokens: 60, model: 'gpt-4' },
|
||
] as UsageMetadata[];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledTimes(2);
|
||
expect(result).toEqual({ input_tokens: 100, output_tokens: 110 });
|
||
});
|
||
});
|
||
|
||
describe('sequential execution (tool calls)', () => {
|
||
it('should calculate tokens correctly for sequential tool calls', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
{ input_tokens: 150, output_tokens: 30, model: 'gpt-4' },
|
||
{ input_tokens: 180, output_tokens: 20, model: 'gpt-4' },
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledTimes(3);
|
||
expect(result?.output_tokens).toBe(100); // 50 + 30 + 20
|
||
expect(result?.input_tokens).toBe(100); // First entry's input
|
||
});
|
||
});
|
||
|
||
describe('summarization usage segregation', () => {
|
||
it('includes summarization output tokens in total while billing under separate context', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
usage_type: 'message',
|
||
input_tokens: 120,
|
||
output_tokens: 40,
|
||
model: 'gpt-4',
|
||
},
|
||
{
|
||
usage_type: 'summarization',
|
||
input_tokens: 30,
|
||
output_tokens: 12,
|
||
model: 'gpt-4.1-mini',
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(result).toEqual({ input_tokens: 120, output_tokens: 52 });
|
||
expect(mockSpendTokens).toHaveBeenCalledTimes(2);
|
||
expect(mockSpendTokens).toHaveBeenNthCalledWith(
|
||
1,
|
||
expect.objectContaining({ context: 'message', model: 'gpt-4' }),
|
||
expect.any(Object),
|
||
);
|
||
expect(mockSpendTokens).toHaveBeenNthCalledWith(
|
||
2,
|
||
expect.objectContaining({
|
||
context: 'summarization',
|
||
model: 'gpt-4.1-mini',
|
||
}),
|
||
expect.any(Object),
|
||
);
|
||
});
|
||
});
|
||
|
||
describe('subagent usage segregation', () => {
|
||
it('bills subagent entries under separate context while excluding them from reported totals', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
usage_type: 'message',
|
||
input_tokens: 120,
|
||
output_tokens: 40,
|
||
model: 'gpt-4',
|
||
},
|
||
{
|
||
usage_type: 'subagent',
|
||
input_tokens: 900,
|
||
output_tokens: 700,
|
||
model: 'claude-haiku-4-5',
|
||
provider: 'anthropic',
|
||
},
|
||
{
|
||
usage_type: 'subagent',
|
||
input_tokens: 1100,
|
||
output_tokens: 300,
|
||
model: 'claude-haiku-4-5',
|
||
provider: 'anthropic',
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
/**
|
||
* `input_tokens` comes from the first MESSAGE usage; `output_tokens`
|
||
* excludes subagent completions — the result becomes the parent
|
||
* response message's tokenCount, and child output the parent never
|
||
* saw must not distort next-turn context accounting.
|
||
*/
|
||
expect(result).toEqual({ input_tokens: 120, output_tokens: 40 });
|
||
/** ...but every subagent call is still billed against balance. */
|
||
expect(mockSpendTokens).toHaveBeenCalledTimes(3);
|
||
expect(mockSpendTokens).toHaveBeenNthCalledWith(
|
||
2,
|
||
expect.objectContaining({
|
||
context: 'subagent',
|
||
model: 'claude-haiku-4-5',
|
||
}),
|
||
{ promptTokens: 900, completionTokens: 700 },
|
||
);
|
||
expect(mockSpendTokens).toHaveBeenNthCalledWith(
|
||
3,
|
||
expect.objectContaining({
|
||
context: 'subagent',
|
||
model: 'claude-haiku-4-5',
|
||
}),
|
||
{ promptTokens: 1100, completionTokens: 300 },
|
||
);
|
||
});
|
||
|
||
it('bills hidden sequential-agent usage but excludes it from reported totals', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ usage_type: 'message', input_tokens: 120, output_tokens: 40, model: 'gpt-4' },
|
||
{ usage_type: 'sequential', input_tokens: 800, output_tokens: 250, model: 'gpt-4' },
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, { ...baseParams, collectedUsage });
|
||
|
||
/** Hidden intermediate output stays out of the parent's tokenCount... */
|
||
expect(result).toEqual({ input_tokens: 120, output_tokens: 40 });
|
||
/** ...but is still billed under its own context. */
|
||
expect(mockSpendTokens).toHaveBeenCalledTimes(2);
|
||
expect(mockSpendTokens).toHaveBeenNthCalledWith(
|
||
2,
|
||
expect.objectContaining({ context: 'sequential', model: 'gpt-4' }),
|
||
{ promptTokens: 800, completionTokens: 250 },
|
||
);
|
||
});
|
||
|
||
it('does not let a leading subagent entry hijack reported input_tokens', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
usage_type: 'subagent',
|
||
input_tokens: 5000,
|
||
output_tokens: 900,
|
||
model: 'claude-haiku-4-5',
|
||
},
|
||
{
|
||
input_tokens: 120,
|
||
output_tokens: 40,
|
||
model: 'gpt-4',
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(result).toEqual({ input_tokens: 120, output_tokens: 40 });
|
||
expect(mockSpendTokens).toHaveBeenCalledTimes(2);
|
||
});
|
||
|
||
it('uses structured spend for subagent entries with cache tokens', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
{
|
||
usage_type: 'subagent',
|
||
input_tokens: 200,
|
||
output_tokens: 80,
|
||
model: 'claude-haiku-4-5',
|
||
provider: 'anthropic',
|
||
input_token_details: { cache_creation: 60, cache_read: 30 },
|
||
},
|
||
];
|
||
|
||
await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledTimes(1);
|
||
/** Anthropic input_tokens is cache-inclusive, so cache is subtracted out
|
||
* of the billed input: 200 − 60 − 30 = 110 (not double-charged). */
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({
|
||
context: 'subagent',
|
||
model: 'claude-haiku-4-5',
|
||
}),
|
||
{
|
||
promptTokens: { input: 110, write: 60, read: 30 },
|
||
completionTokens: 80,
|
||
},
|
||
);
|
||
});
|
||
});
|
||
|
||
describe('parallel execution (multiple agents)', () => {
|
||
it('should handle parallel agents with independent input tokens', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
{ input_tokens: 80, output_tokens: 40, model: 'gpt-4' },
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledTimes(2);
|
||
expect(result?.output_tokens).toBe(90); // 50 + 40
|
||
expect(result?.output_tokens).toBeGreaterThan(0);
|
||
});
|
||
|
||
it('should NOT produce negative output_tokens for parallel execution', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 200, output_tokens: 100, model: 'gpt-4' },
|
||
{ input_tokens: 50, output_tokens: 30, model: 'gpt-4' },
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(result?.output_tokens).toBeGreaterThan(0);
|
||
expect(result?.output_tokens).toBe(130); // 100 + 30
|
||
});
|
||
|
||
it('should calculate correct total output for multiple parallel agents', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
{ input_tokens: 120, output_tokens: 60, model: 'gpt-4-turbo' },
|
||
{ input_tokens: 80, output_tokens: 40, model: 'claude-3' },
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledTimes(3);
|
||
expect(result?.output_tokens).toBe(150); // 50 + 60 + 40
|
||
});
|
||
});
|
||
|
||
describe('cache token handling - subset providers (input_tokens already includes cache)', () => {
|
||
it('subtracts cache from input_tokens for OpenAI to avoid double-counting', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 100,
|
||
output_tokens: 50,
|
||
model: 'gpt-4',
|
||
provider: 'openAI',
|
||
input_token_details: {
|
||
cache_creation: 20,
|
||
cache_read: 10,
|
||
},
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledTimes(1);
|
||
expect(mockSpendTokens).not.toHaveBeenCalled();
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ model: 'gpt-4' }),
|
||
{
|
||
promptTokens: { input: 70, write: 20, read: 10 },
|
||
completionTokens: 50,
|
||
},
|
||
);
|
||
expect(result?.input_tokens).toBe(100);
|
||
});
|
||
|
||
it('does not double-count cache_read for Gemini — issue #12855', async () => {
|
||
// Real numbers from the issue report
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 11125,
|
||
output_tokens: 20,
|
||
model: 'gemini-3-flash-preview',
|
||
provider: 'google',
|
||
input_token_details: { cache_read: 7441 },
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledTimes(1);
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ model: 'gemini-3-flash-preview' }),
|
||
{
|
||
promptTokens: { input: 3684, write: 0, read: 7441 },
|
||
completionTokens: 20,
|
||
},
|
||
);
|
||
expect(result?.input_tokens).toBe(11125);
|
||
});
|
||
|
||
it('also applies to Vertex AI', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 5000,
|
||
output_tokens: 100,
|
||
model: 'gemini-2.5-pro',
|
||
provider: 'vertexai',
|
||
input_token_details: { cache_read: 4000 },
|
||
},
|
||
];
|
||
|
||
await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ model: 'gemini-2.5-pro' }),
|
||
{
|
||
promptTokens: { input: 1000, write: 0, read: 4000 },
|
||
completionTokens: 100,
|
||
},
|
||
);
|
||
});
|
||
|
||
it('handles cache_read >= input_tokens defensively (clamps inputOnly to 0)', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 1000,
|
||
output_tokens: 30,
|
||
model: 'gemini-2.5-pro',
|
||
provider: 'google',
|
||
input_token_details: { cache_read: 1000 },
|
||
},
|
||
];
|
||
|
||
await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ model: 'gemini-2.5-pro' }),
|
||
{
|
||
promptTokens: { input: 0, write: 0, read: 1000 },
|
||
completionTokens: 30,
|
||
},
|
||
);
|
||
});
|
||
|
||
it('falls through to additive (historical default) when provider is missing', async () => {
|
||
// Defensive: an unclassified or pre-this-PR usage entry should keep old behavior
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 100,
|
||
output_tokens: 50,
|
||
model: 'gpt-4',
|
||
input_token_details: { cache_creation: 20, cache_read: 10 },
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ model: 'gpt-4' }),
|
||
{
|
||
promptTokens: { input: 100, write: 20, read: 10 },
|
||
completionTokens: 50,
|
||
},
|
||
);
|
||
expect(result?.input_tokens).toBe(130);
|
||
});
|
||
});
|
||
|
||
describe('cache token handling - Anthropic format', () => {
|
||
it('should use spendStructuredTokens for cache tokens (cache_*_input_tokens)', async () => {
|
||
/** Cache-inclusive input_tokens (140 = 100 fresh + 25 write + 15 read). */
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 140,
|
||
output_tokens: 50,
|
||
model: 'claude-3',
|
||
provider: 'anthropic',
|
||
cache_creation_input_tokens: 25,
|
||
cache_read_input_tokens: 15,
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledTimes(1);
|
||
expect(mockSpendTokens).not.toHaveBeenCalled();
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ model: 'claude-3' }),
|
||
{
|
||
promptTokens: { input: 100, write: 25, read: 15 },
|
||
completionTokens: 50,
|
||
},
|
||
);
|
||
expect(result?.input_tokens).toBe(140);
|
||
});
|
||
});
|
||
|
||
describe('reasoning token handling - issue #13006', () => {
|
||
it('uses total - input when output_tokens undercounts (Vertex stream undercount with details present)', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 80657,
|
||
output_tokens: 766,
|
||
total_tokens: 83265,
|
||
output_token_details: { reasoning: 1842 },
|
||
model: 'gemini-3-flash-preview',
|
||
provider: 'vertexai',
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ model: 'gemini-3-flash-preview' }),
|
||
{ promptTokens: 80657, completionTokens: 2608 },
|
||
);
|
||
expect(result?.output_tokens).toBe(2608);
|
||
});
|
||
|
||
it('uses total - input even when output_token_details is missing (raw langchain google-common path)', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 12,
|
||
output_tokens: 135,
|
||
total_tokens: 309,
|
||
model: 'gemini-3-flash-preview',
|
||
provider: 'vertexai',
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ model: 'gemini-3-flash-preview' }),
|
||
{ promptTokens: 12, completionTokens: 297 },
|
||
);
|
||
expect(result?.output_tokens).toBe(297);
|
||
});
|
||
|
||
it('does not change output when invariant already holds (OpenAI o-series, reasoning already a subset)', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 100,
|
||
output_tokens: 500,
|
||
total_tokens: 600,
|
||
output_token_details: { reasoning: 200 },
|
||
model: 'o1-preview',
|
||
provider: 'openAI',
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ model: 'o1-preview' }),
|
||
{ promptTokens: 100, completionTokens: 500 },
|
||
);
|
||
expect(result?.output_tokens).toBe(500);
|
||
});
|
||
|
||
it('routes correction through structured spend when cache tokens are present', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 80657,
|
||
output_tokens: 766,
|
||
total_tokens: 83265,
|
||
output_token_details: { reasoning: 1842 },
|
||
input_token_details: { cache_read: 30000 },
|
||
model: 'gemini-3-flash-preview',
|
||
provider: 'vertexai',
|
||
},
|
||
];
|
||
|
||
await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ model: 'gemini-3-flash-preview' }),
|
||
{
|
||
promptTokens: { input: 50657, write: 0, read: 30000 },
|
||
completionTokens: 2608,
|
||
},
|
||
);
|
||
});
|
||
|
||
it('no-op when total_tokens is absent or zero', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 100,
|
||
output_tokens: 50,
|
||
model: 'gpt-4',
|
||
provider: 'openAI',
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(expect.anything(), {
|
||
promptTokens: 100,
|
||
completionTokens: 50,
|
||
});
|
||
expect(result?.output_tokens).toBe(50);
|
||
});
|
||
});
|
||
|
||
describe('mixed cache and non-cache entries', () => {
|
||
it('should handle mixed entries correctly', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
{
|
||
input_tokens: 150,
|
||
output_tokens: 30,
|
||
model: 'gpt-4',
|
||
input_token_details: { cache_creation: 10, cache_read: 5 },
|
||
},
|
||
{ input_tokens: 200, output_tokens: 20, model: 'gpt-4' },
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledTimes(2);
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledTimes(1);
|
||
expect(result?.output_tokens).toBe(100); // 50 + 30 + 20
|
||
});
|
||
});
|
||
|
||
describe('model fallback', () => {
|
||
it('should use usage.model when available', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4-turbo' },
|
||
];
|
||
|
||
await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
model: 'fallback-model',
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ model: 'gpt-4-turbo' }),
|
||
expect.any(Object),
|
||
);
|
||
});
|
||
|
||
it('should fallback to param model when usage.model is missing', async () => {
|
||
const collectedUsage: UsageMetadata[] = [{ input_tokens: 100, output_tokens: 50 }];
|
||
|
||
await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
model: 'param-model',
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ model: 'param-model' }),
|
||
expect.any(Object),
|
||
);
|
||
});
|
||
});
|
||
|
||
describe('real-world scenarios', () => {
|
||
it('should correctly sum output tokens for sequential tool calls with growing context', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 31596, output_tokens: 151, model: 'claude-opus' },
|
||
{ input_tokens: 35368, output_tokens: 150, model: 'claude-opus' },
|
||
{ input_tokens: 58362, output_tokens: 295, model: 'claude-opus' },
|
||
{ input_tokens: 112604, output_tokens: 193, model: 'claude-opus' },
|
||
{ input_tokens: 257440, output_tokens: 2217, model: 'claude-opus' },
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(result?.input_tokens).toBe(31596);
|
||
expect(result?.output_tokens).toBe(3006); // 151 + 150 + 295 + 193 + 2217
|
||
expect(mockSpendTokens).toHaveBeenCalledTimes(5);
|
||
});
|
||
|
||
it('should handle cache tokens with multiple tool calls', async () => {
|
||
/** Cache-inclusive Anthropic wire: input_tokens = fresh + write + read. */
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 31596, // 788 fresh + 30808 write
|
||
output_tokens: 163,
|
||
model: 'claude-opus',
|
||
provider: 'anthropic',
|
||
input_token_details: { cache_read: 0, cache_creation: 30808 },
|
||
},
|
||
{
|
||
input_tokens: 35378, // 3802 fresh + 30808 read + 768 write
|
||
output_tokens: 149,
|
||
model: 'claude-opus',
|
||
provider: 'anthropic',
|
||
input_token_details: { cache_read: 30808, cache_creation: 768 },
|
||
},
|
||
{
|
||
input_tokens: 58384, // 26808 fresh + 31576 read
|
||
output_tokens: 225,
|
||
model: 'claude-opus',
|
||
provider: 'anthropic',
|
||
input_token_details: { cache_read: 31576, cache_creation: 0 },
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
// input_tokens = total prompt of first call (cache-inclusive)
|
||
expect(result?.input_tokens).toBe(31596);
|
||
// output_tokens = 163 + 149 + 225 = 537
|
||
expect(result?.output_tokens).toBe(537);
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledTimes(3);
|
||
expect(mockSpendTokens).not.toHaveBeenCalled();
|
||
});
|
||
});
|
||
|
||
describe('error handling', () => {
|
||
it('should catch and log errors from spendTokens without throwing', async () => {
|
||
mockSpendTokens.mockRejectedValue(new Error('DB error'));
|
||
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(result).toEqual({ input_tokens: 100, output_tokens: 50 });
|
||
});
|
||
|
||
it('should catch and log errors from spendStructuredTokens without throwing', async () => {
|
||
mockSpendStructuredTokens.mockRejectedValue(new Error('DB error'));
|
||
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 100,
|
||
output_tokens: 50,
|
||
model: 'gpt-4',
|
||
provider: 'openAI',
|
||
input_token_details: { cache_creation: 20, cache_read: 10 },
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
// openAI is a subset provider → input_tokens already includes cache
|
||
expect(result).toEqual({ input_tokens: 100, output_tokens: 50 });
|
||
});
|
||
});
|
||
|
||
describe('transaction metadata', () => {
|
||
it('should pass all metadata fields to spend functions', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
];
|
||
|
||
const endpointTokenConfig = { 'gpt-4': { prompt: 0.01, completion: 0.03, context: 8192 } };
|
||
|
||
await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
messageId: 'msg-123',
|
||
endpointTokenConfig,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(
|
||
{
|
||
user: 'user-123',
|
||
conversationId: 'convo-123',
|
||
model: 'gpt-4',
|
||
context: 'message',
|
||
messageId: 'msg-123',
|
||
balance: { enabled: true },
|
||
transactions: { enabled: true },
|
||
endpointTokenConfig,
|
||
},
|
||
{ promptTokens: 100, completionTokens: 50 },
|
||
);
|
||
});
|
||
|
||
it('prices each usage with its agent endpoint config (multi-endpoint graph)', async () => {
|
||
/** Two endpoints share a model id but bill at different rates. */
|
||
const primaryConfig = { 'shared-model': { prompt: 0.01, completion: 0.03, context: 8192 } };
|
||
const subagentConfig = { 'shared-model': { prompt: 0.05, completion: 0.15, context: 8192 } };
|
||
const byAgent: Record<string, typeof primaryConfig> = {
|
||
primary: primaryConfig,
|
||
sub: subagentConfig,
|
||
};
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
usage_type: 'message',
|
||
agentId: 'primary',
|
||
input_tokens: 100,
|
||
output_tokens: 50,
|
||
model: 'shared-model',
|
||
},
|
||
{
|
||
usage_type: 'subagent',
|
||
agentId: 'sub',
|
||
input_tokens: 200,
|
||
output_tokens: 80,
|
||
model: 'shared-model',
|
||
},
|
||
];
|
||
|
||
await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
endpointTokenConfig: primaryConfig,
|
||
resolveEndpointTokenConfig: (usage) =>
|
||
usage.agentId != null ? byAgent[usage.agentId] : undefined,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({
|
||
context: 'message',
|
||
model: 'shared-model',
|
||
endpointTokenConfig: primaryConfig,
|
||
}),
|
||
{ promptTokens: 100, completionTokens: 50 },
|
||
);
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({
|
||
context: 'subagent',
|
||
model: 'shared-model',
|
||
endpointTokenConfig: subagentConfig,
|
||
}),
|
||
{ promptTokens: 200, completionTokens: 80 },
|
||
);
|
||
});
|
||
|
||
it('trusts the resolver result, including undefined (built-in pricing for known agents)', async () => {
|
||
const batch = { 'gpt-4': { prompt: 0.01, completion: 0.03, context: 8192 } };
|
||
await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage: [
|
||
{ agentId: 'known-openai', input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
],
|
||
endpointTokenConfig: batch,
|
||
/** A known agent with no configured rates: the resolver returns undefined
|
||
* and the billing path must honor it (built-in pricing), NOT re-apply
|
||
* the batch/primary config. */
|
||
resolveEndpointTokenConfig: () => undefined,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ endpointTokenConfig: undefined }),
|
||
{ promptTokens: 100, completionTokens: 50 },
|
||
);
|
||
});
|
||
|
||
it('should use default context "message" when not provided', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
];
|
||
|
||
await recordCollectedUsage(deps, {
|
||
user: 'user-123',
|
||
conversationId: 'convo-123',
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ context: 'message' }),
|
||
expect.any(Object),
|
||
);
|
||
});
|
||
|
||
it('should allow custom context like "title"', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
];
|
||
|
||
await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
context: 'title',
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ context: 'title' }),
|
||
expect.any(Object),
|
||
);
|
||
});
|
||
});
|
||
|
||
describe('messageId propagation', () => {
|
||
it('should pass messageId to spendTokens', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 10, output_tokens: 5, model: 'gpt-4' },
|
||
];
|
||
|
||
await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
messageId: 'msg-1',
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ messageId: 'msg-1' }),
|
||
expect.any(Object),
|
||
);
|
||
});
|
||
|
||
it('should pass messageId to spendStructuredTokens for cache paths', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 100,
|
||
output_tokens: 50,
|
||
model: 'claude-3',
|
||
cache_creation_input_tokens: 25,
|
||
cache_read_input_tokens: 15,
|
||
},
|
||
];
|
||
|
||
await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
messageId: 'msg-cache-1',
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ messageId: 'msg-cache-1' }),
|
||
expect.any(Object),
|
||
);
|
||
expect(mockSpendTokens).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it('should pass undefined messageId when not provided', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 10, output_tokens: 5, model: 'gpt-4' },
|
||
];
|
||
|
||
await recordCollectedUsage(deps, {
|
||
user: 'user-123',
|
||
conversationId: 'convo-123',
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ messageId: undefined }),
|
||
expect.any(Object),
|
||
);
|
||
});
|
||
|
||
it('should propagate messageId across multiple usage entries', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
{ input_tokens: 200, output_tokens: 60, model: 'gpt-4' },
|
||
{
|
||
input_tokens: 150,
|
||
output_tokens: 30,
|
||
model: 'gpt-4',
|
||
input_token_details: { cache_creation: 10, cache_read: 5 },
|
||
},
|
||
];
|
||
|
||
await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
messageId: 'msg-multi',
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledTimes(2);
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledTimes(1);
|
||
|
||
for (const call of mockSpendTokens.mock.calls) {
|
||
expect(call[0]).toEqual(expect.objectContaining({ messageId: 'msg-multi' }));
|
||
}
|
||
expect(mockSpendStructuredTokens.mock.calls[0][0]).toEqual(
|
||
expect.objectContaining({ messageId: 'msg-multi' }),
|
||
);
|
||
});
|
||
});
|
||
|
||
describe('bulk write path', () => {
|
||
let mockInsertMany: jest.Mock;
|
||
let mockUpdateBalance: jest.Mock;
|
||
let mockPricing: PricingFns;
|
||
let mockBulkWriteOps: BulkWriteDeps;
|
||
let bulkDeps: RecordUsageDeps;
|
||
|
||
beforeEach(() => {
|
||
mockInsertMany = jest.fn().mockResolvedValue(undefined);
|
||
mockUpdateBalance = jest.fn().mockResolvedValue({});
|
||
mockPricing = {
|
||
getMultiplier: jest.fn().mockReturnValue(1),
|
||
getCacheMultiplier: jest.fn().mockReturnValue(null),
|
||
};
|
||
mockBulkWriteOps = {
|
||
insertMany: mockInsertMany,
|
||
updateBalance: mockUpdateBalance,
|
||
};
|
||
bulkDeps = {
|
||
spendTokens: mockSpendTokens,
|
||
spendStructuredTokens: mockSpendStructuredTokens,
|
||
pricing: mockPricing,
|
||
bulkWriteOps: mockBulkWriteOps,
|
||
};
|
||
});
|
||
|
||
it('should use bulk path when pricing and bulkWriteOps are provided', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
];
|
||
|
||
const result = await recordCollectedUsage(bulkDeps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockInsertMany).toHaveBeenCalledTimes(1);
|
||
expect(mockSpendTokens).not.toHaveBeenCalled();
|
||
expect(mockSpendStructuredTokens).not.toHaveBeenCalled();
|
||
expect(result).toEqual({ input_tokens: 100, output_tokens: 50 });
|
||
});
|
||
|
||
it('should batch all entries into a single insertMany call', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
{ input_tokens: 200, output_tokens: 60, model: 'gpt-4' },
|
||
{ input_tokens: 300, output_tokens: 70, model: 'gpt-4' },
|
||
];
|
||
|
||
await recordCollectedUsage(bulkDeps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockInsertMany).toHaveBeenCalledTimes(1);
|
||
const insertedDocs = mockInsertMany.mock.calls[0][0];
|
||
expect(insertedDocs.length).toBe(6); // 2 per entry (prompt + completion)
|
||
});
|
||
|
||
it('should call updateBalance once when balance is enabled', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
{ input_tokens: 200, output_tokens: 60, model: 'gpt-4' },
|
||
];
|
||
|
||
await recordCollectedUsage(bulkDeps, {
|
||
...baseParams,
|
||
balance: { enabled: true },
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockUpdateBalance).toHaveBeenCalledTimes(1);
|
||
expect(mockUpdateBalance).toHaveBeenCalledWith(
|
||
expect.objectContaining({
|
||
user: 'user-123',
|
||
incrementValue: expect.any(Number),
|
||
}),
|
||
);
|
||
});
|
||
|
||
it('should not call updateBalance when balance is disabled', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
];
|
||
|
||
await recordCollectedUsage(bulkDeps, {
|
||
...baseParams,
|
||
balance: { enabled: false },
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockInsertMany).toHaveBeenCalledTimes(1);
|
||
expect(mockUpdateBalance).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it('should handle cache tokens via bulk path', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 100,
|
||
output_tokens: 50,
|
||
model: 'gpt-4',
|
||
input_token_details: { cache_creation: 20, cache_read: 10 },
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(bulkDeps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockInsertMany).toHaveBeenCalledTimes(1);
|
||
expect(mockSpendStructuredTokens).not.toHaveBeenCalled();
|
||
expect(result).toBeDefined();
|
||
});
|
||
|
||
it('should handle mixed cache and non-cache entries in bulk', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
{
|
||
input_tokens: 150,
|
||
output_tokens: 30,
|
||
model: 'gpt-4',
|
||
input_token_details: { cache_creation: 10, cache_read: 5 },
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(bulkDeps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockInsertMany).toHaveBeenCalledTimes(1);
|
||
expect(mockSpendTokens).not.toHaveBeenCalled();
|
||
expect(mockSpendStructuredTokens).not.toHaveBeenCalled();
|
||
expect(result?.output_tokens).toBe(80);
|
||
});
|
||
|
||
it('should fall back to legacy path when pricing is missing', async () => {
|
||
const legacyDeps: RecordUsageDeps = {
|
||
spendTokens: mockSpendTokens,
|
||
spendStructuredTokens: mockSpendStructuredTokens,
|
||
bulkWriteOps: mockBulkWriteOps,
|
||
// no pricing
|
||
};
|
||
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
];
|
||
|
||
await recordCollectedUsage(legacyDeps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledTimes(1);
|
||
expect(mockInsertMany).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it('should fall back to legacy path when bulkWriteOps is missing', async () => {
|
||
const legacyDeps: RecordUsageDeps = {
|
||
spendTokens: mockSpendTokens,
|
||
spendStructuredTokens: mockSpendStructuredTokens,
|
||
pricing: mockPricing,
|
||
// no bulkWriteOps
|
||
};
|
||
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
];
|
||
|
||
await recordCollectedUsage(legacyDeps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockSpendTokens).toHaveBeenCalledTimes(1);
|
||
expect(mockInsertMany).not.toHaveBeenCalled();
|
||
});
|
||
|
||
it('should handle errors in bulk write gracefully', async () => {
|
||
mockInsertMany.mockRejectedValue(new Error('DB error'));
|
||
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
];
|
||
|
||
const result = await recordCollectedUsage(bulkDeps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(result).toEqual({ input_tokens: 100, output_tokens: 50 });
|
||
});
|
||
});
|
||
|
||
describe('Bedrock prompt caching — completion token inflation regression', () => {
|
||
it('does not fold cache_creation into completion on the first cached step', async () => {
|
||
// Bedrock: total = input + output + cache_creation (additive, not subset).
|
||
// Before fix: resolveCompletionTokens returned output + cache_creation (5500)
|
||
// instead of output (500).
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 100,
|
||
output_tokens: 500,
|
||
total_tokens: 5600,
|
||
cache_creation_input_tokens: 5000,
|
||
cache_read_input_tokens: 0,
|
||
model: 'claude-sonnet-4-6',
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, { ...baseParams, collectedUsage });
|
||
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ model: 'claude-sonnet-4-6' }),
|
||
{
|
||
promptTokens: { input: 100, write: 5000, read: 0 },
|
||
completionTokens: 500,
|
||
},
|
||
);
|
||
expect(result?.output_tokens).toBe(500);
|
||
});
|
||
|
||
it('does not fold cache_read into completion on subsequent cached steps', async () => {
|
||
// Bedrock: total = input + output + cache_read on every read step.
|
||
// Before fix: each step returned output + cache_read instead of output.
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 200,
|
||
output_tokens: 300,
|
||
total_tokens: 4500,
|
||
cache_read_input_tokens: 4000,
|
||
cache_creation_input_tokens: 0,
|
||
model: 'claude-sonnet-4-6',
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, { ...baseParams, collectedUsage });
|
||
|
||
expect(mockSpendStructuredTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({ model: 'claude-sonnet-4-6' }),
|
||
{
|
||
promptTokens: { input: 200, write: 0, read: 4000 },
|
||
completionTokens: 300,
|
||
},
|
||
);
|
||
expect(result?.output_tokens).toBe(300);
|
||
});
|
||
|
||
it('handles cache tokens in input_token_details format (alternate field path)', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
input_tokens: 200,
|
||
output_tokens: 300,
|
||
total_tokens: 4500,
|
||
input_token_details: { cache_read: 4000, cache_creation: 0 },
|
||
model: 'claude-sonnet-4-6',
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(deps, { ...baseParams, collectedUsage });
|
||
|
||
expect(result?.output_tokens).toBe(300);
|
||
});
|
||
|
||
it('accumulates only true output across a multi-step cached agent run', async () => {
|
||
// 1 write step + 4 read steps. Without the fix, each step folds its
|
||
// cache tokens into completion, inflating the total by the full cache size.
|
||
const writeStep: UsageMetadata = {
|
||
input_tokens: 100,
|
||
output_tokens: 500,
|
||
total_tokens: 5600,
|
||
cache_creation_input_tokens: 5000,
|
||
cache_read_input_tokens: 0,
|
||
model: 'claude-sonnet-4-6',
|
||
};
|
||
const readSteps: UsageMetadata[] = Array.from({ length: 4 }, (_, i) => ({
|
||
input_tokens: 200,
|
||
output_tokens: 300 + i * 50,
|
||
total_tokens: 200 + (300 + i * 50) + 5000,
|
||
cache_read_input_tokens: 5000,
|
||
cache_creation_input_tokens: 0,
|
||
model: 'claude-sonnet-4-6',
|
||
}));
|
||
|
||
const result = await recordCollectedUsage(deps, {
|
||
...baseParams,
|
||
collectedUsage: [writeStep, ...readSteps],
|
||
});
|
||
|
||
// True output: 500 + 300 + 350 + 400 + 450 = 2000
|
||
const trueOutput = 500 + readSteps.reduce((sum, s) => sum + (s.output_tokens ?? 0), 0);
|
||
expect(result?.output_tokens).toBe(trueOutput);
|
||
});
|
||
});
|
||
|
||
describe('bulk write with summarization usage', () => {
|
||
let mockInsertMany: jest.Mock;
|
||
let mockUpdateBalance: jest.Mock;
|
||
let mockPricing: PricingFns;
|
||
let mockBulkWriteOps: BulkWriteDeps;
|
||
let bulkDeps: RecordUsageDeps;
|
||
|
||
beforeEach(() => {
|
||
mockInsertMany = jest.fn().mockResolvedValue(undefined);
|
||
mockUpdateBalance = jest.fn().mockResolvedValue({});
|
||
mockPricing = {
|
||
getMultiplier: jest.fn().mockReturnValue(1),
|
||
getCacheMultiplier: jest.fn().mockReturnValue(null),
|
||
};
|
||
mockBulkWriteOps = {
|
||
insertMany: mockInsertMany,
|
||
updateBalance: mockUpdateBalance,
|
||
};
|
||
bulkDeps = {
|
||
spendTokens: mockSpendTokens,
|
||
spendStructuredTokens: mockSpendStructuredTokens,
|
||
pricing: mockPricing,
|
||
bulkWriteOps: mockBulkWriteOps,
|
||
};
|
||
});
|
||
|
||
it('combines message and summarization docs into a single bulk write', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
usage_type: 'message',
|
||
input_tokens: 200,
|
||
output_tokens: 80,
|
||
model: 'gpt-4',
|
||
},
|
||
{
|
||
usage_type: 'summarization',
|
||
input_tokens: 50,
|
||
output_tokens: 20,
|
||
model: 'gpt-4.1-mini',
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(bulkDeps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockInsertMany).toHaveBeenCalledTimes(1);
|
||
expect(mockUpdateBalance).toHaveBeenCalledTimes(1);
|
||
expect(mockSpendTokens).not.toHaveBeenCalled();
|
||
expect(mockSpendStructuredTokens).not.toHaveBeenCalled();
|
||
|
||
const insertedDocs = mockInsertMany.mock.calls[0][0];
|
||
// 2 docs per entry (prompt + completion) x 2 entries = 4 docs
|
||
expect(insertedDocs).toHaveLength(4);
|
||
|
||
const messageContextDocs = insertedDocs.filter(
|
||
(d: Record<string, unknown>) => d.context === 'message',
|
||
);
|
||
const summarizationContextDocs = insertedDocs.filter(
|
||
(d: Record<string, unknown>) => d.context === 'summarization',
|
||
);
|
||
expect(messageContextDocs).toHaveLength(2);
|
||
expect(summarizationContextDocs).toHaveLength(2);
|
||
|
||
expect(result).toEqual({ input_tokens: 200, output_tokens: 100 });
|
||
});
|
||
|
||
it('handles summarization-only usage in bulk mode', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{
|
||
usage_type: 'summarization',
|
||
input_tokens: 60,
|
||
output_tokens: 25,
|
||
model: 'gpt-4.1-mini',
|
||
},
|
||
];
|
||
|
||
const result = await recordCollectedUsage(bulkDeps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockInsertMany).toHaveBeenCalledTimes(1);
|
||
expect(mockSpendTokens).not.toHaveBeenCalled();
|
||
expect(mockSpendStructuredTokens).not.toHaveBeenCalled();
|
||
|
||
const insertedDocs = mockInsertMany.mock.calls[0][0];
|
||
expect(insertedDocs).toHaveLength(2);
|
||
|
||
const summarizationContextDocs = insertedDocs.filter(
|
||
(d: Record<string, unknown>) => d.context === 'summarization',
|
||
);
|
||
expect(summarizationContextDocs).toHaveLength(2);
|
||
|
||
expect(result).toEqual({ input_tokens: 0, output_tokens: 25 });
|
||
});
|
||
|
||
it('handles message-only usage in bulk mode', async () => {
|
||
const collectedUsage: UsageMetadata[] = [
|
||
{ input_tokens: 100, output_tokens: 50, model: 'gpt-4' },
|
||
{ input_tokens: 200, output_tokens: 60, model: 'gpt-4' },
|
||
];
|
||
|
||
const result = await recordCollectedUsage(bulkDeps, {
|
||
...baseParams,
|
||
collectedUsage,
|
||
});
|
||
|
||
expect(mockInsertMany).toHaveBeenCalledTimes(1);
|
||
expect(mockSpendTokens).not.toHaveBeenCalled();
|
||
expect(mockSpendStructuredTokens).not.toHaveBeenCalled();
|
||
|
||
const insertedDocs = mockInsertMany.mock.calls[0][0];
|
||
// 2 docs per entry x 2 entries = 4 docs
|
||
expect(insertedDocs).toHaveLength(4);
|
||
|
||
const messageContextDocs = insertedDocs.filter(
|
||
(d: Record<string, unknown>) => d.context === 'message',
|
||
);
|
||
expect(messageContextDocs).toHaveLength(4);
|
||
|
||
expect(result).toEqual({ input_tokens: 100, output_tokens: 110 });
|
||
});
|
||
});
|
||
});
|
||
|
||
describe('createSubagentUsageSink', () => {
|
||
const makeEvent = (overrides: Partial<SubagentUsageEvent> = {}): SubagentUsageEvent => ({
|
||
usage: { input_tokens: 900, output_tokens: 700, total_tokens: 1600 },
|
||
model: 'claude-haiku-4-5',
|
||
provider: 'anthropic',
|
||
subagentType: 'researcher',
|
||
subagentRunId: 'run-1_sub_abc',
|
||
subagentAgentId: 'researcher',
|
||
runId: 'run-1',
|
||
...overrides,
|
||
});
|
||
|
||
it('pushes usage tagged with usage_type subagent and the child model/provider', () => {
|
||
const collectedUsage: UsageMetadata[] = [];
|
||
const sink = createSubagentUsageSink(collectedUsage);
|
||
|
||
sink(makeEvent());
|
||
|
||
expect(collectedUsage).toEqual([
|
||
{
|
||
usage_type: 'subagent',
|
||
input_tokens: 900,
|
||
output_tokens: 700,
|
||
total_tokens: 1600,
|
||
model: 'claude-haiku-4-5',
|
||
provider: 'anthropic',
|
||
agentId: 'researcher',
|
||
},
|
||
]);
|
||
});
|
||
|
||
it('tags the child agent id so the host can price with the subagent endpoint config', () => {
|
||
const collectedUsage: UsageMetadata[] = [];
|
||
const emitted: UsageMetadata[] = [];
|
||
const sink = createSubagentUsageSink(collectedUsage, (u) => {
|
||
emitted.push(u);
|
||
});
|
||
|
||
sink(makeEvent({ subagentAgentId: 'agent_xyz' }));
|
||
|
||
expect(collectedUsage[0].agentId).toBe('agent_xyz');
|
||
/** The same tagged object is handed to onUsage (the live emitter). */
|
||
expect(emitted[0]).toBe(collectedUsage[0]);
|
||
expect(emitted[0].agentId).toBe('agent_xyz');
|
||
});
|
||
|
||
it('prices graph usage with the member agent instead of the synthetic execution subject', () => {
|
||
const collectedUsage: UsageMetadata[] = [];
|
||
const sink = createSubagentUsageSink(collectedUsage);
|
||
|
||
sink(
|
||
makeEvent({
|
||
subagentKind: 'graph',
|
||
subagentAgentId: 'graph:research_team',
|
||
memberAgentId: 'agent_writer',
|
||
}),
|
||
);
|
||
|
||
expect(collectedUsage[0].agentId).toBe('agent_writer');
|
||
});
|
||
|
||
it('falls back to the execution subject when the member agent id is empty', () => {
|
||
const collectedUsage: UsageMetadata[] = [];
|
||
const sink = createSubagentUsageSink(collectedUsage);
|
||
|
||
sink(makeEvent({ subagentAgentId: 'agent_researcher', memberAgentId: '' }));
|
||
|
||
expect(collectedUsage[0].agentId).toBe('agent_researcher');
|
||
});
|
||
|
||
it('preserves cache token details from the child call', () => {
|
||
const collectedUsage: UsageMetadata[] = [];
|
||
const sink = createSubagentUsageSink(collectedUsage);
|
||
|
||
sink(
|
||
makeEvent({
|
||
usage: {
|
||
input_tokens: 200,
|
||
output_tokens: 80,
|
||
total_tokens: 280,
|
||
input_token_details: { cache_creation: 60, cache_read: 30 },
|
||
} as SubagentUsageEvent['usage'],
|
||
}),
|
||
);
|
||
|
||
expect(collectedUsage[0].input_token_details).toEqual({
|
||
cache_creation: 60,
|
||
cache_read: 30,
|
||
});
|
||
});
|
||
|
||
it('omits model/provider tags when the event carries none', () => {
|
||
const collectedUsage: UsageMetadata[] = [];
|
||
const sink = createSubagentUsageSink(collectedUsage);
|
||
|
||
sink(makeEvent({ model: undefined, provider: undefined }));
|
||
|
||
expect(collectedUsage).toHaveLength(1);
|
||
expect(collectedUsage[0].usage_type).toBe('subagent');
|
||
expect(collectedUsage[0]).not.toHaveProperty('model');
|
||
expect(collectedUsage[0]).not.toHaveProperty('provider');
|
||
});
|
||
|
||
it('ignores events without usage', () => {
|
||
const collectedUsage: UsageMetadata[] = [];
|
||
const sink = createSubagentUsageSink(collectedUsage);
|
||
|
||
sink(makeEvent({ usage: undefined as unknown as SubagentUsageEvent['usage'] }));
|
||
|
||
expect(collectedUsage).toEqual([]);
|
||
});
|
||
|
||
it('routes detached usage to its awaited billing and durable child collectors', async () => {
|
||
const collectedUsage: UsageMetadata[] = [];
|
||
const detachedUsage: UsageMetadata[] = [];
|
||
const emitted: UsageMetadata[] = [];
|
||
const recordDetachedUsage = jest.fn().mockResolvedValue(undefined);
|
||
const sink = createSubagentUsageSink(
|
||
collectedUsage,
|
||
(usage) => {
|
||
emitted.push(usage);
|
||
},
|
||
recordDetachedUsage,
|
||
);
|
||
|
||
await runWithDetachedSubagentUsage(detachedUsage, async () => {
|
||
await sink(makeEvent());
|
||
});
|
||
|
||
expect(collectedUsage).toEqual([]);
|
||
expect(detachedUsage).toHaveLength(1);
|
||
expect(emitted[0]).toBe(detachedUsage[0]);
|
||
expect(recordDetachedUsage).toHaveBeenCalledWith(detachedUsage[0]);
|
||
|
||
/** The same sink still batches ordinary foreground subagents with the parent. */
|
||
await sink(makeEvent({ subagentRunId: 'foreground-child' }));
|
||
expect(collectedUsage).toHaveLength(1);
|
||
expect(recordDetachedUsage).toHaveBeenCalledTimes(1);
|
||
});
|
||
|
||
it('still records detached usage when the auxiliary emitter throws', async () => {
|
||
const collectedUsage: UsageMetadata[] = [];
|
||
const detachedUsage: UsageMetadata[] = [];
|
||
const recordDetachedUsage = jest.fn().mockResolvedValue(undefined);
|
||
const sink = createSubagentUsageSink(
|
||
collectedUsage,
|
||
() => {
|
||
throw new Error('parent transport was disposed');
|
||
},
|
||
recordDetachedUsage,
|
||
);
|
||
|
||
await runWithDetachedSubagentUsage(detachedUsage, async () => {
|
||
await sink(makeEvent());
|
||
});
|
||
|
||
expect(collectedUsage).toEqual([]);
|
||
expect(detachedUsage).toHaveLength(1);
|
||
expect(recordDetachedUsage).toHaveBeenCalledWith(detachedUsage[0]);
|
||
});
|
||
|
||
it('round-trips into recordCollectedUsage as billed subagent transactions', async () => {
|
||
const collectedUsage: UsageMetadata[] = [];
|
||
const sink = createSubagentUsageSink(collectedUsage);
|
||
|
||
/** Parent's own call, collected by ModelEndHandler as usual. */
|
||
collectedUsage.push({ input_tokens: 120, output_tokens: 40, model: 'gpt-4' });
|
||
/** Child calls reported through the sink mid-run. */
|
||
sink(makeEvent());
|
||
sink(
|
||
makeEvent({
|
||
usage: { input_tokens: 1100, output_tokens: 300, total_tokens: 1400 },
|
||
}),
|
||
);
|
||
|
||
const spendTokens = jest.fn().mockResolvedValue(undefined);
|
||
const spendStructuredTokens = jest.fn().mockResolvedValue(undefined);
|
||
const result = await recordCollectedUsage(
|
||
{ spendTokens, spendStructuredTokens },
|
||
{
|
||
user: 'user-123',
|
||
conversationId: 'convo-123',
|
||
model: 'gpt-4',
|
||
collectedUsage,
|
||
},
|
||
);
|
||
|
||
/** All three calls billed; child output excluded from reported totals. */
|
||
expect(spendTokens).toHaveBeenCalledTimes(3);
|
||
expect(spendTokens).toHaveBeenNthCalledWith(
|
||
2,
|
||
expect.objectContaining({ context: 'subagent', model: 'claude-haiku-4-5' }),
|
||
{ promptTokens: 900, completionTokens: 700 },
|
||
);
|
||
expect(result).toEqual({ input_tokens: 120, output_tokens: 40 });
|
||
});
|
||
});
|
||
|
||
describe('computeUsageCostUSD', () => {
|
||
/** Stub pricing: base prompt 3 / completion 15, cache write 3.75 / read 0.3;
|
||
* a premium tier above 200k input prompt tokens (8 / 40). Mirrors the
|
||
* shape of the real getMultiplier(inputTokenCount) premium switch. */
|
||
const pricing: PricingFns = {
|
||
getMultiplier: ({ tokenType, inputTokenCount }) => {
|
||
const premium = (inputTokenCount ?? 0) > 200000;
|
||
if (tokenType === 'completion') {
|
||
return premium ? 40 : 15;
|
||
}
|
||
return premium ? 8 : 3;
|
||
},
|
||
getCacheMultiplier: ({ cacheType }) => (cacheType === 'write' ? 3.75 : 0.3),
|
||
};
|
||
|
||
it('prices a standard call at base rates', () => {
|
||
const cost = computeUsageCostUSD(
|
||
{ input_tokens: 1000, output_tokens: 500, model: 'gpt-4', provider: 'openAI' },
|
||
pricing,
|
||
);
|
||
expect(cost).toBeCloseTo((1000 * 3 + 500 * 15) / 1e6);
|
||
});
|
||
|
||
it('applies the premium tier when the call exceeds the input threshold', () => {
|
||
/** The exact gap finding A flagged: a long-context call bills premium */
|
||
const cost = computeUsageCostUSD(
|
||
{ input_tokens: 300000, output_tokens: 1000, model: 'gpt-5.5', provider: 'openAI' },
|
||
pricing,
|
||
);
|
||
expect(cost).toBeCloseTo((300000 * 8 + 1000 * 40) / 1e6);
|
||
});
|
||
|
||
it('prices cache-inclusive Anthropic input by subtracting cache, then cache at its own rates', () => {
|
||
/** input_tokens is cache-inclusive (13000 = 1000 fresh + 2000 write + 10000
|
||
* read). The fresh 1000 is billed at the input rate, the cache at its own
|
||
* rates — never the cache portion at the input rate too (LibreChat#13795). */
|
||
const cost = computeUsageCostUSD(
|
||
{
|
||
input_tokens: 13000,
|
||
output_tokens: 500,
|
||
model: 'claude-haiku-4-5',
|
||
provider: 'anthropic',
|
||
input_token_details: { cache_creation: 2000, cache_read: 10000 },
|
||
},
|
||
pricing,
|
||
);
|
||
expect(cost).toBeCloseTo((1000 * 3 + 2000 * 3.75 + 10000 * 0.3 + 500 * 15) / 1e6);
|
||
});
|
||
|
||
it('routes nested cache_write_tokens to the write bucket, not the input rate', () => {
|
||
/** OpenAI GPT-5.6 reports cache writes as `cache_write_tokens`. Those 2000
|
||
* tokens must bill at the write rate (3.75), not fold into input (3);
|
||
* inputOnly = 13000 - 2000 write - 10000 read = 1000. */
|
||
const cost = computeUsageCostUSD(
|
||
{
|
||
input_tokens: 13000,
|
||
output_tokens: 500,
|
||
model: 'gpt-5.6',
|
||
provider: 'openAI',
|
||
input_token_details: { cache_read: 10000, cache_write_tokens: 2000 },
|
||
},
|
||
pricing,
|
||
);
|
||
expect(cost).toBeCloseTo((1000 * 3 + 2000 * 3.75 + 10000 * 0.3 + 500 * 15) / 1e6);
|
||
});
|
||
|
||
it('routes top-level cache_write_tokens to the write bucket alongside the premium input tier', () => {
|
||
/** Top-level `cache_write_tokens` (Chat/Responses flattened shape) is also
|
||
* recognized; inputOnly = 280000 - 5000 - 15000 = 260000, above the premium
|
||
* threshold so input/completion price at the premium tier (8 / 40). */
|
||
const cost = computeUsageCostUSD(
|
||
{
|
||
input_tokens: 280000,
|
||
output_tokens: 500,
|
||
model: 'gpt-5.6',
|
||
provider: 'openAI',
|
||
input_token_details: { cache_read: 15000 },
|
||
cache_write_tokens: 5000,
|
||
},
|
||
pricing,
|
||
);
|
||
expect(cost).toBeCloseTo((260000 * 8 + 5000 * 3.75 + 15000 * 0.3 + 500 * 40) / 1e6);
|
||
});
|
||
});
|
||
|
||
describe('aggregateEmittedUsage', () => {
|
||
it('returns null for no emitted events', () => {
|
||
expect(aggregateEmittedUsage([])).toBeNull();
|
||
});
|
||
|
||
it('normalizes each call into display units (input excludes cache) and sums cost', () => {
|
||
const events: TTokenUsageEvent[] = [
|
||
{
|
||
input_tokens: 100,
|
||
output_tokens: 20,
|
||
total_tokens: 120,
|
||
model: 'gpt-4o-mini',
|
||
provider: 'openAI',
|
||
cost: 0.001,
|
||
},
|
||
{
|
||
input_tokens: 150,
|
||
output_tokens: 10,
|
||
total_tokens: 160,
|
||
input_token_details: { cache_creation: 30, cache_read: 50 },
|
||
model: 'gpt-4o-mini',
|
||
provider: 'openAI',
|
||
cost: 0.002,
|
||
},
|
||
];
|
||
/** openAI is cache-subset: input excludes cache (150−30−50=70) */
|
||
expect(aggregateEmittedUsage(events)).toEqual({
|
||
input: 170,
|
||
output: 30,
|
||
cacheWrite: 30,
|
||
cacheRead: 50,
|
||
cost: 0.003,
|
||
});
|
||
});
|
||
|
||
it('omits cost when no event carried it (contextCost off)', () => {
|
||
const rollup = aggregateEmittedUsage([
|
||
{ input_tokens: 100, output_tokens: 20, total_tokens: 120, provider: 'openAI' },
|
||
]);
|
||
expect(rollup).toEqual({ input: 100, output: 20, cacheWrite: 0, cacheRead: 0 });
|
||
expect(rollup?.cost).toBeUndefined();
|
||
});
|
||
|
||
it('omits cost when any call lacked it (partial pricing failure)', () => {
|
||
/** One call priced, one missing cost (e.g. computeUsageCostUSD threw) →
|
||
* the sum would under-report, so coverage is incomplete and cost omitted. */
|
||
const rollup = aggregateEmittedUsage([
|
||
{ input_tokens: 100, output_tokens: 20, provider: 'openAI', cost: 0.01 },
|
||
{ input_tokens: 50, output_tokens: 10, provider: 'openAI' },
|
||
]);
|
||
expect(rollup?.cost).toBeUndefined();
|
||
expect(rollup?.input).toBe(150);
|
||
expect(rollup?.output).toBe(30);
|
||
});
|
||
|
||
it('normalizes mixed-provider calls per their own provider before summing', () => {
|
||
/** bedrock is additive (cache separate from input), openAI is subset */
|
||
const rollup = aggregateEmittedUsage([
|
||
{
|
||
input_tokens: 100,
|
||
output_tokens: 20,
|
||
provider: 'bedrock',
|
||
input_token_details: { cache_read: 40 },
|
||
cost: 0.01,
|
||
},
|
||
{
|
||
input_tokens: 90,
|
||
output_tokens: 5,
|
||
usage_type: 'subagent',
|
||
provider: 'openAI',
|
||
input_token_details: { cache_read: 30 },
|
||
cost: 0.02,
|
||
},
|
||
]);
|
||
/** bedrock input stays 100 (additive); openAI input 90−30=60 → 160 */
|
||
expect(rollup?.input).toBe(160);
|
||
expect(rollup?.output).toBe(25);
|
||
expect(rollup?.cacheRead).toBe(70);
|
||
expect(rollup?.cost).toBeCloseTo(0.03);
|
||
});
|
||
|
||
it('uses the magnitude fallback for provider-less cached events (matches live)', () => {
|
||
/** No provider: the client's normalizeUsageUnits uses a magnitude heuristic
|
||
* (cache ≤ input ⇒ input includes cache), so the rollup must too — billing
|
||
* splitUsage would treat it as additive and leave input at 1000, diverging
|
||
* from the live display after reload. */
|
||
const rollup = aggregateEmittedUsage([
|
||
{ input_tokens: 1000, output_tokens: 100, input_token_details: { cache_read: 400 } },
|
||
]);
|
||
expect(rollup).toEqual({ input: 600, output: 100, cacheWrite: 0, cacheRead: 400 });
|
||
});
|
||
});
|
||
|
||
describe('buildPersistedContextUsage', () => {
|
||
const baseSnapshot: TContextUsageEvent = {
|
||
runId: 'run-1',
|
||
breakdown: {
|
||
maxContextTokens: 8000,
|
||
instructionTokens: 100,
|
||
systemMessageTokens: 80,
|
||
dynamicInstructionTokens: 20,
|
||
toolSchemaTokens: 30,
|
||
summaryTokens: 0,
|
||
toolCount: 2,
|
||
messageCount: 3,
|
||
messageTokens: 500,
|
||
availableForMessages: 7000,
|
||
toolTokenCounts: { add: 15, noop: 0 },
|
||
},
|
||
contextBudget: 7800,
|
||
};
|
||
|
||
it('persists positive per-tool schema counts', () => {
|
||
const result = buildPersistedContextUsage(baseSnapshot);
|
||
expect(result.breakdown.toolTokenCounts).toEqual({ add: 15 });
|
||
expect(result.contextBudget).toBe(7800);
|
||
});
|
||
|
||
it('omits a schema-count record with no positive counts', () => {
|
||
const result = buildPersistedContextUsage({
|
||
...baseSnapshot,
|
||
breakdown: { ...baseSnapshot.breakdown, toolTokenCounts: { add: 0 } },
|
||
});
|
||
expect(result.breakdown.toolTokenCounts).toBeUndefined();
|
||
});
|
||
|
||
it('passes through a snapshot without tool counts', () => {
|
||
const { toolTokenCounts: _omit, ...breakdown } = baseSnapshot.breakdown;
|
||
const result = buildPersistedContextUsage({ ...baseSnapshot, breakdown });
|
||
expect(result.breakdown.toolTokenCounts).toBeUndefined();
|
||
expect(result.breakdown.messageTokens).toBe(500);
|
||
});
|
||
|
||
it('passes a non-zero toolMessageTokens split through to the blob', () => {
|
||
const result = buildPersistedContextUsage({
|
||
...baseSnapshot,
|
||
breakdown: { ...baseSnapshot.breakdown, toolMessageTokens: 220 },
|
||
});
|
||
expect(result.breakdown.toolMessageTokens).toBe(220);
|
||
expect(result.breakdown.messageTokens).toBe(500);
|
||
});
|
||
|
||
it('keeps invocation-inclusive totals separate from per-tool result shares', () => {
|
||
const result = buildPersistedContextUsage({
|
||
...baseSnapshot,
|
||
breakdown: {
|
||
...baseSnapshot.breakdown,
|
||
toolMessageTokens: 10,
|
||
toolMessageTokenCounts: { search: 4 },
|
||
},
|
||
});
|
||
expect(result.breakdown.toolMessageTokens).toBe(10);
|
||
expect(result.breakdown.toolMessageTokenCounts).toEqual({ search: 4 });
|
||
expect(
|
||
Object.values(result.breakdown.toolMessageTokenCounts ?? {}).reduce(
|
||
(total, count) => total + count,
|
||
0,
|
||
),
|
||
).toBeLessThan(result.breakdown.toolMessageTokens ?? 0);
|
||
});
|
||
|
||
it('trims zero-valued result-message entries and clamps their sum', () => {
|
||
const result = buildPersistedContextUsage({
|
||
...baseSnapshot,
|
||
breakdown: {
|
||
...baseSnapshot.breakdown,
|
||
toolMessageTokens: 2,
|
||
toolMessageTokenCounts: { grep: 180, read_file: 0 },
|
||
},
|
||
});
|
||
expect(result.breakdown.toolMessageTokenCounts).toEqual({ grep: 2 });
|
||
expect(
|
||
Object.values(result.breakdown.toolMessageTokenCounts ?? {}).reduce(
|
||
(total, count) => total + count,
|
||
0,
|
||
),
|
||
).toBeLessThanOrEqual(result.breakdown.toolMessageTokens ?? 0);
|
||
});
|
||
|
||
it('preserves a known-zero tool-message total', () => {
|
||
const result = buildPersistedContextUsage({
|
||
...baseSnapshot,
|
||
breakdown: { ...baseSnapshot.breakdown, toolMessageTokens: 0 },
|
||
});
|
||
expect(result.breakdown.toolMessageTokens).toBe(0);
|
||
expect(Object.prototype.hasOwnProperty.call(result.breakdown, 'toolMessageTokens')).toBe(true);
|
||
});
|
||
it('safely persists prototype-sensitive names and ignores malformed counts', () => {
|
||
const counts = JSON.parse('{"__proto__":3,"constructor":3,"invalid":"4"}') as Record<
|
||
string,
|
||
number
|
||
>;
|
||
const sdkExtension = {
|
||
nested: { source: 'retained-sdk-field', flags: ['opaque'] },
|
||
invocation: { estimated: 4, providerOnly: true },
|
||
};
|
||
const snapshot = {
|
||
...baseSnapshot,
|
||
breakdown: {
|
||
...baseSnapshot.breakdown,
|
||
toolMessageTokens: 5,
|
||
toolMessageTokenCounts: counts,
|
||
sdkExtension,
|
||
},
|
||
};
|
||
const result = buildPersistedContextUsage(snapshot);
|
||
const persistedCounts = result.breakdown.toolMessageTokenCounts;
|
||
expect(persistedCounts?.['__proto__']).toBe(3);
|
||
expect(persistedCounts?.constructor).toBe(2);
|
||
expect(persistedCounts?.invalid).toBeUndefined();
|
||
expect(Object.keys(persistedCounts ?? {})).toEqual(['__proto__', 'constructor']);
|
||
expect(result.breakdown).toMatchObject({ sdkExtension });
|
||
expect(snapshot.breakdown.toolMessageTokenCounts).toBe(counts);
|
||
});
|
||
|
||
it('records the final primary call output as completedOutputTokens', () => {
|
||
/** The latest snapshot precedes the final call, so its post-snapshot delta
|
||
* is that call's output — not the full multi-call response tokenCount. */
|
||
const events: TTokenUsageEvent[] = [
|
||
{ input_tokens: 100, output_tokens: 40, total_tokens: 140, provider: 'openAI' },
|
||
{ input_tokens: 200, output_tokens: 25, total_tokens: 225, provider: 'openAI' },
|
||
{
|
||
input_tokens: 50,
|
||
output_tokens: 12,
|
||
usage_type: 'subagent',
|
||
provider: 'openAI',
|
||
},
|
||
];
|
||
const result = buildPersistedContextUsage(baseSnapshot, events);
|
||
/** Last PRIMARY call's completion (25), skipping the trailing subagent event */
|
||
expect(result.completedOutputTokens).toBe(25);
|
||
});
|
||
|
||
it.each(['openAI', 'bedrock'])('persists the final primary cache split for %s', (provider) => {
|
||
const events: TTokenUsageEvent[] = [
|
||
{ runId: 'run-1', input_tokens: 100, input_token_details: { cache_read: 10 } },
|
||
{
|
||
runId: 'run-1',
|
||
provider,
|
||
model: 'primary-model',
|
||
input_tokens: 200,
|
||
output_tokens: 25,
|
||
input_token_details: { cache_read: 80, cache_creation: 40 },
|
||
},
|
||
{ runId: 'run-2', input_tokens: 900, input_token_details: { cache_read: 900 } },
|
||
{ usage_type: 'summarization', input_tokens: 500, input_token_details: { cache_read: 500 } },
|
||
{ usage_type: 'subagent', input_tokens: 600, input_token_details: { cache_read: 600 } },
|
||
];
|
||
const result = buildPersistedContextUsage(baseSnapshot, events);
|
||
expect(result).toMatchObject({
|
||
cacheRead: 80,
|
||
cacheWrite: 40,
|
||
model: 'primary-model',
|
||
provider,
|
||
completedOutputTokens: 25,
|
||
});
|
||
});
|
||
|
||
it('replaces an earlier cache split with a final uncached call', () => {
|
||
const result = buildPersistedContextUsage({ ...baseSnapshot, cacheRead: 80, cacheWrite: 40 }, [
|
||
{ input_tokens: 200, output_tokens: 25 },
|
||
]);
|
||
expect(result).toMatchObject({ cacheRead: 0, cacheWrite: 0 });
|
||
expect(buildPersistedContextUsage(baseSnapshot).cacheRead).toBeUndefined();
|
||
expect(buildPersistedContextUsage(baseSnapshot).model).toBeUndefined();
|
||
});
|
||
|
||
it('omits completedOutputTokens when there are no primary calls', () => {
|
||
expect(buildPersistedContextUsage(baseSnapshot, []).completedOutputTokens).toBeUndefined();
|
||
});
|
||
|
||
it('reconciles the inflated estimate to the final call’s real prompt tokens', () => {
|
||
/** Real web-search + summarization turn: calibration pinned at 5 inflated
|
||
* messageTokens to 187471 (used 213375), but the answer call's true prompt was
|
||
* 55773 (Anthropic input_tokens is cache-inclusive; 2071 of it is cache read).
|
||
* The persisted blob must show the real context so a reload isn't stuck
|
||
* several× too high. */
|
||
const inflated: TContextUsageEvent = {
|
||
runId: 'run-1',
|
||
breakdown: {
|
||
maxContextTokens: 250000,
|
||
instructionTokens: 4205,
|
||
systemMessageTokens: 384,
|
||
dynamicInstructionTokens: 1525,
|
||
toolSchemaTokens: 2296,
|
||
summaryTokens: 1938,
|
||
toolCount: 1,
|
||
messageCount: 2,
|
||
messageTokens: 187471,
|
||
availableForMessages: 233295,
|
||
},
|
||
contextBudget: 237500,
|
||
remainingContextTokens: 24125,
|
||
calibrationRatio: 5,
|
||
};
|
||
const events: TTokenUsageEvent[] = [
|
||
{
|
||
input_tokens: 55773, // cache-inclusive: 53702 fresh + 2071 read
|
||
output_tokens: 3780,
|
||
input_token_details: { cache_read: 2071, cache_creation: 0 },
|
||
provider: 'anthropic',
|
||
runId: 'run-1',
|
||
},
|
||
];
|
||
const result = buildPersistedContextUsage(inflated, events);
|
||
expect(237500 - (result.remainingContextTokens ?? 0)).toBe(55773);
|
||
expect(result.breakdown.messageTokens).toBe(55773 - 4205 - 1938);
|
||
expect(result.completedOutputTokens).toBe(3780);
|
||
});
|
||
|
||
it('attributes completedOutputTokens to the snapshot run, not a parallel run', () => {
|
||
/** Parallel/direct runs interleave: this snapshot is run-1, but run-2 emits a
|
||
* later primary usage. The persisted delta must be run-1's own output (40),
|
||
* never run-2's trailing output (99). */
|
||
const events: TTokenUsageEvent[] = [
|
||
{
|
||
input_tokens: 100,
|
||
output_tokens: 40,
|
||
total_tokens: 140,
|
||
provider: 'openAI',
|
||
runId: 'run-1',
|
||
},
|
||
{
|
||
input_tokens: 200,
|
||
output_tokens: 99,
|
||
total_tokens: 299,
|
||
provider: 'openAI',
|
||
runId: 'run-2',
|
||
},
|
||
];
|
||
const result = buildPersistedContextUsage(baseSnapshot, events);
|
||
expect(result.completedOutputTokens).toBe(40);
|
||
});
|
||
|
||
it('omits completedOutputTokens when only other runs emitted usage', () => {
|
||
/** The snapshot run never completed (no matching primary); a sibling run's
|
||
* output must not be borrowed — fall back to the per-message estimate. */
|
||
const events: TTokenUsageEvent[] = [
|
||
{
|
||
input_tokens: 200,
|
||
output_tokens: 99,
|
||
total_tokens: 299,
|
||
provider: 'openAI',
|
||
runId: 'run-2',
|
||
},
|
||
];
|
||
expect(buildPersistedContextUsage(baseSnapshot, events).completedOutputTokens).toBeUndefined();
|
||
});
|
||
|
||
it('matches untagged usage events for back-compat (older lib / resume)', () => {
|
||
/** Events without a runId predate run tagging; they match any snapshot so the
|
||
* last primary (25) is still recorded. */
|
||
const events: TTokenUsageEvent[] = [
|
||
{ input_tokens: 100, output_tokens: 40, total_tokens: 140, provider: 'openAI' },
|
||
{ input_tokens: 200, output_tokens: 25, total_tokens: 225, provider: 'openAI' },
|
||
];
|
||
expect(buildPersistedContextUsage(baseSnapshot, events).completedOutputTokens).toBe(25);
|
||
});
|
||
});
|
||
|
||
describe('computeSummaryUsedTokens', () => {
|
||
const summarized = (over?: Partial<TContextUsageEvent>): TContextUsageEvent => ({
|
||
runId: 'run-1',
|
||
breakdown: {
|
||
maxContextTokens: 1000,
|
||
instructionTokens: 50,
|
||
systemMessageTokens: 50,
|
||
dynamicInstructionTokens: 0,
|
||
toolSchemaTokens: 0,
|
||
summaryTokens: 80,
|
||
toolCount: 0,
|
||
messageCount: 1,
|
||
messageTokens: 20,
|
||
availableForMessages: 900,
|
||
},
|
||
contextBudget: 1000,
|
||
remainingContextTokens: 700,
|
||
effectiveInstructionTokens: 50,
|
||
...over,
|
||
});
|
||
|
||
it('uses contextBudget − remainingContextTokens when available', () => {
|
||
expect(computeSummaryUsedTokens(summarized())).toBe(300);
|
||
});
|
||
|
||
it('falls back to instructions + summary + messages, including summaryTokens', () => {
|
||
/** summaryTokens is a separate breakdown field, so the no-remaining fallback
|
||
* must add it: 50 + 80 + 20 = 150, not 70. */
|
||
expect(computeSummaryUsedTokens(summarized({ remainingContextTokens: undefined }))).toBe(150);
|
||
});
|
||
|
||
it('returns undefined when the turn did not summarize', () => {
|
||
expect(
|
||
computeSummaryUsedTokens(
|
||
summarized({ breakdown: { ...summarized().breakdown, summaryTokens: 0 } }),
|
||
),
|
||
).toBeUndefined();
|
||
expect(computeSummaryUsedTokens(null)).toBeUndefined();
|
||
expect(computeSummaryUsedTokens(undefined)).toBeUndefined();
|
||
});
|
||
|
||
it('subtracts the response’s earlier tool-loop outputs from the marker', () => {
|
||
/** 300 baseUsed − 90 earlier outputs = 210, so the client estimate
|
||
* (summaryBaseline + full response tokenCount) doesn’t double-count them. */
|
||
expect(computeSummaryUsedTokens(summarized(), 90)).toBe(210);
|
||
});
|
||
|
||
it('clamps to undefined when the prior outputs exceed the baseline', () => {
|
||
expect(computeSummaryUsedTokens(summarized(), 5000)).toBeUndefined();
|
||
});
|
||
});
|
||
|
||
describe('priorRunOutputTokens', () => {
|
||
const ev = (over: Partial<TTokenUsageEvent>): TTokenUsageEvent => ({
|
||
input_tokens: 10,
|
||
output_tokens: 0,
|
||
total_tokens: 10,
|
||
provider: 'openAI',
|
||
...over,
|
||
});
|
||
|
||
it('sums primary outputs before the index for the matching run', () => {
|
||
const events = [
|
||
ev({ output_tokens: 20, runId: 'run-1' }),
|
||
ev({ output_tokens: 30, runId: 'run-1' }),
|
||
ev({ output_tokens: 99, runId: 'run-1' }), // at/after the index — excluded
|
||
];
|
||
expect(priorRunOutputTokens(events, 2, 'run-1')).toBe(50);
|
||
});
|
||
|
||
it('counts run-matched primary + summarization, skips subagent/sequential and other runs', () => {
|
||
/** Both the primary and the summarization output are in this run's tokenCount
|
||
* AND baseline, so both are subtracted; subagent/sequential are excluded from
|
||
* the reported output total; a parallel run's primary is not this snapshot's. */
|
||
const events = [
|
||
ev({ output_tokens: 20, runId: 'run-1' }), // primary, matches
|
||
ev({ output_tokens: 8, runId: 'run-1', usage_type: 'summarization' }), // counted
|
||
ev({ output_tokens: 5, runId: 'run-1', usage_type: 'subagent' }), // skipped
|
||
ev({ output_tokens: 7, runId: 'run-1', usage_type: 'sequential' }), // skipped
|
||
ev({ output_tokens: 40, runId: 'run-2' }), // other-run primary — skipped
|
||
];
|
||
expect(priorRunOutputTokens(events, 5, 'run-1')).toBe(28);
|
||
});
|
||
|
||
it('does not subtract a parallel sibling run’s summarization output', () => {
|
||
/** The summarize detour inherits the graph run id (traceConfig), so a sibling
|
||
* run's summary carries a DIFFERENT runId; its summary is in the sibling's
|
||
* baseline, not this snapshot's, so subtracting it would under-report. */
|
||
const events = [
|
||
ev({ output_tokens: 20, runId: 'run-1' }),
|
||
ev({ output_tokens: 8, runId: 'run-2', usage_type: 'summarization' }), // sibling — skipped
|
||
];
|
||
expect(priorRunOutputTokens(events, 2, 'run-1')).toBe(20);
|
||
});
|
||
|
||
it('matches untagged events for back-compat and returns 0 with no prior calls', () => {
|
||
const events = [ev({ output_tokens: 20 }), ev({ output_tokens: 30 })];
|
||
expect(priorRunOutputTokens(events, 2, 'run-1')).toBe(50);
|
||
expect(priorRunOutputTokens(events, 0, 'run-1')).toBe(0);
|
||
});
|
||
});
|
||
|
||
describe('buildAbortedResponseMetadata', () => {
|
||
it('returns undefined for an empty job', () => {
|
||
expect(buildAbortedResponseMetadata(undefined)).toBeUndefined();
|
||
expect(buildAbortedResponseMetadata({})).toBeUndefined();
|
||
expect(buildAbortedResponseMetadata({ tokenUsage: 'not json' })).toBeUndefined();
|
||
});
|
||
|
||
it('rebuilds the usage/cost rollup from the job’s persisted emitted usage', () => {
|
||
const events: TTokenUsageEvent[] = [
|
||
{ input_tokens: 100, output_tokens: 20, total_tokens: 120, provider: 'openAI', cost: 0.001 },
|
||
];
|
||
const result = buildAbortedResponseMetadata({ tokenUsage: JSON.stringify(events) });
|
||
expect(result?.usage).toEqual({
|
||
input: 100,
|
||
output: 20,
|
||
cacheWrite: 0,
|
||
cacheRead: 0,
|
||
cost: 0.001,
|
||
});
|
||
});
|
||
|
||
it('never persists a breakdown for a stopped response (avoids final-call over-count)', () => {
|
||
const events: TTokenUsageEvent[] = [
|
||
{ input_tokens: 100, output_tokens: 20, total_tokens: 120, provider: 'openAI' },
|
||
];
|
||
const result = buildAbortedResponseMetadata({ tokenUsage: JSON.stringify(events) });
|
||
expect(result).toEqual({ usage: { input: 100, output: 20, cacheWrite: 0, cacheRead: 0 } });
|
||
expect((result as { contextUsage?: unknown }).contextUsage).toBeUndefined();
|
||
});
|
||
|
||
const abortSnapshot: TContextUsageEvent = {
|
||
runId: 'run-1',
|
||
breakdown: {
|
||
maxContextTokens: 1000,
|
||
instructionTokens: 50,
|
||
systemMessageTokens: 50,
|
||
dynamicInstructionTokens: 0,
|
||
toolSchemaTokens: 0,
|
||
summaryTokens: 80,
|
||
toolCount: 0,
|
||
messageCount: 1,
|
||
messageTokens: 20,
|
||
availableForMessages: 900,
|
||
},
|
||
contextBudget: 1000,
|
||
remainingContextTokens: 700,
|
||
};
|
||
|
||
it('persists the summary marker (but not the full snapshot) for a stopped summarized turn', () => {
|
||
/** A single-call stopped turn: the interrupted call emitted no usage, so the
|
||
* marker is the full baseUsed (300) with nothing to subtract. */
|
||
const result = buildAbortedResponseMetadata({
|
||
tokenUsage: JSON.stringify([]),
|
||
contextUsage: JSON.stringify(abortSnapshot),
|
||
});
|
||
expect(result?.summaryUsedTokens).toBe(300);
|
||
/** The full snapshot stays off the abort path (completedOutputTokens ambiguity). */
|
||
expect((result as { contextUsage?: unknown }).contextUsage).toBeUndefined();
|
||
});
|
||
|
||
it('does not subtract any output from the marker on a stopped turn', () => {
|
||
/** The abort tokenCount comes from countTokens(text) or is absent — it does
|
||
* NOT fold in summarization/earlier-call output the way recordCollectedUsage
|
||
* does. So the marker is the full baseUsed (300); subtracting the summarization
|
||
* (8) or the primary (20) here would under-report after reload. */
|
||
const events: TTokenUsageEvent[] = [
|
||
{
|
||
input_tokens: 100,
|
||
output_tokens: 20,
|
||
total_tokens: 120,
|
||
provider: 'openAI',
|
||
runId: 'run-1',
|
||
},
|
||
{
|
||
input_tokens: 60,
|
||
output_tokens: 8,
|
||
total_tokens: 68,
|
||
provider: 'openAI',
|
||
runId: 'run-1',
|
||
usage_type: 'summarization',
|
||
},
|
||
];
|
||
const result = buildAbortedResponseMetadata({
|
||
tokenUsage: JSON.stringify(events),
|
||
contextUsage: JSON.stringify(abortSnapshot),
|
||
});
|
||
expect(result?.summaryUsedTokens).toBe(300);
|
||
});
|
||
});
|
||
|
||
describe('resolveAgentTokenConfig', () => {
|
||
const primary = { 'gpt-4': { prompt: 0.01, completion: 0.03, context: 8192 } };
|
||
const subagent = { 'gpt-4': { prompt: 0.05, completion: 0.15, context: 8192 } };
|
||
|
||
it('returns the producing agent’s own config', () => {
|
||
const byAgentId = new Map([
|
||
['primary', primary],
|
||
['sub', subagent],
|
||
]);
|
||
expect(resolveAgentTokenConfig({ agentId: 'sub', byAgentId, fallback: primary })).toBe(
|
||
subagent,
|
||
);
|
||
});
|
||
|
||
it('observes a selected lazy subagent added after the resolver is created', () => {
|
||
const byAgentId = new Map([['primary', primary]]);
|
||
const resolveForUsage = (agentId: string) =>
|
||
resolveAgentTokenConfig({ agentId, byAgentId, fallback: primary });
|
||
|
||
byAgentId.set('lazy-subagent', subagent);
|
||
|
||
expect(resolveForUsage('lazy-subagent')).toBe(subagent);
|
||
});
|
||
|
||
it('returns undefined for a known agent with no configured rates (built-in pricing)', () => {
|
||
/** A known non-custom agent (e.g. a normal OpenAI agent) is recorded with an
|
||
* undefined config; it must NOT inherit the custom-primary rates. */
|
||
const byAgentId = new Map<string, typeof primary | undefined>([
|
||
['primary', primary],
|
||
['known-openai', undefined],
|
||
]);
|
||
expect(
|
||
resolveAgentTokenConfig({ agentId: 'known-openai', byAgentId, fallback: primary }),
|
||
).toBeUndefined();
|
||
});
|
||
|
||
it('falls back to the primary config for an untagged usage', () => {
|
||
const byAgentId = new Map([['primary', primary]]);
|
||
expect(resolveAgentTokenConfig({ agentId: undefined, byAgentId, fallback: primary })).toBe(
|
||
primary,
|
||
);
|
||
});
|
||
|
||
it('falls back to the primary config for an unknown agent id', () => {
|
||
const byAgentId = new Map([['primary', primary]]);
|
||
expect(resolveAgentTokenConfig({ agentId: 'ghost', byAgentId, fallback: primary })).toBe(
|
||
primary,
|
||
);
|
||
});
|
||
|
||
it('returns the fallback when there is no per-agent map (single-endpoint graphs)', () => {
|
||
expect(resolveAgentTokenConfig({ agentId: 'primary', fallback: primary })).toBe(primary);
|
||
});
|
||
});
|
||
|
||
describe('createDetachedSubagentUsageRecorder', () => {
|
||
it('snapshots per-agent pricing and records each call as subagent usage', async () => {
|
||
const spendTokens = jest.fn().mockResolvedValue(undefined);
|
||
const childConfig = { 'child-model': { prompt: 0.01, completion: 0.02, context: 4096 } };
|
||
const configs = new Map([['child-agent', childConfig]]);
|
||
const recorder = createDetachedSubagentUsageRecorder(
|
||
{
|
||
spendTokens,
|
||
spendStructuredTokens: jest.fn().mockResolvedValue(undefined),
|
||
},
|
||
{
|
||
user: 'user-1',
|
||
conversationId: 'parent-1',
|
||
messageId: 'response-1',
|
||
model: 'parent-model',
|
||
endpointTokenConfigByAgentId: configs,
|
||
},
|
||
);
|
||
configs.set('child-agent', {
|
||
'child-model': { prompt: 99, completion: 99, context: 4096 },
|
||
});
|
||
|
||
await recorder({
|
||
usage_type: 'subagent',
|
||
input_tokens: 12,
|
||
output_tokens: 4,
|
||
model: 'child-model',
|
||
agentId: 'child-agent',
|
||
});
|
||
|
||
expect(spendTokens).toHaveBeenCalledWith(
|
||
expect.objectContaining({
|
||
user: 'user-1',
|
||
conversationId: 'parent-1',
|
||
messageId: 'response-1',
|
||
context: 'subagent',
|
||
model: 'child-model',
|
||
endpointTokenConfig: childConfig,
|
||
}),
|
||
{ promptTokens: 12, completionTokens: 4 },
|
||
);
|
||
});
|
||
|
||
it('does not recreate billing records after the owning principal is fenced or deleted', async () => {
|
||
const spendTokens = jest.fn().mockResolvedValue(undefined);
|
||
const updateBalance = jest.fn().mockResolvedValue(undefined);
|
||
const insertMany = jest.fn().mockResolvedValue(undefined);
|
||
const recorder = createDetachedSubagentUsageRecorder(
|
||
{
|
||
spendTokens,
|
||
spendStructuredTokens: jest.fn().mockResolvedValue(undefined),
|
||
bulkWriteOps: { updateBalance, insertMany },
|
||
isPrincipalActive: jest.fn().mockResolvedValue(false),
|
||
},
|
||
{
|
||
user: 'deleted-user',
|
||
conversationId: 'parent-1',
|
||
messageId: 'response-1',
|
||
model: 'child-model',
|
||
},
|
||
);
|
||
|
||
await recorder({
|
||
usage_type: 'subagent',
|
||
input_tokens: 12,
|
||
output_tokens: 4,
|
||
model: 'child-model',
|
||
});
|
||
|
||
expect(spendTokens).not.toHaveBeenCalled();
|
||
expect(updateBalance).not.toHaveBeenCalled();
|
||
expect(insertMany).not.toHaveBeenCalled();
|
||
});
|
||
});
|