Commit graph

6 commits

Author SHA1 Message Date
Marco Beretta
e2f5772961
🧮 feat: Surface Retained Tool-Message Context Usage (#15796)
Some checks are pending
Docker Dev Images Build / publish (push) Waiting to run
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
* feat: surface retained tool-message context usage

* fix: correct post-snapshot context accounting and lockfile registry

Adjudicated the review findings on the retained tool-message context work:

- package-lock.json resolved four entries (@librechat/agents, call-bind-apply-helpers,
  es-object-atoms, get-intrinsic) from a private mirror, breaking `npm ci` outside
  that network. Restored the public registry hosts; the integrity hashes already
  match the public tarballs.
- `finalizeUsage` migrated every usage atom except `subagentUsageFamily` before
  dropping the temporary `new` key, so a first-turn subagent total vanished when
  the conversation id was assigned.
- The runway projection divided the pre-invoke `remainingContextTokens` by per-call
  growth, ignoring the output generated after that snapshot, and the compaction hint
  subtracted a post-call exchange from a pre-call message total. Both now apply the
  post-snapshot output, so each side is measured at the same point in time.
- The estimate path rendered its tool-call share as a peer row although it is a
  subset of Input/Output/Estimated, so the visible rows summed past `usedTokens`.
  It is now an indented subtotal; the counted rows stay provider-accurate.
- The Totals block presented conversation-wide session subagent usage beside
  branch-scoped rows; it is now labeled all-branches.
- Rewrote the truncated `rgb-series-8` rationale as one comment on the series block.

* fix: keep subset accounting whole across cache, legacy snapshots, and custom themes

Adjudicated the second review round on the retained tool-message context work.
One cause per finding, each measured on the same basis as what it is compared to:

- The all-branches subagent subtotal summed input+output only, so a cached
  subagent call was under-reported and a fully cached call with no output
  disappeared from the row while the Cache read/write rows above still counted
  its traffic. It now sums every normalized bucket.
- `collectAnchorSeries` read an absent `remainingContextTokens` as zero and
  recorded that snapshot as having spent its entire budget, which distorted the
  growth delta feeding the runway projection. A snapshot saved in the older
  shape now derives its used total from the breakdown, exactly as the render
  path's `baseUsed` does, and one that carries neither basis is skipped. Each
  reading is tagged with its basis and the projection compares only readings of
  the same basis instead of reporting the difference between two measurements.
- `resolveTheme` filled an omitted `rgb-series-8` from the bundled base, so a
  stored or environment theme that paints its own seven-slot scale got
  LibreChat's indigo on surfaces it never checked: 2.41:1 against that theme's
  meter track, under the 3:1 mark floor the scale claims. An owned scale now
  derives the slot from the theme's own neutral text — 11.5:1 on the same
  reference theme — and naming `rgb-series-8` still opts out.
- Ordered the `--series-*` declarations in the light block by slot.

* fix: keep cached shares out of the peer rows and track resolved text for slot 8

Third round on the retained tool-message context work, correcting the two
variants of invariants this PR already fixed elsewhere rather than patching
them locally:

- The snapshot path rendered the reconciling call's Cached and Cache write
  shares as peer rows with window percentages, although reconciliation already
  counted those tokens inside the segments above. A fully cached prompt showed
  its tokens twice and the visible rows summed past the meter. They now render
  as an indented subtotal, the same treatment the estimate path's tool-call
  share received. The existing test claimed this invariant in its name but only
  asserted the rows exist, so it now asserts they are not peers.
- The `rgb-series-8` fallback required the theme to name its own text, so a
  theme that repaints the scale and its surfaces while inheriting LibreChat's
  text colours still received the bundled indigo. It now derives from the
  RESOLVED secondary text, so the stop tracks whatever the theme reads its body
  copy against whether it names one or inherits ours.

* fix: settle the subagent share with its run and measure both reclaim sides alike

Fourth round on the retained tool-message context work:

- The subagent share was committed on arrival while the usage it is a subset of
  stayed pending, so a run that ended with no salvageable response left its
  tokens in the Totals after `resetLive` discarded them from every rollup, and a
  resume re-folded the same events into the cumulative figure. It now
  accumulates in `pendingSubagentUsageFamily` beside the pending usage and
  settles with it: committed by `flushPendingInto` when the response keeps its
  usage, dropped by `resetLive` otherwise, migrated with the conversation id.
  The view merges committed and in-flight shares, so the live row is unchanged.
- `latestExchangeTokens` subtracted a summarizing turn's whole response,
  including the summarization completion the backend folds into `tokenCount`
  while the snapshot holds it in `breakdown.summaryTokens` rather than
  `messageTokens`. The reclaim hint was understated by the summary and could
  reach zero after a long one; the summary block's size is now excluded from the
  tail's contribution when that turn compacted.
- `sumBranch` exposed the tail's char estimate and tool share even when the tail
  carried a `tokenCount`, so a resumed partial response stayed in the message
  total while its tool traffic was dropped from the Tool calls split. Both tail
  figures are now zero for a counted tail, which also stops the estimate path
  subtracting an estimate that was never summed.

* chore: drop the lockfile churn the base branch already carries

The SDK bump this PR set out to make landed on dev as 3.8.5, so the branch no
longer changes any package.json and its lockfile edits (the 3.8.4 entry plus the
transitive call-bind-apply-helpers/es-object-atoms/get-intrinsic bumps that
install pulled in) are churn against the base.

* fix: give the eighth series slot a value every bundled theme can carry

The base branch grew high-contrast palettes that must declare every registry
token, and a test that a series slot can carry `text-on-status` at the 3:1 mark
floor, because a slot also fills a badge chip under that glyph. Slot 8 failed
both: the high-contrast palettes could not name a token that did not exist when
they were written, and the dark indigo #8c98e6 carried white at 2.71:1.

Dark slot 8 is muted to #8082be, which reads 3.66:1 under the status label and
still clears 3.66:1 on every series surface. The high-contrast palettes take
their own neutral text as slot 8 — the same stop `resolveTheme` derives for a
third-party theme that paints its own scale — because seven saturated hues is
all either canvas affords at their stated CVD separation; it is distinguished
by lightness at 21:1 on the canvas instead of by hue.

* test: exercise the context split through the mock harness

Nine scenarios of the acceptance contract, one tagged Playwright test each,
driving the real application: a tool round-trip through the e2e MCP server for
the Messages/Tool calls split, its keyboard disclosure of per-tool counts, the
peer rows staying inside the meter and surviving a reload; a real subagent run
for the all-branches total; seeded conversations for the legacy snapshot whose
used figure comes from its breakdown and for the estimate path's nested tool
share; and a stored theme that owns the series scale for the eighth slot's
fallback and its opt-out.

* test: keep the tool and subagent scenarios on the viewports that expose them

* 📌 feat: Unified Pinned Section With Sidebar Drag Interactions (#15057)

* feat: unified pinned section with drag interactions and sidebar polish

- Pinned chats and favorite models/agents/specs render as one interleaved,
  reorderable list persisted via users.pinnedOrder
- Drag conversations onto project rows to file them, onto Chats to unfile,
  and onto the Pinned section to pin (empty section materializes as a drop
  zone during a conversation drag)
- Row-level unpin button on pinned chats matching the favorites' button,
  and the row action menu shares the same hover/focus treatment
- Agent Marketplace entry moved under the New Chat button in the side rail
- Bookmarks trigger sized to the search bar, search focus no longer swaps
  its border color, tag filtering shows a loading state, and opening a
  project shows row-shaped skeletons

* fix: add pinnedOrder to IUser type and sort imports

* fix: pinned drag threshold, unpin focus target, and marketplace link

* fix: surface failed pinned reorders and skip no-op order writes

* fix: address review findings on unified pinned section

Move the pinned-order endpoints into packages/api as TypeScript handlers and
route their URLs through the data-provider endpoint registry. Raise the entry
and key caps so a valid model favorite and an unbounded pin count can both be
persisted.

Keep the marketplace entry reachable from the mobile drawer, which is the only
sidebar surface on small screens. Merge a reorder into the stored order while a
bookmark filter hides part of the list, instead of dropping the hidden keys.
Add Alt+Arrow keyboard reordering with a live-region announcement, restore the
focus handoff when a favorite is removed, and hold the local arrangement until
the stored order catches up. Serialize overlapping order writes and discard the
optimistic value when the first one fails. Derive the tag-filter loading state
from isPreviousData rather than a flag that could stick on.

* fix: release drag while renaming and dedupe repeated favorite keys

A draggable ancestor swallows drag-select inside the rename input, so both the
row's own drag source and the pinned wrapper's release while a title is being
edited.

The favorites payload validator accepts a repeated entry, and two rows keyed
the same would collide in React, point every drag at the first copy, and fail
the order endpoint's uniqueness check. Build the pinned list keyed-unique.

* fix: address PR review bot findings

Make the model favorite key injective. Joining endpoint and model on '::'
collided for components containing the delimiter, so two distinct favorites
keyed the same and the dedupe hid one; the endpoint is now length-prefixed.

Replace the pinned-order entry cap with a total size guard. Pinning has no
membership cap and the sidebar query drains every cursor, so any count limit
rejects a legitimate list rather than bounding anything; what needs bounding is
the user document.

Let only the newest order write touch the cache. An earlier write failing
rolled back over a newer arrangement that was still in flight, leaving the UI
on a stale order while the server held the current one. A confirmed write now
takes the server's answer, and a failed newest write discards the value that
was never accepted.

Persist the favorites array only once the order write succeeds, so a rejected
write cannot leave the two stores disagreeing with the rolled-back display.

* fix: address second round of PR review bot findings

Read favorites membership at write time rather than capturing it at drag end.
A favorite added or removed while the order request was in flight was undone by
writing the older list back; only the relative order now comes from the drag.

Move the pinned-order queue to module scope. The sidebar unmounts whole
sections while a search is active, so a component-local ref let a remounted
hook start a second queue alongside a request that was still running.

Serialize project assignment per conversation. Every drop target owns its own
mutation instance and the write is an unconditional update, so two quick drops
let whichever request reached the database last decide the project.

Read the pinned-order request body null-safely: destructuring a nullish body
threw past the handler into Express's generic error path instead of returning
the intended 400.

* fix: address third round of PR review bot findings

Drop the secondary favorites write. No consumer of the favorites array reads
its order, so mirroring the arrangement there bought nothing while racing the
membership mutations that share that whole-array endpoint; pinnedOrder is the
single source of ordering.

Always merge the committed order into the stored one. The visible list is not
always the whole list, since a bookmark filter hides rows and the pinned query
publishes partial results while draining its cursor, and replacing wholesale
dropped every key it could not see.

Release the local arrangement when the write settles instead of when the stored
order matches it key for key. A pin arriving mid-request made that equality
unreachable, stranding the snapshot and hiding the new row until remount.

Abandon a queued pinned-order write if the signed-in user changed, so a
deferred request cannot write one account's order into another's document.

* fix: address fourth round of PR review bot findings

Guard the pinned-order completion callbacks on the signed-in account. The
pre-send check only covers a write that has not left yet; a response arriving
after a session change would publish one account's order into the next one's
cache, which this query never refetches on its own.

Commit a reorder only when the pinned rows handled the drop. Dragging a pin
onto a project row or into Chats sweeps the pointer across sibling rows, whose
hover handlers shift the order, so a filing action was also saving a move the
user never asked for.

* fix: address fifth round of PR review bot findings

Deselect pinnedOrder at schema level. It is display-only and allowed to grow
large, but every authentication request loads the user document, so it was
adding its whole payload to paths that never read it.

Refuse to reorder until the stored order has loaded. Merging against an absent
order and then cancelling its fetch posted only the visible keys and discarded
saved positions for good.

Prune keys only once the visible list is known to be the whole list. Merging
unconditionally grew the array until the size guard rejected every write, so
completeness now gates compaction rather than the filter flag alone.

Track every renaming row, not just the most recent. RenameForm has no blur
cancellation, so a second form can open while the first is still mounted and
the single key reconnected the first row's drag source under its open input.

Judge drop eligibility by the queued destination. A drag item carries the
project its row had when the drag began, so re-dropping a chat back where it
started while an assignment was queued read as a no-op.

Take mutation ownership from the user atom AuthContext fills on login rather
than the user query, which is still empty in that window, and never treat an
unknown owner as a match.

Restore the pinned section's 30vh cap. Matching the Projects section's 42vh put
84vh of non-shrinking content above Chats and starved it on short viewports.

* fix: address sixth round of PR review bot findings

Gate reordering and pruning on successful fetches rather than attempted ones.
A failed pinned-order GET still reports as fetched, and a failed later page of
the pinned drain still publishes partial data and stops fetching, so both gates
were opening on exactly the states they were meant to exclude.

Scope the post-unpin focus handoff to this section by its own marker. The whole
conversations pane is a labelled region too, and the project lists inside it
render the same row, where unpinning only clears the flag and the row stays
put, so focus was being moved off a row that never went away.

Clear the account tracker when the mutation hook unmounts. The sidebar drops
that section during a search, leaving a stale non-null id that also shadowed
the cache fallback, so a deferred write could pass its owner check after a
session change.

* fix: gate pruning on a delivered favorites list

A favorites fetch that exhausted its retries leaves isLoading false with the
list empty, so an otherwise complete pinned load enabled pruning and dropped
every favorite key from the stored order. useFavorites now exposes the query's
success state and the pruning gate requires it, matching the pinned-order and
pinned-membership sources.

* fix: address eighth round of PR review bot findings

Identify queued assignments by token rather than by destination. Comparing
destination values cannot tell two drops onto the same project apart, so a
B, A, B run let the first B's cleanup discard an entry the still-queued later
drops owned, and the next drop was then judged against a stale project.

Report the rename ending when a row unmounts. A pinned row removed mid-rename,
for instance by unpinning the same chat from the project list that also renders
it, left its key in the renaming set and came back permanently undraggable.

* fix: address ninth round of PR review bot findings

Stand the keyboard reorder down wherever the pointer drag does. Alt/Option plus
an arrow moves the caret word-wise inside a text field, so the shortcut was
moving and saving the whole row out from under an open rename input.

Hold a settled assignment destination until a drag item shows the new project.
The mutation's cache invalidations are not awaited, so a row still reported its
old project for a moment after the write succeeded and a quick drag back to
where it started read as a no-op.

* fix: address tenth round of PR review bot findings

Serialize project assignment inside the mutation instead of at one caller. The
drag targets, the row menu and the project dialog each hold their own instance,
so a queued drop could still overwrite a newer menu write; moving the queue
down covers every surface and lets the drag helper drop its own.

Tie the live snapshot's release to the arrangement that created it. An older
write settling mid-drag cleared the newer arrangement, snapping the rows back
to a stale order while the drag still held its own, so releasing saved an order
the user could no longer see.

* fix: address eleventh round of PR review bot findings

Move assignment destination tracking next to the queue that owns it. Keeping it
in the drag helper meant a menu or dialog move left the drag helper still
reporting its own older destination, with no row left to observe that would
ever clear it, so a later move back was rejected for good.

Resolve a conversation's project from the write the mutation confirmed rather
than from whichever list rendered the row. The same chat appears in the Pinned
section and its expanded project, and those caches refresh independently, so
observing one of them was never a sound way to decide the other had caught up.
The conversation cache is written synchronously on success, which makes the
lazy self-clearing and its settled flag unnecessary.

* fix: cancel an in-flight conversation refetch before confirming an assignment

Navigation invalidates the exact conversation query, so a refetch that read the
old project can be in flight when an assignment resolves and land afterwards,
reverting the move in the cache that now answers where a conversation lives.

* fix: keep queued pinned-order writes attributable across an unmount

Refuse a queued write only when a different account is positively established.
Clearing the identity on unmount stopped stale attributions but stranded work
that belonged to the same account, since the sidebar drops the section that
owns this hook whenever a search is active, and it denied that work its
rollback as well.

Correct the cache from the request rather than from onError. The observer
callbacks do not run once that section unmounts, so an order the server never
accepted could survive on a query that refetches on nothing.

* fix: publish the session identity from the user atom

Queued pinned-order writes were attributed by a tracker that only the pinned
section's hook updated, and the sidebar unmounts that section during a search.
A retained value went stale across a sign-in, and login clears the user query
before the next account's fetch resolves, so a write queued by the previous
account could pass its own check and post on the new account's credentials.

A Recoil atom effect publishes the identity instead: it runs wherever the atom
is set, including sign-out, so attribution comes from the state itself rather
than from a component that happens to be mounted. That makes a signed-out
session foreign to every owner and lets the check be a strict comparison again,
rather than a rule that had to guess when identity was merely unreadable.

* fix: discard reorders that ended on a refused external target

A drop the pinned rows did not handle usually means a reorder, but a project
row or the Chats section refusing one reports the same, so dragging a chat onto
the project it already belongs to was saving the incidental shift its pointer
caused on the way out.

The rows and those targets now report their hovers, so the last one under the
pointer settles it, which also allows for a drag that strays over a project and
returns to reorder after all.

* fix: attribute queued writes to the credentials they will travel with

Publishing the account from React state, whether an effect or a Recoil atom
effect, always lagged the credential change: setUserContext installs the new
Authorization header in one synchronous call, and a write queued by the
previous account and sent in that gap read its own owner back and went out as
the new one. A test against the atom effect showed onSet is deferred too, so
the atom was no better than the effect it replaced.

The header is the thing that decides who the server sees, so ownership is read
from it. The id claim is compared rather than the token, since a silent refresh
rotates the token for the same account and discarding that work would strand
something the user is still entitled to.

Leaves AuthContext and the user atom untouched.

* fix: attribute a reorder to the account that started it

onMutate awaits cancelQueries before the request is built, and the mutation
function was reading the session again afterwards, so credentials turning over
inside that gap sent one account's order as the next. The owner is now recorded
in onMutate's first synchronous statement and carried on the order itself, and
the optimistic write is skipped when the session moved on, since that cache is
shared and showing an order to the wrong account is the same mistake as sending
it.

Record a pinned hover before the guards rather than after. Hovering the dragged
row is a no-op for ordering but still means the pointer is inside the list, so
a drag that wandered onto a project and returned to its own position was being
discarded as a filing action.

* fix: address a further round of PR review bot findings

Mark a hover anywhere in the pinned section as internal. The header and the
padding around the rows are part of the list, so a drag that wandered onto a
project and came back to rest on one of them ran no row hover and was discarded
as a filing action.

Hold off pruning while a favorites write is unsettled. The earlier GET's
success stands through an optimistic removal, so pruning then dropped the
ordering key of a write that had not landed, and a failed one came back without
its saved position.

Declare the reorder shortcut on the elements that take focus. The section's
written hint is skipped by anyone tabbing straight to a row, leaving the only
non-pointer way to reorder undiscoverable.

* fix: reach a real New Chat control, and let the shortcut prop through the memo

The unpin fallback looked for a test id no component carries, so once the last
pinned row unmounted focus fell to the document rather than to New Chat. It now
tries the controls that exist, in sidebar order. The test that covered this had
fabricated the missing id, so it proved nothing.

Convo is memoized on conversation fields and isGenerating alone, so the
shortcut appearing once the pinned order loaded never reached the element that
announces it. The comparator now accounts for it.

* fix: identify OpenID sessions, and see favorites writes from every hook

With OpenID token reuse the bearer is the provider's own id_token, which names
its subject as sub under an issuer and carries no LibreChat id, so every such
session resolved to nobody and two different accounts compared equal. That is
the case the ownership guard exists for, so it read as safe while being exactly
wrong. Sessions are identified by whatever the credential offers now, and an
owner that cannot be identified is foreign to everything including itself.

Count favorites writes across the query client rather than from one observer.
A favorite row runs its own useFavorites, so a removal started there was
invisible to the pinned section and its ordering key could still be pruned.

* fix: close the assignment cache gap, and let a failed drain be retried

The assignment released its pending destination as soon as the request settled,
while the cache write waited on an awaited cancellation in onSuccess. A drag
started in that gap saw neither, fell back to the stale row and refused a valid
drop. The write now happens in the request, before the entry is released, which
also survives the owning component unmounting.

Also clear the pagination guard when the Chats section collapses. A drain that
exhausted its retries leaves the conversations array unchanged, so the guard
barred every later attempt and the remaining chats stayed unreachable for the
session. Reopening the section now tries once more. That guard came from #14860
rather than this branch, so it can be split out if preferred.

* fix: compose the marketplace link, and keep focus on a surviving row

The marketplace control was a raw link with restated appearance classes, and
in restating them it left out the focus ring entirely, so keyboard focus on it
was invisible. It composes through the shared button now, which carries the
ring, the hover fill and the theme timing.

Unpinning from a project list leaves the row in place but removes the pin badge
that had focus, and the pinned-section successor search deliberately returns
nothing there, so focus fell to the document. It moves to the row's own link.

Reconcile the pinned order on window focus and reconnect. It is one shared
per-user record, so a second tab would otherwise show a stale arrangement and
its next reorder would post a full array undoing the other tab's.

* fix: keep a rename from restoring a conversation's other fields

A rename request carries only a title, but its response is the whole
conversation as the server saw it, and writing all of that back restores the
pre-request value of every other field. An assignment confirmed while the
rename was in flight was therefore reverted in the cache that now answers which
project a conversation belongs to, and a drag back to the original project read
as a no-op.

Only the title and its timestamp are taken from the response. Cancelling
queries cannot help here, since this is another mutation's write rather than a
fetch.

* fix: compose the unpin badge through the shared row-action button

The badge restated a reusable icon control's sizing, radius, hover and focus
classes. The client package already had a row-action variant and an icon-xs
size that match it exactly, so it uses those instead of its own copy.

* fix: reconcile before reordering, and scope assignments to their session

A reorder stayed enabled while the order's focus refetch was in flight, so
returning to a tab could cancel that refetch and post the arrangement it was
about to replace. Pruning had the matching gap: the pinned list keeps its data
fresh for five minutes, so a tab whose order had reconciled could prune against
membership that had not, dropping the position of a conversation another tab
pinned. Both now wait for data that is settled and not stale.

Assignments carry the account that started them, like the order writes already
did. Conversation ids are per-user, so a queued one travelling with the next
account's credentials would act on whatever that id names over there, and its
answer would describe one account's conversation to another in a cache that is
not scoped by user.

The row's overflow trigger derives from the shared row-action variant too, so
the two actions in a row stop keeping separate copies of the same recipe.

* fix: reject foreign assignment results, and share the section header

An assignment settling after a session change returned normally, so the success
path still ran: the caller reported success and the response could be applied to
the next account's conversation, since ids repeat across accounts. It rejects
now, so nothing downstream treats it as a result.

The collapsible section header existed as three identical copies, Chats,
Projects and Pinned, and this branch added the third. It moves into the shared
button as a section-header variant and all three compose from it.

* fix: let favorites membership reconcile between tabs

The favorites query disabled focus, reconnect and mount refetches, so a second
tab kept its old list indefinitely. Its pruning gate waits for a fetch, and
none was ever going to happen, so a reorder there could delete the saved
position of a favorite the first tab had added. It reconciles on focus and
reconnect now, like the order it is arranged alongside.

* fix: hold reconciling refetches off while a write is in flight

Enabling focus and reconnect refetches on the order and favorites queries let
one start after onMutate's single cancellation, read the pre-write state and
land after the write, putting the old value back. Both now decline those
refetches while a write for that resource is outstanding, keeping the cross-tab
reconciliation itself.

The check asks the query client at the moment the event fires rather than
reading useIsMutating: that hook only reaches the query's options on the next
render, which is a race against the very event being gated. A test that
dispatches focus mid-write fails without it.

* fix: release abandoned assignments and gate pruning on current membership

An assignment rejected before entering the try block kept its pending entry
forever, so a later session read a destination that was never sent. The
ownership check now sits inside the block that releases it, and it runs again
after the conversation query cancellation resolves, since the session can turn
over while that await is pending.

Pruning the pinned order now requires membership at least as current as the
order it is pruned against. The pinned query holds its data for five minutes
while the order has no such window, so focusing a tab reconciled the order
alone and the next reorder dropped the key of a conversation another tab had
pinned in between.

The assignment spec resets its mocks between tests and gives each test its own
conversation id, so one test's unconsumed mock and leftover queue entries no
longer decide the next test's result.

* fix: address the sixth round of review findings on the pinned section

The section header recipe asked for the shared button variant without a size,
so CVA emitted the default `h-10 px-4` after it and the merge kept it, putting
a 40px control in a 32px header row. The variant opts out of default sizing
through a compound now, since a heading is sized by its own text.

Restore the search field's focus border. Removing it left keyboard focus
signalled only by a background change that hover already uses, so the field
had no distinguishable focus state at all.

An assignment can reach the database before a concurrent rename and still
answer after it, carrying a conversation that predates the rename. Only the
project field is merged into the cached conversation and published to the
active one now, matching what the rename handler already does.

An order refetch that starts mid drag no longer lets the drop post an array
built from the order that refetch is about to replace: the arrangement is held
and written once the fresh order lands. Reorder writes superseded by a newer
arrangement are skipped rather than sent, so holding the shortcut costs two
writes instead of one per intermediate position. The order and favorites
queries reconcile on mount as well, since a sidebar search unmounts the
section and a query with no observer never sees the focus event.

* fix: preserve pinned order until favorites membership catches up

* fix: reconcile pinned membership and preserve auth and focus state

* fix: preserve reconciled keys during active pinned arrangements

* style: sort pinned order regression imports

* fix: reconcile favorite rollbacks before pinned-order pruning

* 🧊 fix: Withhold Rerun Controls on Compaction Turns (#15797)

* 🧊 fix: Withhold Rerun Controls on Compaction Turns

A compaction turn's response parents onto the leaf it summarized, not onto a
user message, so the hover Edit's Rerun replayed an assistant message in the
user slot: the request minted a user message under the existing assistant
leaf's id and ran the turn with empty text, which fails. Editing offered
nothing either, since SUMMARY is not an editable part type.

Edit, Regenerate and Continue are now withheld on a compaction turn, and both
rerun paths refuse a parent that is not user-created, matching the guard
`useChatFunctions.regenerate` already applied.

* 🧭 fix: Gate Rerun Controls on the Replayed Parent Turn

Withholding Edit, Regenerate and Continue on summary-only content also hit an
ordinary turn that auto-summarized and was cancelled before its first answer
token: that response is parented to the user's message and is exactly the turn
a rerun is for. The row now resolves the message it would replay and withholds
the rerun shapes only when a model turn hangs off another model turn, which is
the manual compaction shape. An unresolvable parent withholds nothing.

`isCompactionTurn` is gone with it, and `isCompactedLeaf` keeps its original
body; its docstring now says per part what it always measured per part.

* 🧮 fix: Identify a Compaction Turn by Content and Parent

Gating the rerun controls on parent authorship alone withheld Edit, Regenerate
and Continue from any model turn hanging off another model turn. An imported
conversation chains messages sequentially, so a skipped empty human message
leaves an ordinary reply parented to the previous reply; its stored content is
editable and its controls must stay. It also made every model row resolve its
parent out of the flat message list: 100 full-thread scans while mounting a
200-message conversation.

A manual compaction is now both halves of its shape — summary-only content and
a parent that is not the user's turn — so only such a row consults the thread.
`isSummaryOnlyContent` is the content half, in data-provider next to
`isCompactedLeaf`.

* 🧷 test: Pin the Rerun Refusal on a Chained Model Turn

A rerun replays its parent as the user turn, so a model turn parented onto
another model turn must submit nothing rather than mint a user message under an
existing response's id. Removing the guard makes the case call `ask` once.

* 🧱 fix: Gate the Editor on Content and the Rerun on the Parent

Identifying a compaction turn by content shape missed the ones the server
persists with an error part (a skipped or failed compaction keeps its
explanation instead of a summary), which left Edit, Regenerate and Continue on
a turn where every rerun shape is inert.

The two controls answer two different questions now. A rerun replays the
message's parent as the turn's user message, so Regenerate and Continue go
whenever a model turn hangs off a model turn. The editor goes only when that
turn also has no part to type in: an imported chain's reply is saved directly
and stays editable, while a compaction — summary or error part — would open an
empty editor with one inert Rerun.

`findMessageById` indexes the messages array once per identity, so resolving
the parent no longer scans the thread per row: 100 full-array walks while
mounting a 200-message conversation become one.

* 🏷️ fix: Honor the Server's Compaction Marker on Any Leaf

Compact runs on whatever leaf the branch ends with, so a compaction can hang off
a user message rather than off the answer it summarized. Inferring the turn from
its position alone left the rerun controls on that one: replaying the parent
would answer the user's earlier message again instead of redoing the compaction.

`isUserInitiatedCompaction` reads the marker the server already stamps —
`markCompactionSummary` sets `initiatedBy: 'user'`, and nothing else writes it,
so an automatic summary detour keeps its controls. A marked turn is excluded
from the rerun shapes wherever it hangs.

* 🎨 fix: Ask the Editor What It Can Edit

The hover row had its own idea of an editable part and counted artifact-bearing
text as one, while `EditContentParts` keeps an artifact in its read-only
renderer. A chained model turn whose only content is an artifact therefore still
offered Edit, and opening it gave a read-only artifact, a disabled Save and a
Rerun the guards refuse.

`editableParts.ts` now owns that rule for both: `isEditablePart` is the
editor's own filter, `hasEditablePart` asks it per message. The row resolves it
lazily — the artifact check parses markdown, and only a turn with nothing to
replay needs the answer — so a streaming row never pays for it.

* 🚫 fix: Withhold the Editor's Rerun Instead of Refusing It

Both editors resolved the parent inside the handler and returned when it was not
a user turn, so a chained model turn — an imported thread with no user message
between two replies — kept an enabled Rerun that did nothing, with Ctrl/Cmd+Enter
bound to it.

Each editor resolves that parent at render and withholds the action instead: the
button is gone, the shortcut is inert and `aria-keyshortcuts` no longer advertises
it, while Save keeps writing the edit. An unresolvable parent keeps the action,
matching the hover row.

* 🧹 style: Sort the New Imports in Both Editors

CI's import-sort check failed on the two files the previous commit touched.

* 💬 fix: Report Unsaved Changes When No Rerun Is Offered

A save-only editor still warned about the rerun. An answer's draft is discarded
by the rerun, and multi-part edits must be saved before one, but neither is true
where the action is withheld — a model turn chained onto another model turn — so
the status slot claimed a precondition for something the footer does not offer.

Both statuses are gated on the same `canRerun` the button is: the plain-text
editor reports unsaved changes, and `rerunRequiresSave` cannot hold without a
rerun to require it.

* 🧪 test: Exercise the Compaction Controls in the Browser

Eight mock-harness scenarios drive the real row and the real editors: a
finished, failed, error-part and user-parented compaction withhold Edit,
Regenerate and Continue while Copy stays; a turn that only auto-summarized
keeps all three; a reply chained onto a reply keeps a save-only editor whose
status reports unsaved changes; an artifact-only chained reply loses the
editor; and the context indicator still offers Compact after a failed
compaction.

`SeedMessage` carries `content`, `finish_reason` and `model` so a branch can be
seeded in the shapes the mock model cannot produce — a persisted summary, an
error part, an artifact.

* 🩻 feat: improve agent filter controls for insights (#15807)

* feat(insights): improve agent filter controls

* feat(insights): add search and fixed agent selection controls

* fix(insights): show search before all agents control

* test(e2e): allow steer consumption before arm response

* 🔠 fix: Name Artifact Downloads After Their Titles (#15836)

* fix: Use artifact titles for download filenames

* fix: Match artifact filenames to downloaded content

* fix: Preserve raw artifact extensions and GFM headings

* fix: Download complete artifact files and retain filename provenance

* fix: Distinguish cached artifact exports from originals

* fix: Name unnamed artifact originals by their media type

* fix: Recognize legacy CSV MIME in artifact filenames

* fix: preserve context insights across reloads and configuration changes

* fix: retain primary call details when resuming context snapshots

* fix: preserve resumed output and retained tool-result estimates

* fix: keep resumed output compatible with older clients

---------

Co-authored-by: Ravi Kumar L <upman@users.noreply.github.com>
Co-authored-by: Danny Avila <danny@librechat.ai>
2026-09-11 15:53:29 -04:00
Danny Avila
d18d62e7c1
🪙 refactor: Reconcile Context Gauge to Actual Provider Tokens (#13780)
* 🪙 fix: Reconcile Context Gauge to Actual Provider Tokens

The context gauge could read several× too high (e.g. 213K when the real prompt
was 56K) and stay there across reloads. Root cause: the SDK's calibrationRatio is
`cumulativeProviderReported / cumulativeRawSent`, but a provider's server-side
web search injects large fetched content into the prompt that the SDK never sent
or counted — pinning the ratio at its cap (5) and multiplying every later message
estimate, including post-summary ones. The gauge rendered (and persisted) that
inflated estimate, never the provider's actual token count.

Fix: reconcile the snapshot to the call's ACTUAL prompt tokens (input + cache),
which already arrive in on_token_usage. Only messageTokens is calibration-scaled
(instructions/summary are raw tiktoken), so keep those and set messageTokens to
the remainder, recomputing free space. Shared `promptTokensFromUsage` +
`reconcileContextUsage` in data-provider; applied server-side in
buildPersistedContextUsage (reload-stable) and client-side in useUsageHandler on
each primary usage (corrects at turn-end, no follow-up needed). Also drop the
summary double-count from the Breakdown Messages row.

Deferred (separate agents PR): the SDK over-calibration also fires summarization
prematurely; fixing it needs decoupling real-content estimation from server-side
injection headroom without weakening pruning-overflow safety.

* 🪙 fix: Harden Token Reconciliation for Provider-less + Resume Paths

Codex review on the reconciliation:
- promptTokensFromUsage: when the provider is absent (custom/OpenAI-compatible
  payloads), fall back to the same magnitude heuristic normalizeUsageUnits uses
  (cache ≤ input ⇒ already included) so cached events aren't re-inflated.
- Resume: backfillUsage restores a primary call's usage without replaying a live
  on_token_usage (Redis mode), so the live reconcile never ran and a reconnected
  session stayed on the inflated estimate. New reconcileBackfill reconciles the
  restored snapshot from the final primary call after contextHandler installs it.

* 🪙 fix: Reconcile Resume Snapshot Server-Side, Not via Backfill

Codex: the client reconcileBackfill scanned the resumed run's collectedUsage and
applied the final primary to the latest snapshot — but on a mid-call resume that
usage belongs to an EARLIER call, corrupting the restored gauge.

Move the resume reconciliation server-side: GenerationJobManager.persistTokenUsage
reconciles the stored contextUsage to a primary usage's actual prompt tokens as it
arrives. That usage is the post-invoke truth for the call the latest stored
snapshot precedes (no snapshot is captured between a call's pre-invoke dispatch
and its usage), so it's correct by construction and run-matched. A mid-call resume
(no usage yet) keeps the raw snapshot instead of mis-applying an earlier call's
tokens; it reconciles once the call completes. Removed client reconcileBackfill;
the live-path reconcile (non-resume) stays.

* 🪙 fix: Guard Reconciliation Against Replays and Snapshot Races

Two Codex concurrency findings on the reconciliation:
- Client: reconcile only on a NEWLY folded primary usage. A replayed duplicate
  (folded=false on resume) can be an earlier tool-loop call sharing the run id,
  which would overwrite the latest snapshot with an earlier, smaller prompt. Moved
  the reconcile after the folded guard.
- Server: serialize the context-usage write through the same per-stream queue as
  the token-usage write. persistTokenUsage reconciles the stored snapshot
  (read-modify-write); an unserialized trackContextUsage could store a newer
  snapshot between the read and write — or a stale reconciled write could land
  after a newer snapshot — clobbering the newer run's gauge when calls interleave.
  FIFO keeps each call's snapshot ahead of its own usage and behind the next.

* chore: import order in GenerationJobManager.ts
2026-06-16 11:05:44 -04:00
Danny Avila
44c253d48a
🪙 fix: Correct Context Usage Gauge After Summarization (#13744)
* 🪙 fix: Persist Context Snapshot + Summary Marker After Summarization

The post-summarization context is correctly compacted by the SDK, but the
breakdown wasn't reliably reaching the client, leaving the gauge on the
whole-history estimate (stuck at 100% forever once a conversation compacts).

Two server changes in buildResponseMetadata:
- Snapshot guard: persist the breakdown when a PRIMARY usage event follows the
  latest snapshot (tracked via contextUsageSink.latestUsageIndex, recorded in
  the on_context_usage handler) instead of a brittle snapshot-vs-primary count.
  A summarization detour adds an extra snapshot whose only following usage is
  tagged 'summarization', which the count guard could miscount and drop.
- Summary marker: whenever a turn compacts (summaryTokens > 0), persist a
  lightweight metadata.summaryUsedTokens (the pre-invoke compacted context size)
  UNCONDITIONALLY — so even when the full snapshot can't be saved (interrupted
  final call) or never reaches the client, the per-message estimate has a signal
  to cap the discarded history.

Tests: client.contextMetadata.spec (guard + marker, incl. marker-survives-drop)
and a real-pipeline summarization integration test.

* 🪙 fix: Cap the Context Estimate at the Summary Marker

When the gauge falls back to the per-message estimate (no usable snapshot on the
branch), sumBranch summed the ENTIRE branch history — after a summarization that
discarded most of it, this over-counts and pins the gauge at 100% in perpetuity.

sumBranch now stops at the deepest summarized response (metadata.summaryUsedTokens)
and records it as summaryBaseline; the walk counts only post-summary messages,
and useTokenUsage adds the baseline. So the estimate reflects the compacted
context (summary + recent turns), not the discarded history. USD/default
behavior unchanged when no marker is present.

Test: sumBranch caps a huge pre-summary history at the compacted baseline.

* 🪙 fix: Address Codex Review on the Summarization Marker

- Branch cost/usage is no longer truncated at the summary marker — sumBranch
  caps only the CONTEXT-window count there and keeps accumulating provider
  usage/cost to the root (cumulative spend isn't discarded by compaction).
- findBranchSnapshotAnchor stops at a summarized response with no snapshot of its
  own, so it can't recover a stale PRE-summary snapshot and show discarded
  history; the summary-baseline estimate is used instead.
- Abort path: buildAbortedResponseMetadata now persists the summaryUsedTokens
  marker (pre-invoke, no completedOutputTokens ambiguity, so safe on abort) so a
  STOPPED summarized turn isn't re-summed on reload.
- Marker baseline fallback now includes summaryTokens (a separate breakdown
  field) so it doesn't under-report the compacted size. DRY'd into a shared
  computeSummaryUsedTokens used by the completion and abort paths.
- Estimate popover surfaces the summary baseline as a row so the displayed rows
  reconcile with the header total.

Tests: sumBranch cost-not-truncated + anchor-stops-at-marker (client);
computeSummaryUsedTokens fallback + abort marker (packages/api).

* 🪙 fix: Attribute Persisted Context Usage to the Snapshot Run

Match the post-snapshot primary usage to the latest snapshot's runId before
persisting metadata.contextUsage. Parallel/direct runs interleave snapshots and
usage (A snapshot → B snapshot → A usage → B no-usage); the prior index-only
guard persisted B's snapshot with A's output. finalCallOutputTokens now filters
completedOutputTokens to the snapshot's run. Untagged events (older lib/resume)
match any run for back-compat.

* 🪙 fix: Harden Summary Marker Against Tool-Loops, Stale Anchors, and Emit Races

Codex round on the summarization marker:

- Avoid double-counting earlier tool-loop outputs in the summary marker: those
  outputs sit in BOTH the latest snapshot's pre-invoke baseline AND the response
  message's tokenCount the client estimate adds on top. computeSummaryUsedTokens
  now subtracts the run's prior primary outputs (priorRunOutputTokens) — the live
  path bounds them by the snapshot's usage index, the abort path by all primaries
  (an interrupted final call emits none). Single-call turns subtract 0.
- Stop treating pre-summary anchors as active: sumBranch no longer sets
  containsAnchor once the context is capped at a summary marker, so a stale
  pre-summary snapshot can't override the summary-baseline estimate.
- Capture latestUsageIndex BEFORE awaiting emitEvent: a yield (resumable SSE /
  Redis) during parallel runs could let this call's own usage advance the index
  past the event that proves the snapshot completed, dropping a valid breakdown.

* 🪙 fix: Subtract Summarization Output from the Summary Marker

recordCollectedUsage folds the summarization call's completion into the response
message's tokenCount, while the generated summary is also in the snapshot baseline
as summaryTokens. The client estimate (summaryBaseline + responseTokenCount) thus
counted the summary twice — inflating the gauge after compaction even on a
single-call turn whenever the full snapshot is unavailable. priorRunOutputTokens
now also counts summarization-tagged output (still excluding subagent/sequential,
which recordCollectedUsage keeps out of the reported total), so the marker
subtracts it. Updated unit + guard tests.

* 🪙 fix: Refine Marker Subtraction for Summarization RunId and Abort Boundary

Two Codex follow-ups on the marker-subtraction logic:

- Subtract summarization output regardless of runId: the summarize detour is its
  own model-end call that may carry a distinct runId, but its output still lands
  in this response's tokenCount AND the snapshot baseline (summaryTokens). It is
  now counted unconditionally (still within the response's own usageEmitSink),
  while primaries keep the parallel-run runId filter.
- Don't subtract primaries on the abort path: the job stores no snapshot/usage
  boundary, so a primary that completed AFTER the latest snapshot is NOT in the
  baseline; subtracting it would cancel real output and under-report. priorRun-
  OutputTokens gains an includePrimary flag (false for abort) — abort subtracts
  only the always-pre-snapshot summarization output.

* 🪙 fix: Run-Scope Summary Subtraction and Stop Subtracting on Abort

Two Codex follow-ups, resolved by reverting the round-4 detour:

- Run-scope the summarization subtraction: the summarize detour inherits the
  graph run id (traceConfig spreads config.metadata.run_id), so its usage shares
  the answer snapshot's runId — it is NOT a distinct run. priorRunOutputTokens now
  filters summarization by runId like primaries, so a parallel sibling run's
  summary (different runId, in the sibling's baseline) is no longer subtracted from
  this branch's marker. Drops the includePrimary flag added last round.
- Stop subtracting on the abort path: abort tokenCount is countTokens(text)
  (abortMiddleware) or absent (agents route) — it does not fold in summarization or
  earlier-call output the way recordCollectedUsage does, so the marker must keep
  the full baseline. buildAbortedResponseMetadata now subtracts nothing.
2026-06-14 18:23:30 -04:00
Danny Avila
4ee68d5240
💸 feat: Per-Agent Endpoint Token Config in Multi-Endpoint Billing (#13738)
* 💸 feat: Per-Agent Endpoint Token Config in Multi-Endpoint Billing

Price each collected/emitted usage item with the producing agent's resolved
endpoint token config, instead of the primary agent's for the whole graph.

Previously AgentClient.recordCollectedUsage and the subagent usage emitter used
a single this.options.endpointTokenConfig (the primary's) for every usage item.
A connected agent or subagent on a different custom endpoint that shares a model
id with an entry in the primary's tokenConfig was therefore mis-priced (a model
absent from it already fell back to the built-in rate map — no regression).

- Tag each usage with its producing agent: ModelEndHandler stamps
  usage.agentId = agentContext.agentId; createSubagentUsageSink stamps the
  child's subagentAgentId (UsageMetadata gains an optional agentId).
- buildAgentToolContext retains endpointTokenConfig so initialize.js can build
  an agentId -> endpointTokenConfig map from agentToolContexts (the one map that
  holds every agent, including pure subagents pruned from agentConfigs).
- AgentClient.resolveAgentEndpointTokenConfig(usage) looks up that map by
  agentId, falling back to the primary config; used by both the billing path
  (new optional resolveEndpointTokenConfig on recordCollectedUsage) and the
  subagent cost emitter.
- recordCollectedUsage's resolver is optional and falls back to the batch
  endpointTokenConfig, so the shared responses.js/openai.js call sites are
  unchanged.
- Tests: two-endpoint graph with a colliding model id prices per-agent; resolver
  nullish falls back to batch; subagent sink tags the child agent id.

* fix: Align emit-path cost with per-agent billing; honor known-agent built-in pricing

Addresses Codex review on the per-agent endpoint token config:
- Emit path (callbacks.js) now prices each on_token_usage event with the
  producing agent's config (resolved via usageCost.resolveEndpointTokenConfig),
  so streamed/persisted metadata.usage.cost matches the per-agent balance
  transaction. The agentId tag is resolved server-side and stripped from the
  emitted/persisted payload.
- Resolver (resolveAgentTokenConfig) now treats a known agent's config as
  authoritative, including undefined → built-in pricing, so a known non-custom
  agent in a custom-primary graph is no longer charged the primary's rates.
  Only untagged/unknown usage falls back to the primary config.
- endpointTokenConfigByAgentId records every known agent (value may be
  undefined) so the resolver distinguishes known-no-rates from unknown.
2026-06-14 12:00:32 -04:00
Danny Avila
b03b2a0a29
💾 feat: Persist Context Breakdown & Branch/Total Usage Cost (#13734)
* 💾 feat: Persist Context Breakdown & Branch/Total Usage Cost

Persist the granular context breakdown and per-response usage/cost on the
response message metadata, and re-derive branch + total usage/cost from a
per-message index so the popover survives reloads and is branch-aware live.

- Add aggregateEmittedUsage + buildPersistedContextUsage helpers in
  packages/api; capture the latest visible snapshot and every emitted
  on_token_usage payload via contextUsageSink/usageEmitSink.
- Attach metadata.contextUsage (Part A) and metadata.usage (Part B) on the
  agents response message in sendCompletion.
- Carry per-message usage on the token index; add sumTotalUsage/setEntryUsage
  and branch-scoped usage on sumBranch.
- Repurpose the session accumulator into a single in-flight pending holder;
  flush it into the index at finalize; hydrate breakdowns on load.
- Render branch cost with a conditional all-branches total in the breakdown.

* 🧹 chore: Remove orphaned com_ui_session_cost i18n key

* 🩹 fix: Address Codex review — normalize usage server-side, fix reload deltas

- Persist per-event-normalized display units in metadata.usage (TResponseUsage)
  so reloaded mixed-provider turns match the live session; client reads them
  directly instead of re-normalizing with a single stamped provider (P2).
- Persist completedOutputTokens (final call output) on metadata.contextUsage so
  a reloaded multi-call turn adds the post-snapshot delta, not the full
  tokenCount the snapshot already counts (P2).
- buildIndex preserves a prior entry's immutable usage when a rebuilt cache
  message lacks metadata.usage, so a mid-session rebuild (regenerate) keeps a
  sibling branch's flushed cost (fixes the e2e regenerate failure).
- Track costKnown so turns saved with contextCost off don't render $0.00 when
  cost display is later enabled (P3).
- Use an epsilon for the all-branches cost comparison to avoid a spurious total
  row from float summation order (P3).
- Update unit/integration/e2e tests for the new shapes; regenerate e2e asserts
  the all-branches total after reload (deterministic via persisted metadata).

* 🩹 fix: Address Codex round 2 — pending leak, cost coverage, reload delta

- Clear the in-flight pending usage on terminal abort/error (resetLive), so a
  stopped generation's tokens no longer merge into the next response (P2).
- costKnown now means COMPLETE coverage (ANDed): a branch mixing cost-bearing
  and cost-less turns is flagged incomplete and the cost row is hidden rather
  than rendering an under-reported total (P2).
- Drop the tokenCount fallback for completedOutputTokens on reload: only the
  persisted post-snapshot delta is used, so a multi-call turn whose provider
  emitted no usage_metadata no longer double-counts earlier output (P2).
- Update tokens.spec for AND coverage semantics + incomplete-cost case.

* 🩹 fix: Address Codex round 3 — no-usage snapshots, total coverage, provider-less cache

- Skip persisting metadata.contextUsage when the response emitted no primary
  usage event: without a known post-snapshot output the granular gauge would
  undercount the reply on reload, so fall back to the coarse per-message
  estimate instead (P2).
- Gate the all-branches cost row on totalUsage.costKnown so an incomplete total
  (a sibling saved without cost) never renders an under-reported figure (P2).
- aggregateEmittedUsage/finalCallOutputTokens now normalize per-event with the
  client's magnitude fallback (normalizeEventUnits) instead of billing
  splitUsage, so provider-less cached events match live on reload (P2).
- Add backend test for the provider-less cached case.

* 🩹 fix: Address Codex round 4 — abort attribution, complete cost coverage

- aggregateEmittedUsage persists cost only when EVERY call was priced; a partial
  pricing failure now omits cost so the client treats coverage as unknown rather
  than reading an under-reported sum as authoritative (P2).
- finalizeUsage flushes pending into the response entry only when events were
  folded this session (eventCount > 0), so a late/second resumable subscriber
  carrying persisted metadata.usage keeps it instead of being overwritten with
  an empty pending record (P2).
- On user stop, attribute the in-flight pending usage to the partial response
  (new attributePending handler) instead of discarding it in resetLive — the
  stopped reply's billed tokens are kept and still can't leak into the next
  response; resetLive's discard remains for the error path (P2).

* 🐛 fix: Persist branch cost across branch switches via sticky usage history

Branch cost vanished on switching to a sibling branch (until a new turn) — the
cost analog of the granularity bug. buildIndex rebuilds the token index from the
messages cache; a sibling generated this session whose cache message lacks
metadata.usage (and is transiently dropped from the cache during regenerate)
lost its live-flushed usage, so sumBranch found none and the cost row hid.

Fix: a sticky per-response usage map (conversationId → messageId → usage),
written by setEntryUsage and never rebuilt from the cache — the usage counterpart
of snapshotsByAnchorFamily for the breakdown. buildIndex/upsertEntries restore an
entry's usage from it when the message carries none; cleared on convo switch and
migrated with the index. Add unit coverage for the drop-then-readd regression and
an e2e assertion that branch cost survives a branch switch.

* 🐛 fix: Re-index on branch switch so branch cost survives the switch

The sticky usage history alone didn't fix the reported branch-switch cost drop:
on a branch switch no cache `updated` event fires, so the index subscriber never
re-ran, and the post-regenerate rebuild was skipped while `isSubmitting` was
still true — leaving the index stale and missing the now-viewed branch's
response entirely (sticky can only restore entries present in a rebuild).

Re-index from the messages cache on every tail change (created/finalize AND
branch switch), not just while submitting. The cache holds the full message set
at switch time, so the viewed branch's response is re-added and its usage
restored from metadata.usage or the sticky history → sumBranch finds it and the
branch cost renders. Verified locally: the branch-switch e2e now passes (the
cost section shows both the branch row and the all-branches total). Also fixed
that e2e assertion to target a single cost value (strict-mode safe).

* 🩹 fix: Handle stopped-stream usage — reset pending + persist abort metadata

Codex round (stop/abort edges):
- Resumable explicit-stop (intentional SSE close) reset UI state but never
  cleared pendingUsageFamily, so usage folded before the stop leaked into the
  next response in the conversation. Discard pending on intentional close
  (resetLive); a resume re-folds via backfillUsage, so nothing is lost.
- The abort save path (abortMiddleware) persisted the stopped response without
  metadata.usage/contextUsage, so its cost + breakdown vanished on reload.
  Rebuild both from the job's persisted tokenUsage (emitted payloads incl. cost)
  and contextUsage snapshot — parity with the normal sendCompletion path;
  breakdown gated on a primary usage event like buildResponseMetadata.

Deferred (per scope decision): mid-stream branch-switch transiently shows the
streaming branch's pending on the viewed sibling (cosmetic, until finalize).

* 🩹 fix: Persist abort metadata on the real agents route + tighten snapshot gate

Codex round (corrects last round's wrong-path fixes):
- Stopped AGENTS responses are saved by routes/agents/index.js (/chat/abort),
  not abortMiddleware — so last round's metadata fix never ran for them. Moved
  the rollup/snapshot builder into packages/api as buildAbortedResponseMetadata
  (shared, unit-tested) and applied it in BOTH abort save paths, so a stopped
  agent reply keeps its cost + breakdown on reload.
- Persist the breakdown only when the FINAL visible call emitted usage: track a
  per-response snapshot count and require primaryUsageCount >= snapshotCount.
  Previously any earlier primary usage event passed the gate, so a multi-call
  turn whose final call emitted no usage_metadata used an earlier call's output
  as completedOutputTokens (already counted by the latest snapshot) → reload
  over-reported. Now it falls back to the coarse estimate.

Resumable stop pending-reset (prior round, 3cde6fe035) already flows through
clearAllSubmissions → SSE close → the intentional-close handler's resetLive.
Deferred per scope: mid-stream branch-switch pending attribution (tracked).

* 🩹 fix: Abort breakdown over-count + resume re-fold after pending discard

Codex round (on the re-applied abort/snapshot work):
- buildAbortedResponseMetadata now persists ONLY the usage/cost rollup, not the
  context breakdown. The abort path can't tell whether the final call emitted
  usage (the job stores only the latest snapshot, not a count), so persisting
  the breakdown risked reusing an earlier call's output as completedOutputTokens
  (already in the snapshot) → reload over-count. Stopped/incomplete responses
  now fall back to the coarse gauge estimate, which is safe and apt.
- resetLive now also forgets the conversation's folded usage-event identities
  (clearUsageFolded). Discarding pending on a terminal/intentional close left
  the folded keys set, so a later resume's backfillUsage saw the persisted
  events as duplicates and never rebuilt pending — leaving the response's usage
  missing until a full reload. Clearing them lets the resume re-fold.
2026-06-14 10:48:07 -04:00
Danny Avila
db7011d567
📊 feat: Real-Time Context Window & Token Usage Tracking (#13670)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
* 📊 feat: Real-Time Context Window & Token Usage Tracking

* 🧪 fix: Align Pricing Spec Dep Signatures with TxDeps

* 🩹 fix: Resolve Codex Findings for Context Usage Tracking

* 📊 feat: Granular Tool Token Breakdown with Deferred Splits

* 🧪 test: Cover Session Cost in Mock E2E and Scope Usage Selectors

* 🧪 test: Live Host-Pipeline Usage Verification (Env-Gated)

* 🧪 test: Local Real-Provider Multi-Turn E2E Harness

* 🪙 fix: Keep Tagged Usage Buckets Out of the Live Context Estimate

* 🩹 fix: Scoped Token-Config Fallback and Sequential Visibility for Usage Events

* 🩹 fix: Address Usage Review Findings — Cost Timing, Scoped Caches, Finalized Output

- carry the post-snapshot output estimate into the context snapshot at
  finalize so the gauge keeps the last response after live resets
- accumulate per-rate billable units and price the session cost at
  render, so usage events arriving before the token-config load still
  count once it resolves
- pass user-scoped token-config cache keys through loadConfigModels
  fetches and drop the controller's unscoped fallback to prevent serving
  another user's resolved config
- tag emitted usage events with a per-run seq so resume dedupe never
  drops a distinct call with an identical payload
- admit the static tokenConfig override in the custom endpoint schema so
  it survives zod parsing into req.config

* 🩹 fix: Align Client Usage Accounting with Backend Cost Semantics

- classify cache tokens by provider (shared inputTokensIncludesCache from
  data-provider, consumed by both the backend billing path and the client)
  instead of a magnitude heuristic, so Anthropic/Bedrock turns where cache
  is smaller than uncached input no longer under-bill input
- mirror resolveCompletionTokens on the client so Vertex-style hidden
  thinking tokens are reflected in the Output row and session cost
- prefer endpoint pricing over adapter-provider pricing so a custom
  endpoint can price a known model name without built-in rates shadowing it
- carry static cacheRead/cacheWrite overrides through the tokenConfig
  schema and buildTokenConfigMap

* 🩹 fix: Honor Static Token Config in Billing; Tighten Usage Freshness

- initializeCustom now uses a static endpoint tokenConfig as the agent's
  endpointTokenConfig (billing + balance checks), not just the advertised
  UI config — previously the gauge showed admin rates while the agent
  billed against built-in tables
- invalidate the token-config query alongside models on user-key add/
  revoke so context windows and pricing refresh without a reload
- include maxContextTokens in ChatForm's stabilized conversation memo so
  the gauge reflects a changed context-window setting immediately
- feed the live output estimate from the legacy content path (direct and
  assistants streams), setting from cumulative part text rather than
  accumulating deltas

* 🩹 fix: Resume Usage Dedup, Agent Pricing, and Partial Override Billing

- fold usage events idempotently by (runId, seq) so resume backfill no
  longer resets the conversation totals — a mid-stream reconnect keeps the
  usage of prompts already completed earlier in the session
- tap replayed pending message/reasoning/content events so output streamed
  past the resume snapshot reaches the live estimate, not just the message
- resolve cost against the agent's backing endpoint (Agents conversations
  report endpoint `agents` / provider `openAI`, neither of which keys a
  custom endpoint's tokenConfig)
- getMultiplier/getCacheMultiplier fall back to the standard tables for
  models absent from a partial endpointTokenConfig, so a partial static
  override no longer bills non-listed models at defaultRate while the UI
  shows the correct pattern rate

* 🩹 fix: Repaired Output in Gauge, Cache-Rate Keys, Config Gate, Usage Cleanup

- live/completed gauge counts the repaired completion (normalized output),
  so under-reporting providers don't drop the response from used context
- translate static tokenConfig cacheWrite/cacheRead onto the write/read
  keys getCacheMultiplier reads, so cache tokens bill at the configured
  rate instead of the prompt-rate fallback
- clear the token index and usage atoms when leaving a conversation, so
  visited histories don't accumulate in memory for the tab's lifetime
- wait for startupConfig before mounting the gauge, so a deployment with
  contextUsage disabled never briefly mounts it or fires the token-config
  query on first load

* 🩹 fix: Move Token-Config Resolution to TS; Key Live Usage by Created Convo

- extract the token-config resolution (override gathering + cache lookup +
  buildTokenConfigMap) into resolveTokenConfigMap in packages/api, leaving
  the /api controller a thin request-scoped wrapper (CLAUDE.md TS rule)
- getConvoKey prefers the user message's real conversationId once the
  `created` event stamps it, so a new chat's first-response live gauge and
  totals land under the id TokenUsage subscribes to instead of NEW_CONVO

* 🩹 fix: Clear Stale Redis Job Usage; Live-Tap Legacy Streams; Share Fetched Config

- DEL the Redis job hash before re-creating it so a reused streamId can't
  inherit a prior run's contextUsage/tokenUsage and backfill stale usage
- tap the legacy {message,text} stream branch (non-agent OpenAI/Anthropic
  streams) into the live estimate, not just the content path
- copy a deduped fetch's token config to every sibling endpoint sharing the
  baseURL/key/headers, so /token-config resolves each by its own name

* ⏪ revert: Don't DEL Redis job hash in createJob (breaks cross-replica resume)

createJob is an idempotent join — a second replica calls it for the same
streamId to share an in-flight stream's state. DELeting the hash wiped the
prior replica's persisted created/usage state, so a joining replica missed
the created event (GenerationJobManager cross-replica integration test).
Reverts the F1 change from 2bfce0c34b; the stale-usage concern doesn't
arise in practice (streamId is unique per generation).

* 🩹 fix: Best-Effort Usage Emit; Tag Hidden Sequential-Agent Usage

- wrap the ModelEndHandler usage emit in try/catch so a failed telemetry
  delivery (closed SSE / Redis publish error) can't abort the handler
  before thought-signature capture, which would break resumed tool calls
- tag hidden sequential-agent usage as 'sequential' (non-primary) so the
  client folds it into session cost/totals but not the live context gauge,
  instead of letting an undefined usage_type inflate the visible gauge

* 🩹 fix: Refetch Stale Token Config on Mount; Normalize Vertex for Lookup

- useTokenConfigQuery refetches on mount when stale, so a user-key change
  that invalidates tokenConfig while the gauge is unmounted takes effect on
  return instead of serving the prior key's resolved config
- normalize a Vertex-backed agent's provider (vertexai) to the google
  token-config key, so Gemini context windows and rates resolve instead of
  showing unknown context / $0 cost

* ✨ feat: Server-Side Per-Event Cost (Authoritative Pricing for the Gauge)

Move usage-cost pricing to the single source of truth. The backend prices
each model call with the same billing functions (premium tiers via
getMultiplier(inputTokenCount), cache rates) and emits the USD cost on
on_token_usage when interface.contextCost is enabled; the client sums
emitted costs instead of re-deriving from base token-config rates.

- computeUsageCostUSD reuses prepareTokenSpend/prepareStructuredTokenSpend
  so the emitted cost matches what is billed (incl. premium thresholds)
- getDefaultHandlers gains a usageCost pricing context; initialize.js wires
  db.getMultiplier/getCacheMultiplier gated on contextCost (agents path)
- client UsageTotals carries a summed costUSD; retire the client-side rate
  lookups (costFromUnits/calcUsageCost) that drifted from backend pricing
  and produced the provider-keying / cache-key / Vertex / premium findings
- keep normalizeUsageUnits for the displayed token counts; token-config is
  still used for the context-window meter

Fixes the premium-tier session-cost under-report (gpt-5.x / gemini-3.1
above their input thresholds).

* 🩹 fix: Branch-Accurate Usage Snapshot + Clearer Gauge Track Contrast

- re-anchor the context snapshot from the user message to the response
  message at finalize. Regenerating a response branches off a shared user
  message, so anchoring on it made the snapshot read as "active" on both
  branches — switching to the sibling branch showed the wrong (other
  branch's) context. The response message is branch-unique, so sibling
  branches now correctly fall back to their own per-branch totals.
- raise the gauge ring's track/fill contrast (muted track, prominent fill)
  so the used portion reads clearly as a fill-level indicator

* 🩹 fix: Tag Sequential Usage in Billing; Emit Subagent Cost; Reset Live on Resume Errors

- tag hidden sequential-agent usage `usage_type: 'sequential'` on the
  COLLECTED usage (not just the emit), and treat it as non-primary in
  recordCollectedUsage (billed, excluded from the reported output total) so
  hidden intermediate output stops inflating the parent's tokenCount/pruning
- emit on_token_usage from the subagent usage sink (tagged `subagent`, with
  authoritative cost when contextCost is on) so the gauge's session
  cost/totals include billed subagent usage; it stays out of the live meter
- call resetLive on the resumable 404 and max-retry terminal branches so the
  gauge doesn't keep counting stale in-flight tokens after the stream ends

* 🎨 fix: Contrast the Popup Context Bar; Revert Ring Restyle

- raise the popup breakdown's context progressbar contrast (muted
  surface-tertiary track, prominent text-primary fill) — that's the bar the
  contrast feedback was about
- revert the gauge ring restyle (kept its original border-heavy track /
  text-secondary fill); the ring wasn't the element in question

* 🩹 fix: Stop Snapshot Granularity Leaking Across Branches; Revert Tree Memo

- a null-anchor context snapshot was treated as active on every branch,
  leaking one generation's granular breakdown onto sibling branches. Require
  a non-null (response-message) anchor on the viewed branch instead, so
  siblings without a matching snapshot fall back to their own totals.
- revert the buildTree WeakMap memo in messages.ts. buildTree is pure (builds
  from shallow copies) so the memo was behaviorally identical, but it was the
  feature's only change to core branch-navigation selectors — removing it
  matches upstream and rules it out of branch-navigation debugging.

* 🪙 fix: Thread Endpoint Token Config to Agent Billing, Cost, and Context Limits

Custom-endpoint agents resolve an endpointTokenConfig during agent init but
it never reached the AgentClient, so spending, emitted cost, and runtime
max-token resolution all fell back to default rates for those agents.

- Surface options.endpointTokenConfig on the returned InitializedAgent.
- Pass it to the AgentClient (this.options.endpointTokenConfig) so the
  spending path bills at configured rates.
- Thread it through usageCost to computeUsageCostUSD so emitted per-event
  cost matches billing.
- getModelMaxTokens/getModelMaxOutputTokens fall back to the built-in map
  for models absent from a partial override (matches buildTokenConfigMap);
  consolidates the duplicated fallback in pricing.ts.

* 🪙 fix: Preserve Granular Breakdown Across Branch Switches

The granular context breakdown lives only in the live on_context_usage
snapshot — a single per-conversation slot, anchored to the latest response
and overwritten by each generation. Switching to a branch generated earlier
this session lost its tool/skill/system rows and fell back to coarse totals.

Retain each generation's finalized snapshot in a per-conversation map keyed
by its branch-unique response id (snapshotsByAnchorFamily). When the live
snapshot is off the viewed branch, walk the branch tail for its deepest
stored anchor and render that breakdown. Bounded by generation count and
cleared on conversation switch; the live/just-generated path is unchanged.

* 🪙 fix: Harden Resume Seeding and Subagent Usage Emission

- useResumableSSE: skip the trailing-output live seed when the resume
  carries a context snapshot; the snapshot's messageTokens already counts
  produced output, so seeding it again inflated usage until the next reset.
- AgentClient subagent emitter: await GenerationJobManager.emitChunk like
  every other caller (it persists before publishing), so a floating promise
  can't race job cleanup and a Redis/publish failure is caught by the
  emitter's try/catch instead of surfacing as an unhandled rejection.

* 🧪 test: Playwright Coverage for Context Breakdown Granularity

Add a test-only data-testid distinguishing the granular snapshot breakdown
(context-breakdown) from the coarse message-history estimate
(context-estimate), then assert granularity in the mock e2e harness:

- renders the granular breakdown from the live on_context_usage snapshot
  (guards that the snapshot event actually reaches the popover, not just the
  usage totals).
- preserves the granular breakdown after switching branches — regenerate to
  overwrite the single live snapshot, switch back, and confirm the rows
  survive via the per-anchor snapshot history map.

Branch regenerate/sibling selectors mirror the existing chat.spec branch test.
All three usage specs pass against the mock pipeline.

* 🪙 fix: Correct Resume Live-Seed, Fallback Re-index, and Subagent Emit Flush

Codex round on the prior commit:

- countTrailingOutputChars now counts only output at the very END of the
  aggregated content (0 when the model paused at a tool call), and the resume
  path always seeds it. The earlier skip-trailing-tool-parts behavior plus the
  skip-seed-when-snapshot gate together over- or under-counted in-flight
  output on resume; one rule fixes both — pre-invoke snapshot budget is never
  double-counted, and genuine in-flight output is no longer dropped.
- useTokenUsage re-indexes from the messages cache on tail change while
  submitting. The cache subscriber is muted during streaming, so without a
  context snapshot (non-agent streams) sumBranch missed the created tail and
  dropped history + prompt until finalize. Bounded — tailId only shifts on
  created/finalize/branch-switch.
- AgentClient tracks subagent usage emit promises and flushes them in
  chatCompletion's finally. The sink fires the emitter without awaiting, and
  resume reads the usage emitChunk persists (HSET), so cleanup must not race
  it or resumed clients miss billed subagent usage.
2026-06-13 19:38:28 -04:00