Commit graph

10 commits

Author SHA1 Message Date
Marco Beretta
cd44344647 🔔 feat: Unseen Reply Indicators, Reply Notifications, and Mark as Unread (#15025)
* feat: add unseen reply indicators, reply notifications, and mark as unread

A reply that lands while the user is elsewhere now leaves a trace. The
conversation carries a durable reply stamp written with the assistant message
that backs it, the sidebar row shows an unread dot until that message is
actually rendered, and an optional tab badge, desktop notification and chime
announce arrivals in a tab that is away. Rows can also be marked unread by hand
from the row menu, in the archived view as well as the main list.

The stamp is written by a compare-and-swap so concurrent replies order on the
database rather than on the application hosts' clocks, and only once output that
a reader can actually see is persisted: a stopped turn with nothing to read, an
empty synchronized row, or a failed message write never lights a dot that cannot
be cleared. Read intent survives archives, metadata edits and preset applies,
and a later read always outranks a delayed unread.

* fix: gate reply alerts on config and announce only readable replies

Three review rounds kept finding the same two shapes, so both are answered once
rather than patched again.

The alert capabilities are now operator-configurable through
`interface.replyNotifications` in `librechat.yaml`: the tab badge, desktop
notifications and the chime each have a gate, and the away poll's page size is a
field rather than a constant, with defaults that reproduce the shipped behavior.
A capability the operator turns off reads as off whatever the device stored, and
its settings toggle is not offered.

Deciding whether a persisted turn may raise an indicator now lives in one place
in `packages/api`, with the Responses API controller, the abort route, the
Assistants thread sync and the client's override path all calling it instead of
carrying their own copy. That closes the last case where a reply nobody can read
lit a dot nothing could clear: a `/v1/responses` completion whose output held
only reasoning or tool calls persists an empty row, and stamping it left an
unread conversation that opening could not settle.

Also: the retention backfill no longer advances `updatedAt` past a reply stamp,
which an away tab reads as a metadata-only promotion and withholds the alert
for; arrival evidence is swept of conversations no cached list still holds, so a
long-lived tab stops growing with its lifetime number of replies; and generated
Lighthouse profile directories are ignored where the runner leaves them.

* test: open the sidebar by what the drawer paints, not by its test ids

The drawer became a mounted, sliding panel on canary, so the controls this
helper read no longer say what it assumed. `close-sidebar-button` answers from
the off-canvas copy before the expanded state commits, which returned the helper
with the conversation list still hidden behind the pane, and the panel's own
`new-chat-button` has a rail copy that a plain locator reaches first. Both
readings passed on the previous drawer, which unmounted when closed.

Settling on the close control's accessible name, and on the first candidate that
is actually painted, is what a reader perceives and survives either drawer. One
click is one toggle and the opener sits in the pane the drawer marks inert for
the whole of its travel, so each attempt is given that travel to itself rather
than reissued into it.

Also serves `interface.replyNotifications` in the operator-gate scenario with a
fresh body instead of the upstream response, since reusing its encoding headers
for a re-serialized payload leaves the client unable to read the config, which
looks exactly like an operator who configured nothing.

* fix: gate the assistant save on renderable output and stop overselling the poll limit

Two gaps this round's own work left behind.

`saveAssistantMessage` still decided to stamp from the message id alone, so an
Assistants run that persisted an empty row raised a dot that opening the
conversation could never clear. It now asks `isAnnounceableReply`, the same
predicate every other persistence path asks, which is the point of having moved
that decision into one place.

`pollLimit` advertised a range up to 1000 while `getConvosByCursor` clamps every
conversation read to a hundred-row page, so an operator who set 500 would have
been told it applied and silently served a hundred. The schema now stops at the
page the server will actually return; covering a deployment whose replies
outpace one page needs the server-side unseen query, not a wider page.

* test: let the mobile scenarios use the surface they are actually asserting on

Three scenarios opened the drawer on a page they only ever send a reply from.
On the mobile project that drawer sits over the composer and marks the pane
inert, so there was nothing left to type in and the run waited out its whole
budget. They ask for the composer now, which is what they were always about.

The two that read a row after leaving the list open it again first. A reply sent
from another tab closes this one's drawer, because `sidebarExpanded` is stored
per browser rather than per tab, and opening a chat closes it the way it does
for a reader. Both are what someone would do to look at the row, and neither
changes what is being asserted.

Mark as unread is dispatched rather than clicked, like the row controls around
it: the menu is portaled, and behind the drawer's scrim it inherits
`pointer-events: none` and goes click-dead.

* test: dispatch the row menu's unread item like the controls around it

The portaled menu sits under the mobile drawer's scrim, which intercepts the
pointer, so a real click retried until the budget ran out. The sibling scenarios
and the row's own controls already dispatch; these two were the last real
clicks.

* fix: hold reply alerts until the deployment answers, and read the resolved temporary state

The capability gate treated a startup config that had not loaded yet the same as
one that set nothing, so a device that stored "on" could badge, poll or notify in
the window before an operator's `false` arrived. Nothing is permitted now until
the config is in hand; only a loaded config that lacks the field reads as the
shipped default, which is the case a backend predating the setting produces.

The Responses API announcement read the temporary flag from the request body
alone, while the conversation save beside it resolves it from the stored
conversation first. A temporary chat restored or resolved on the server without
the flag in the body would have been announced through the unread indicators;
both completion branches now resolve it the same way the save does.

* fix: route the last reply stamps through the shared owner and hand back failed claims

Moving the announce decision into packages/api left three callers still deciding
for themselves. The agents controller re-stamped an unfinished turn whose
terminal persistence was skipped, and the resumed legacy turn stamped its
directly saved row, both from the message id alone, so a preempted or stopped
turn that persisted nothing readable lit a dot that could never be
acknowledged. Both ask isAnnounceableReply now. The non-agents abort path
carried an inline copy of the same predicate; it calls announceReply instead.
Error turns keep their direct stamp: the error card is what renders them.

A notification claim was written to shared storage before the notification
was constructed, so a constructor that throws left the reply claimed in every
tab with nothing ever shown. The claim is handed back when construction fails,
removed only while it still names that exact stamp.

* fix: judge the normal reply save by the shared predicate and hand back failed chimes

The regular BaseClient save still stamped any assistant reply that persisted,
readable or not, because the conversation write took a bare reply id. It takes
the persisted reply now and asks isAnnounceableReply before it stamps, which
leaves the error turns as the only direct stamps, and those are rendered by the
error card. saveTurnConversation is exercised against a real store in both
directions. announce.ts derives its write-context type locally so the two
modules do not import each other.

The chime claimed its replies before building the tones, so an output that
failed while they were being scheduled left them claimed in every tab with
nothing played. Those claims are handed back on failure, the same way a
notification that could not be constructed now releases its own.

* feat: let operators set the reply polling cadence

The away poll and the focused refresh were fixed at 30 seconds and five minutes,
which are request-rate levers on the conversation list and nothing an operator
could tune. Both are fields on `interface.replyNotifications` now, bounded and
defaulting to the values they had, and the watcher reads them through the same
capability hook as the rest of the block.

* fix: keep reply identity through list merges and drop rows the server no longer has

The list merge carried three of the four read-state fields by hand and left out
`lastResponseMessageId`, so an SSE update on an unseen conversation erased which
branch the reply landed on, and a missing identity reads as visible. It uses the
same `preserveReadState` helper as the other two merge paths now, and the test
that covered the other three fields covers this one too.

A local cache merge was only inspected on a list's first page for new arrivals.
A title or oldest-first sort leaves a replied row where it is, so its first
reply on a later page went into the alerts baseline without an arrival and its
chime or notification was lost. Local merges scan every loaded page; server
responses keep the first-page rule, and only rows already known are considered.

An unread request that matched nothing means the conversation is gone: the
server's owner-scoped update returns the row even for a no-op. The mutation
rolled back the read fields and kept the row; it removes it from the caches now,
as a delete does.

* fix: discard a job completion that resolves after the watcher has gone

The fetch that settles a finished job had no teardown guard, unlike the away
poll. Signing out or switching accounts unmounts the watcher, and a response
still in flight for the previous session would then merge into the caches the
next one reads. The result is dropped once the watcher has unmounted, tracked
on unmount rather than in the jobs effect's cleanup, which re-runs on every
change to the running set while its earlier completions are still wanted.

* fix: move error-turn stamps into packages/api and settle overlapping unread no-matches

Error turns were stamped in CJS: the error middleware checked eligibility, called
the stamp and shaped the event's conversation snapshot itself, and the agents
controller stamped its own. Both call announceErrorTurn now, and the
skipped-persistence restamp goes through announceReply, so no stamp is issued
directly from /api any more. The existing sendError tests pass against the real
helper unchanged.

An unread call that no longer owned the row dropped a server no-match with the
rest of its stale answers, so a deleted conversation could be restored by a newer
call's rollback. The no-match now evicts before ownership is consulted. It stays
after the superseded check: a superseded call never reached the server, and its
synthetic `modified: false` is not a no-match.

Also: Settings offers the reply toggles only once the startup config has loaded,
matching the capability hook; a cancelled reply-discovery snapshot is restarted
even though nothing observes it; and the focused refresh is capped at the five
minutes the discovery snapshot stays authoritative, since that refresh is what
renews it.

* fix: settle the assistant final in packages/api, announce attachment-only replies

Both assistant controllers carried the same persistence barrier: write the
response, check the row and the conversation both persisted, and unwrap the
settled conversation for the final event. That is behaviour, and it was
duplicated in CJS; settleAssistantFinal owns it now and the controllers call
it. Their existing tests pass unchanged.

The announce predicate ignored attachments, so a reply made only of files,
such as a resumed workspace artifact with empty content and text, raised no
dot, badge or alert. The message body renders attachments and acknowledgement
checks the body, so such a reply can be cleared as well as announced; every
announce path now passes them through.

The stopped-reply announcement took the storage engine's own id type in its
exported signature. It takes plain string ids now, and saveConvo accepts them
alongside its own, since the update casts them against the schema; the id type
stays inside data-schemas.

* fix: normalize the stopped reply's written ids in packages/api

The abort route filtered and stringified the ids of the rows it had written
before handing them to announceStoppedReply. That is data shaping, and it sat
in CJS; the announcement takes the ids as the caller holds them and drops the
unwritten ones itself.

* style: order the conversation row classes for Tailwind v4

The v4 upgrade on canary changed the class order Prettier enforces; these
three rows carried the v3 order.

* fix: replay replies a dead tab's focus lease held back, keep the focused refresh at a full page

A tab killed without firing blur or pagehide leaves its focus lease for up to
a minute. A reply arriving under it was baselined and then suppressed, and
nothing reran when the lease lapsed, so its chime and notification were lost.
Arrivals held back only by another tab's lease are kept and rechecked when
that lease expires; a lease the focused tab keeps refreshing still holds them.

The focused refresh reused the away-poll limit, so an operator who lowered it
also shrank the discovery page that keeps the unseen count complete behind a
filtered list. The limit now bounds only the away poll.

* fix: announce a held reply as soon as the focused tab releases its lease

A tab that blurs normally clears its lease long before the lapse, and a reply
held back for it is due as soon as nobody is looking. Lease changes from other
tabs now trigger the recheck, not only the expiry timer.

(cherry picked from commit b947738d0d)

Original-PR: https://github.com/LibreChat-AI/LibreChat/pull/15025
2026-10-01 03:46:16 +00:00
Danny Avila
f6335cb9d7
💰 fix: Bill Stopped Agent Turns From Provider Usage (#15889)
* 💰 fix: Record Token Usage for Stopped Agent Turns

The agents Stop route only signals the abort; the run's cleanup skipped recordCollectedUsage on an aborted signal, trusting an abort middleware that is mounted only on the assistants abort routes. Stopped and error-aborted agent turns therefore wrote no transactions and never debited the balance.

The run now records its collected usage on every exit, labelling a stopped turn's transactions with context 'abort'. The owner of the run charges, so a cross-replica Stop is still billed exactly once.

* 💰 fix: Make the Run the Single Charger for a Stopped Turn

The legacy assistants-mounted abort route still billed a non-assistants job — provider usage via spendCollectedUsage, or a text-count fallback — on top of what the run records on exit, and its array clear ran only after its own awaited write, so the two could race even on one replica. The route now only stops and persists; the run records its usage once, labelled through resolveRunUsageContext in packages/api.

* 💰 fix: Keep the Text-Count Fallback From Re-Billing Recorded Usage

BaseClient falls back to text-count billing whenever the recorded usage has no positive output count, so a stopped call that reported input tokens and no output was debited by recordCollectedUsage and then charged its estimated prompt again. AgentClient.recordTokenUsage now returns early once provider usage was recorded (hasRecordedProviderUsage in packages/api); an all-zero report still falls through to the estimate.

* 💰 refactor: Lift Fallback Token Billing Into packages/api

recordFallbackTokenUsage owns the whole fallback operation — the recorded-usage guard, the estimate spend, the reasoning row and the error boundary — taking spendTokens as an injected dependency. AgentClient.recordTokenUsage only builds the transaction metadata from the client and calls it.

* 💰 fix: Label Fallback Billing From the Run's Stop State

BaseClient invokes the estimate fallback without a context, so the sole row for an initial turn stopped before any provider usage was labelled 'message'. recordTokenUsage now defaults the context from the run's own abort signal.

* 💰 refactor: Default the Fallback Billing Label in packages/api

recordFallbackTokenUsage takes the run's raw abort state and derives the transaction label itself; AgentClient.recordTokenUsage passes the signal through and no longer chooses the context.

* 💰 fix: Guard Fallback Billing on the Collected Primary Entries

The stream aggregate recordCollectedUsage returns takes its input from the first primary call only, so a later cancelled call that reported input alone was billed yet invisible to the fallback guard, which then charged the estimated prompt again. recordFallbackTokenUsage now also consults the collected entries themselves (hasRecordedPrimaryUsage), ignoring summarization, subagent and sequential usage; AgentClient passes its collected entries through.
2026-09-13 21:51:05 -04:00
Danny Avila
8a46ca070c
🧊 feat: Persist Context Fading Tier Across Agent Runs (#15505)
* 🧊 feat: Persist Context Fading Tier Across Agent Runs

Carry the pruner's latched context-fading tier from `@librechat/agents`
through `contextMeta` the way `calibrationRatio` already travels, so a
historical tool result keeps the same truncated bytes from one run to the
next and Anthropic's prefix-based prompt cache survives across turns
(danny-avila/agents#497).

- Add `IAgentFadingTier` to the message and conversation `contextMeta`
  types, schemas and the data-provider zod schema
- Add `isAgentFadingTier`, `resolvePersistableFadingTier` and
  `resolveRunContextMeta` in `packages/api`; a tier is persisted once it
  carries information (masking active or a budget below the window), and
  a latched tier is kept even at a neutral calibration ratio
- Seed `createRun({ fadingTier })` from the parent response's contextMeta
  regardless of encoding, since caps are character-based; the field rides
  the existing forward-compatible `runConfig` variable and older SDK
  versions ignore it, as does `Run.getFadingTier` being absent
- Accept a valid `fading` field in event actor context and reject a
  malformed one, matching the calibration checks
- Capture run context meta through one module-level helper for both
  completion paths instead of two inline copies

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Seed Fading Tier Across HITL Resume, Validate at the DB Layer

Address review findings on fading tier persistence:

- Leave the "is this tier worth persisting" decision to the SDK, whose
  `Run.getFadingTier()` now returns only informative tiers; comparing
  against the client's window misclassified untouched conversations
  whenever a reserve ratio was configured
- Carry `contextMeta` through the HITL pause projection (staged approval,
  job metadata, resume state) and seed the rebuilt client from it, so a
  paused turn resumes with the same tier and calibration instead of
  re-deriving a shallower one and rewriting the prefix
- Share run seeding between `chatCompletion` and `resumeCompletion`, and
  make capture tolerate partial runs and bare resume contexts
- Validate `contextMeta.fading` in `commitAgentEventActorState` with a
  shared `isAgentFadingTier` guard in data-schemas, so a malformed tier is
  rejected at the DB layer instead of silently cold-starting the actor
- Reuse `IAgentEventActorContextMeta` for `IMessage.contextMeta` instead of
  extending an inline duplicate

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 test: Cover Resume Seeding From Paused Context Meta

Assert the resume controller seeds the rebuilt client from the context meta
captured at the pause before rebuilding the run, and move the capture
helper's JSDoc back above its function.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧹 style: Sort Imports in Fading Tier Plumbing

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Persist Fading Tier Through Redis Jobs and Branches

Address Codex and Copilot review findings on fading tier persistence:

- Deserialize `contextMeta` in `RedisJobStore`, whose explicit read mapper
  rebuilt every other pause field but this one, so Redis-backed
  deployments resumed with no tier to seed after every HITL pause
- Copy server-private `contextMeta` onto the assistant message created by
  `POST /api/messages/branch`, so the next turn seeds its pruner the same
  way it would from the parallel source response
- Constrain the `fading` subdocuments in both Mongoose schemas (version
  enum, positive budget, required fields) to match the zod schema
- Throw a distinct "fading tier is invalid" error from both context meta
  validators instead of reporting a calibration failure

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Carry Context Meta Through Stop and Re-Pause

Addresses the second Codex pass on the fading-tier persistence:

- A re-pause whose resumed segment has nothing persistable now clears the
  job's `contextMeta` in the same transition instead of leaving the first
  pause's calibration and tier for the next resume to seed from.
- The client publishes the run's live calibration and fading tier onto the
  job after each pre-invoke context snapshot (deduplicated on value), and
  the Stop path copies `jobData.contextMeta` onto the stopped response, so
  a follow-up from a stopped turn keeps the prompt prefix stable.
- Mid-run captures (HITL pause, Stop) read the graph's live state; `Run`
  refreshes its own getters only after the stream settles, so the pause
  used to persist the seeds it started from.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Fence Context Meta Publication Ahead of Stop

A Stop handled by another replica reads the job before the owner is
signaled, so a fire-and-forget publish could lose the race, and a run
stopped before its first context snapshot never published the tier it
inherited. The client now publishes the inherited seed before the run
streams or resumes, and the context-usage handler awaits each
snapshot's publish so the write lands before the model call it
describes begins. Failures still only log; the pre-run publish is
optional-called so bare resume contexts keep working.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Read Back Context Meta After the Abort Claim

The run owner publishes its context meta ahead of each model call and
awaits the write, so a publish landing between the abort's initial job
read and its terminal claim describes the call whose partial output the
abort snapshot carries. abortJob now reads the same-epoch job back after
the claim and content re-read and takes its context meta, so the stopped
response persists the tier that produced its bytes. The refresh is
best-effort and logs on failure like the content refresh beside it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Share In-Flight Context Meta Writes and Clear Neutral Live State

Two snapshot callbacks in a parallel-agent run can capture the same tier
before the first write settles; the publisher now caches the in-flight
promise per serialized value so the second caller awaits the same
durable write instead of treating an uncommitted value as published.
A live snapshot with nothing left to persist after an earlier publish now
writes a neutral record (ratio 1, no tier), since a running job's fields
cannot be deleted through the metadata writer, so a Stop no longer
persists an earlier non-neutral state onto the response. The pre-run
seed publish is unchanged and is told apart from live snapshots by a
flag from the context-usage sink.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 chore: Drop Unused Rest Parameter in Context Meta Test

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 feat: Persist Per-Agent Fading Tiers as Compact Context Meta

Reconciles the host with @librechat/agents at 9aacad1a, where graph
history stays canonical and each Run derives a provider-only projection
from the latched tiers. LibreChat therefore persists only compact state:
the calibration ratio, the default agent's tier, and now the per-agent
tiers from Run.getFadingTiers(), stored as validated entries so agent IDs
never become MongoDB field names and restored onto a null-prototype
record for RunConfig.fadingTiers. Truncated messages, canonical tool
content, projection provenance and per-message truncation state are
never persisted; the SDK's latched flag is stripped on capture.

The per-agent map travels everywhere the single tier already did:
response contextMeta, event-actor checkpoint state, HITL pause and
resume job metadata in both stores, the Stop path, and branch creation.
A shared Mongoose definition keeps the message and conversation schemas
identical, the zod schema validates entries, and commitAgentEventActorState
rejects malformed maps. Comments no longer describe graph messages as
truncated; only the provider projection is.

Tests cover: tool results and inputs reach the persisted content parts
at full size; contextMeta carries exactly the compact tier and
calibration fields; a persisted snapshot round-trips into the seeded
RunConfig unchanged and prototype-safe; per-agent tiers survive pause,
Redis job round-trips, and event-actor context validation.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Restore Event Actor Types and Publish Seeds Before Abortable Setup

The previous commit's doc rewording in the conversation types dropped the
event-actor interfaces that precede the fading tier, which broke the
declaration bundle in CI; they are restored unchanged.

The inherited context meta is now published onto the job as soon as the
parent's state is loaded and awaited at the top of chatCompletion and
resumeCompletion, ahead of run creation and the other abortable setup
stages, so a Stop during setup still persists the parent's tiers onto the
stopped response. The partial response saved when every subscriber
disconnects now carries the job's context meta like the Stop and pause
paths do. The branch route keeps the source's context meta in the saved
message but strips it from the HTTP response, matching client reads.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Serialize Distinct Context Meta Publications

Overlapping snapshots from parallel agents could carry different tiers;
each started its own write and both job stores keep whichever write
finished last, so an older snapshot could overwrite a newer one. Each
distinct publication is now issued after the previous one settles, in
call order, so the newest snapshot is what a Stop reads back. Equal
values still share one write.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Project Root Message Reads and Keep Inherited Meta on Setup Failure

The root GET /api/messages paths returned raw documents: the
single-message query read called getMessages without a projection and the
cursor page performed an unprojected find. Both now apply
CLIENT_MESSAGE_SELECT, with getMessagesByCursor accepting a select option,
so the persisted fading tiers never reach the client through the
paginated endpoint; the search-hydration read projects the same way.

A turn that loads a parent's context meta but fails before its run exists
used to replace the inherited meta with the capture of a missing run, so
the persisted error response lost the last valid tiers. Both finalizers
now keep the inherited meta when no run was created while a created run's
neutral state may still clear it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 refactor: Move Context Meta Publication Into packages/api

The publication coordinator (ordering, deduplication, in-flight sharing,
failure state) now lives in packages/api as createContextMetaPublisher,
with selectRunContextMetaToPublish deciding what a publication carries;
the legacy client keeps only the wiring to the job metadata writer. A
transiently failing write is retried with a short backoff before the
publication is reported and forgotten, so a snapshot's model call no
longer proceeds on the first rejection while the job holds the previous
record. The SDK swallows handler errors, so retrying is the only way to
strengthen the fence without failing the turn.

A terminal save now supplies contextMeta: null when the finished run has
nothing to carry, and saveMessage unsets a previously stored value in
that case, so a disconnect snapshot's record cannot outlive the run that
completed neutrally.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Widen the saveMessage Method Type for the Null Unset

The exported method interface still typed contextMeta as the message
field, so the spec exercising the null unset failed typechecking; it now
mirrors the implementation signature.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Publish Every Agent's Snapshot, Sanitize Search Hits, Unbound Agent IDs

The context-usage handler now awaits the run's context meta publish for
every ON_CONTEXT_USAGE event, hidden sequential agents included, while
the visible-snapshot bookkeeping and the client emission keep their gate;
a Stop before the next visible snapshot finds the tier a hidden agent's
model call latched. Search results are built from the hydrated hit minus
its server-private contextMeta, since Meilisearch hydration projects every
schema field. The per-agent tier entry guard no longer caps the agent ID
length: ephemeral agent IDs encode endpoint, model and sender without a
bound, and dropping such an entry would let that agent alone re-derive
its tier on the next turn. Also formats the publication spec so the
static checks pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Unset Context Meta Atomically, Keep Publication State After a Failed Write

- `saveMessage` folds the `contextMeta` unset into the same update that
  persists the terminal response, on both the plain and the provenance-merge
  paths, so a failure between two writes can no longer leave a completed row
  carrying a disconnect snapshot's state.
- `createContextMetaPublisher` tracks whether any record has committed
  separately from the retryable latest publication, so `hasPublished` stays
  true after a later publication exhausts its retries and a following neutral
  snapshot still overwrites the earlier record on the job.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Carry Context Meta Through the Resumable Stop Route and Neutral Resumed Completions

- The resumable Stop route copies the refreshed `jobData.contextMeta` onto the
  terminal assistant message it persists, as the abort middleware already
  does, so a stopped resumable response keeps the tiers that produced its
  partial output.
- A resumed run that completes with neutral context state now saves
  `contextMeta: null`, which unsets the paused segment's record instead of
  letting an omitted field keep it on the completed row.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Trust Only Server-Authored Context Meta, Unset It on a Neutral Stop, Read It From the Job on Disconnect

- `POST /api/messages/:conversationId` and conversation imports drop any
  client-supplied `contextMeta`, and the run seeds its calibration and fading
  tiers only from a server-authored parent response, so a forged tier can never
  shape a provider projection.
- Both Stop paths save `contextMeta: jobData.contextMeta ?? null`, so a job that
  re-paused with neutral state unsets what an earlier pause stored on the row
  instead of leaving it behind.
- The disconnect partial save reads the same-epoch job record for the run's
  published context meta, since the client-facing resume snapshot never carries
  server-private state.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧪 test: Mock the Job Store in the Resume Metadata Spec

The disconnect partial save now reads the same-epoch job record through
`GenerationJobManager.getJobStore()`, so the resume metadata spec's manager
mock provides it as the other disconnect spec already does.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Keep Context Meta Off the POST Message Response

`POST /api/messages/:conversationId` returned the saved row verbatim, so a
client write against an existing server-authored message echoed the row's
stored context meta. The route now projects the row through the same client
projection the branch route uses, which drops the server-private field.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 📦 chore: Bump @librechat/agents to 3.7.16 for Persisted Fading Tiers

3.7.16 is the first release that carries danny-avila/agents#497, so
`Run.getFadingTier()` / `getFadingTiers()` now report the latched tier
this branch persists and `RunConfig.fadingTier` / `fadingTiers` seed the
next run from it. The dependency set is unchanged from 3.7.15; only the
package entry moves.

Adds an SDK-backed round trip to the fading spec: a first pruner latches
an informative tier, `resolveRunContextMeta` reduces it to the persisted
shape, and a second pruner seeded from that shape reproduces the first
run's projected tool-exchange bytes under instruction and calibration
drift, while an unseeded pruner under the same drift relaxes its tier and
rewrites them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-02 20:29:43 -04:00
Danny Avila
0219513115
🛑 fix: Forward Run Abort Signal to Foreground Tool Calls (#15517)
* 🛑 fix: Forward Run Abort Signal to Foreground Tool Calls

Stopping a generation left in-flight tool calls running on the far side.
For MCP over streamable-http the external server kept burning CPU until
the tool finished or timed out, with no `notifications/cancelled` ever
sent.

The MCP layer was never the problem: `MCPManager.callTool` already
spreads `options.signal` into `client.request`, and the SDK's
`Protocol.request` sends `notifications/cancelled` the moment that signal
aborts. The signal simply never arrived.

Agent tools do not execute inside LangGraph's ToolNode — `eventDrivenMode`
dispatches `on_tool_execute` and `createToolExecuteHandler` invokes them.
The agents SDK puts the run's abort signal on the batch request
(`ToolExecuteBatchRequest.signal`, documented as one handlers should
forward), but the handler destructured every field except `signal` and
built its invoke config without it. `config.signal` was therefore
`undefined` in every foreground tool, so `createMCPTool` derived no
signal and the SDK registered no abort listener.

Forward it. This restores cancellation for all signal-aware foreground
tools, not just MCP. The detached background invoke keeps its own
controller, since that work deliberately outlives the turn.

An aborted call now rejects promptly and resolves an error result rather
than hanging, so the abort is logged at debug instead of as a tool error
— a user pressing Stop should not spray the error log.

* 🔇 fix: Keep Cancelled Tool Calls on the Filtered Path and Out of Error Logs

Addresses both findings from the Codex review of 48fded564a.

P1 — the cancellation branch returned before `filteredToolOutputResult`,
so with tool-output filtering configured an aborted call's error text
reached the turn uninspected. Worse, `runSignal.aborted` says the run is
over, not that this rejection was the cancellation: an unrelated failure
racing the Stop took the same unfiltered exit. Cancellation now only
selects the log level; filtering and the result shape are unchanged.

P2 — `createMCPTool` logs its own error before wrapping and rethrowing,
so every Stop during an MCP call still emitted an error-level MCP entry
and could pollute operational alerts. That catch now recognizes the abort
and logs at debug; the wrapped message still reaches the turn.

Both are covered by tests that fail against the previous commit.

* 🎯 fix: Require an Abort-Shaped Error and Spare Shared OAuth Flows

Addresses both findings from the Codex review of b3c67082cd.

An aborted run signal proves the turn is over, not that the rejection in
hand was the cancellation — a permission, OAuth, or upstream failure can
reject in the same tick a user presses Stop. Both quiet-log branches now
require the error to look like an abort as well, so a real failure racing
the Stop stays at error level and visible to operational alerts.
`isAbortError` moves to `@librechat/api` and the copy in
`abortMiddleware` is retired, keeping one implementation. Its spec pulls
the real function through its partial package mock rather than asserting
a stub.

Forwarding the run signal also reached ActionService, where two of the
three OAuth flows are keyed `userId:action_id` and so outlive the run
that opened them: a second run for the same action joins the very same
record, and the browser's OAuth callback reads its metadata to exchange
the code. `monitorFlow` deletes the key when a waiter's signal aborts, so
one Stop could strand a concurrent run and discard an authorization the
user had already granted. Those two flows no longer take the signal; the
run-scoped login flow, keyed by thread and run, still does.

Every guard is covered by a test that fails when the guard is removed.

* 🧷 fix: Detach the Stopped OAuth Waiter Instead of Dropping Its Signal

Addresses the Codex review of d53ff170ab.

Withholding the run signal from the shared Action OAuth flows protected
concurrent runs and the browser callback, but left this invocation
attached to the flow after the Stop. A callback landing late then resumed
`_call` straight into `preparedExecutor.execute` — a consequential API
request running after the user had already stopped the turn. Handing the
signal back is not the answer either, since `monitorFlow` deletes the
shared `userId:action_id` key on abort.

`detachOnAbort` separates the two: the caller stops waiting the moment
the signal aborts, while the shared flow runs on for whoever else needs
it — another run, or the callback exchanging its code. Late settlement of
the detached work is swallowed rather than surfacing as an unhandled
rejection.

Both shared flows now use it. The run-scoped login flow keeps the signal
directly, as nothing outside its run observes it.

* 🪪 fix: Keep an Action OAuth Abort an Abort

Addresses two of the three findings from the Codex review of 70fc0782b1.

Detaching the stopped waiter left the rejection to be misread on the way
out. The refresh catch treated it as a failed refresh and called
`requestLogin()`, emitting an OAuth prompt and opening pending
authorization state for a turn that had already ended. `requestLogin` and
the surrounding auth catch relabelled it `Failed to authenticate OAuth
tool` / `Authentication failed`, and the outer catch handed that to
`logAxiosError` — so an ordinary Stop produced two error-level entries
and returned failure text as the tool's result. Each boundary now lets an
abort through unchanged, and the outer catch logs it at debug and
rethrows so it stays a cancellation end to end.

Deferred: waiter-only cancellation in `FlowStateManager`, so a detached
`monitorFlow` stops polling instead of running out the flow's TTL. That
poller is bounded at three minutes and lives exactly as long as it does
for a waiter that never left, so it changes no lifetime this branch
introduced; giving `FlowStateManager` a non-destructive abort mode is its
own change, shared with MCP and indexSync.

* 🤫 fix: Classify the Abort at the MCPManager Boundary Too

Addresses the Codex review of 9bcf1aa57e.

`MCPManager.callTool` logs every rejection at error level before
rethrowing to `createMCPTool`, so a user Stop still emitted an MCP error
from the inner boundary even after the outer one learned to recognize it.
The same signal-plus-abort-shape classification now applies here.

The review also noted, correctly, that the existing test could not have
caught this: it replaces `getMCPManager` wholesale, so it only ever
exercised the outer logger. The new cases drive the real manager with a
connection whose `client.request` rejects the way the SDK does once a
request signal aborts, and cover both the cancellation and a genuine
failure racing the Stop.

* 🧹 chore: Restore Import Order in the Agents Tool Handler
2026-09-02 13:32:33 -04:00
James Todaro
4d246469dd
🧾 fix: Honor Disabled Transactions on the Abort Paths (#15099)
Resolve the transactions config from the request and forward it to both
abort write paths, so `transactions.enabled: false` is honored when a
generation is stopped.
2026-08-25 08:10:26 -04:00
Danny Avila
bc6b032421
🛑 refactor: Demote User Abort Logs (#13904)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
* fix: Demote user abort logging

* fix: Handle abort causes

* fix: Demote user-aborted agent completion to debug log

The error users still saw originated in AgentClient's completion catch,
which logged every caught error (including user aborts) at error level
before checking the abort signal. Branch on abortController.signal.aborted
so user-initiated aborts log at debug while real failures stay error-classified.

Also give the handleAbortError it.each cases distinct titles.
2026-06-23 09:55:21 -04:00
Danny Avila
0412f05daf
🪢 chore: Consolidate Pricing and Tx Imports After tx.js Module Removal (#12086)
* 🧹 chore: resolve imports due to rebase

* chore: Update model mocks in unit tests for consistency

- Consolidated model mock implementations across various test files to streamline setup and reduce redundancy.
- Removed duplicate mock definitions for `getMultiplier` and `getCacheMultiplier`, ensuring a unified approach in `recordCollectedUsage.spec.js`, `openai.spec.js`, `responses.unit.spec.js`, and `abortMiddleware.spec.js`.
- Enhanced clarity and maintainability of test files by aligning mock structures with the latest model updates.

* fix: Safeguard token credit checks in transaction tests

- Updated assertions in `transaction.spec.ts` to handle potential null values for `updatedBalance` by using optional chaining.
- Enhanced robustness of tests related to token credit calculations, ensuring they correctly account for scenarios where the balance may not be found.

* chore: transaction methods with bulk insert functionality

- Introduced `bulkInsertTransactions` method in `transaction.ts` to facilitate batch insertion of transaction documents.
- Updated test file `transactions.bulk-parity.spec.ts` to utilize new pricing function assignments and handle potential null values in calculations, improving test robustness.
- Refactored pricing function initialization for clarity and consistency.

* refactor: Enhance type definitions and introduce new utility functions for model matching

- Added `findMatchingPattern` and `matchModelName` utility functions to improve model name matching logic in transaction methods.
- Updated type definitions for `findMatchingPattern` to accept a more specific tokensMap structure, enhancing type safety.
- Refactored `dbMethods` initialization in `transactions.bulk-parity.spec.ts` to include the new utility functions, improving test clarity and functionality.

* refactor: Update database method imports and enhance transaction handling

- Refactored `abortMiddleware.js` to utilize centralized database methods for message handling and conversation retrieval, improving code consistency.
- Enhanced `bulkInsertTransactions` in `transaction.ts` to handle empty document arrays gracefully and added error logging for better debugging.
- Updated type definitions in `transactions.ts` to enforce stricter typing for token types, enhancing type safety across transaction methods.
- Improved test setup in `transactions.bulk-parity.spec.ts` by refining pricing function assignments and ensuring robust handling of potential null values.

* refactor: Update database method references and improve transaction multiplier handling

- Refactored `client.js` to update database method references for `bulkInsertTransactions` and `updateBalance`, ensuring consistency in method usage.
- Enhanced transaction multiplier calculations in `transaction.spec.ts` to provide fallback values for write and read multipliers, improving robustness in cost calculations across structured token spending tests.
2026-03-21 14:28:53 -04:00
Danny Avila
8ba2bde5c1
📦 refactor: Consolidate DB models, encapsulating Mongoose usage in data-schemas (#11830)
* chore: move database model methods to /packages/data-schemas

* chore: add TypeScript ESLint rule to warn on unused variables

* refactor: model imports to streamline access

- Consolidated model imports across various files to improve code organization and reduce redundancy.
- Updated imports for models such as Assistant, Message, Conversation, and others to a unified import path.
- Adjusted middleware and service files to reflect the new import structure, ensuring functionality remains intact.
- Enhanced test files to align with the new import paths, maintaining test coverage and integrity.

* chore: migrate database models to packages/data-schemas and refactor all direct Mongoose Model usage outside of data-schemas

* test: update agent model mocks in unit tests

- Added `getAgent` mock to `client.test.js` to enhance test coverage for agent-related functionality.
- Removed redundant `getAgent` and `getAgents` mocks from `openai.spec.js` and `responses.unit.spec.js` to streamline test setup and reduce duplication.
- Ensured consistency in agent mock implementations across test files.

* fix: update types in data-schemas

* refactor: enhance type definitions in transaction and spending methods

- Updated type definitions in `checkBalance.ts` to use specific request and response types.
- Refined `spendTokens.ts` to utilize a new `SpendTxData` interface for better clarity and type safety.
- Improved transaction handling in `transaction.ts` by introducing `TransactionResult` and `TxData` interfaces, ensuring consistent data structures across methods.
- Adjusted unit tests in `transaction.spec.ts` to accommodate new type definitions and enhance robustness.

* refactor: streamline model imports and enhance code organization

- Consolidated model imports across various controllers and services to a unified import path, improving code clarity and reducing redundancy.
- Updated multiple files to reflect the new import structure, ensuring all functionalities remain intact.
- Enhanced overall code organization by removing duplicate import statements and optimizing the usage of model methods.

* feat: implement loadAddedAgent and refactor agent loading logic

- Introduced `loadAddedAgent` function to handle loading agents from added conversations, supporting multi-convo parallel execution.
- Created a new `load.ts` file to encapsulate agent loading functionalities, including `loadEphemeralAgent` and `loadAgent`.
- Updated the `index.ts` file to export the new `load` module instead of the deprecated `loadAgent`.
- Enhanced type definitions and improved error handling in the agent loading process.
- Adjusted unit tests to reflect changes in the agent loading structure and ensure comprehensive coverage.

* refactor: enhance balance handling with new update interface

- Introduced `IBalanceUpdate` interface to streamline balance update operations across the codebase.
- Updated `upsertBalanceFields` method signatures in `balance.ts`, `transaction.ts`, and related tests to utilize the new interface for improved type safety.
- Adjusted type imports in `balance.spec.ts` to include `IBalanceUpdate`, ensuring consistency in balance management functionalities.
- Enhanced overall code clarity and maintainability by refining type definitions related to balance operations.

* feat: add unit tests for loadAgent functionality and enhance agent loading logic

- Introduced comprehensive unit tests for the `loadAgent` function, covering various scenarios including null and empty agent IDs, loading of ephemeral agents, and permission checks.
- Enhanced the `initializeClient` function by moving `getConvoFiles` to the correct position in the database method exports, ensuring proper functionality.
- Improved test coverage for agent loading, including handling of non-existent agents and user permissions.

* chore: reorder memory method exports for consistency

- Moved `deleteAllUserMemories` to the correct position in the exported memory methods, ensuring a consistent and logical order of method exports in `memory.ts`.
2026-03-21 14:28:53 -04:00
Danny Avila
e1e204d6cf
🧮 refactor: Bulk Transactions & Balance Updates for Token Spending (#11996)
* refactor: transaction handling by integrating pricing and bulk write operations

- Updated `recordCollectedUsage` to accept pricing functions and bulk write operations, improving transaction management.
- Refactored `AgentClient` and related controllers to utilize the new transaction handling capabilities, ensuring better performance and accuracy in token spending.
- Added tests to validate the new functionality, ensuring correct behavior for both standard and bulk transaction paths.
- Introduced a new `transactions.ts` file to encapsulate transaction-related logic and types, enhancing code organization and maintainability.

* chore: reorganize imports in agents client controller

- Moved `getMultiplier` and `getCacheMultiplier` imports to maintain consistency and clarity in the import structure.
- Removed duplicate import of `updateBalance` and `bulkInsertTransactions`, streamlining the code for better readability.

* refactor: add TransactionData type and CANCEL_RATE constant to data-schemas

Establishes a single source of truth for the transaction document shape
and the incomplete-context billing rate constant, both consumed by
packages/api and api/.

* refactor: use proper types in data-schemas transaction methods

- Replace `as unknown as { tokenCredits }` with `lean<IBalance>()`
- Use `TransactionData[]` instead of `Record<string, unknown>[]`
  for bulkInsertTransactions parameter
- Add JSDoc noting insertMany bypasses document middleware
- Remove orphan section comment in methods/index.ts

* refactor: use shared types in transactions.ts, fix bulk write logic

- Import CANCEL_RATE from data-schemas instead of local duplicate
- Import TransactionData from data-schemas for PreparedEntry/BulkWriteDeps
- Use tilde alias for EndpointTokenConfig import
- Pass valueKey through to getMultiplier
- Only sum tokenValue for balance-enabled docs in bulkWriteTransactions
- Consolidate two loops into single-pass map

* refactor: remove duplicate updateBalance from Transaction.js

Import updateBalance from ~/models (sourced from data-schemas) instead
of maintaining a second copy. Also import CANCEL_RATE from data-schemas
and remove the Balance model import (no longer needed directly).

* fix: test real spendCollectedUsage instead of IIFE replica

Export spendCollectedUsage from abortMiddleware.js and rewrite the test
file to import and test the actual function. Previously the tests ran
against a hand-written replica that could silently diverge from the real
implementation.

* test: add transactions.spec.ts and restore regression comments

Add 22 direct unit tests for transactions.ts financial logic covering
prepareTokenSpend, prepareStructuredTokenSpend, bulkWriteTransactions,
CANCEL_RATE paths, NaN guards, disabled transactions, zero tokens,
cache multipliers, and balance-enabled filtering.

Restore critical regression documentation comments in
recordCollectedUsage.spec.js explaining which production bugs the
tests guard against.

* fix: widen setValues type to include lastRefill

The UpdateBalanceParams.setValues type was Partial<Pick<IBalance,
'tokenCredits'>> which excluded lastRefill — used by
createAutoRefillTransaction. Widen to also pick 'lastRefill'.

* test: use real MongoDB for bulkWriteTransactions tests

Replace mock-based bulkWriteTransactions tests with real DB tests using
MongoMemoryServer. Pure function tests (prepareTokenSpend,
prepareStructuredTokenSpend) remain mock-based since they don't touch
DB. Add end-to-end integration tests that verify the full prepare →
bulk write → DB state pipeline with real Transaction and Balance models.

* chore: update @librechat/agents dependency to version 3.1.54 in package-lock.json and related package.json files

* test: add bulk path parity tests proving identical DB outcomes

Three test suites proving the bulk path (prepareTokenSpend/
prepareStructuredTokenSpend + bulkWriteTransactions) produces
numerically identical results to the legacy path for all scenarios:

- usage.bulk-parity.spec.ts: mirrors all legacy recordCollectedUsage
  tests; asserts same return values and verifies metadata fields on
  the insertMany docs match what spendTokens args would carry

- transactions.bulk-parity.spec.ts: real-DB tests using actual
  getMultiplier/getCacheMultiplier pricing functions; asserts exact
  tokenValue, rate, rawAmount and balance deductions for standard
  tokens, structured/cache tokens, CANCEL_RATE, premium pricing,
  multi-entry batches, and edge cases (NaN, zero, disabled)

- Transaction.spec.js: adds describe('Bulk path parity') that mirrors
  7 key legacy tests via recordCollectedUsage + bulk deps against
  real MongoDB, asserting same balance deductions and doc counts

* refactor: update llmConfig structure to use modelKwargs for reasoning effort

Refactor the llmConfig in getOpenAILLMConfig to store reasoning effort within modelKwargs instead of directly on llmConfig. This change ensures consistency in the configuration structure and improves clarity in the handling of reasoning properties in the tests.

* test: update performance checks in processAssistantMessage tests

Revise the performance assertions in the processAssistantMessage tests to ensure that each message processing time remains under 100ms, addressing potential ReDoS vulnerabilities. This change enhances the reliability of the tests by focusing on maximum processing time rather than relative ratios.

* test: fill parity test gaps — model fallback, abort context, structured edge cases

- usage.bulk-parity: add undefined model fallback test
- transactions.bulk-parity: add abort context test (txns inserted,
  balance unchanged when balance not passed), fix readTokens type cast
- Transaction.spec: add 3 missing mirrors — balance disabled with
  transactions enabled, structured transactions disabled, structured
  balance disabled

* fix: deduct balance before inserting transactions to prevent orphaned docs

Swap the order in bulkWriteTransactions: updateBalance runs before
insertMany. If updateBalance fails (after exhausting retries), no
transaction documents are written — avoiding the inconsistent state
where transactions exist in MongoDB with no corresponding balance
deduction.

* chore: import order

* test: update config.spec.ts for OpenRouter reasoning in modelKwargs

Same fix as llm.spec.ts — OpenRouter reasoning is now passed via
modelKwargs instead of llmConfig.reasoning directly.
2026-03-01 12:26:36 -05:00
Danny Avila
36c5a88c4e
💰 fix: Multi-Agent Token Spending & Prevent Double-Spend (#11433)
* fix: Token Spending Logic for Multi-Agents on Abort Scenarios

* Implemented logic to skip token spending if a conversation is aborted, preventing double-spending.
* Introduced `spendCollectedUsage` function to handle token spending for multiple models during aborts, ensuring accurate accounting for parallel agents.
* Updated `GenerationJobManager` to store and retrieve collected usage data for improved abort handling.
* Added comprehensive tests for the new functionality, covering various scenarios including cache token handling and parallel agent usage.

* fix: Memory Context Handling for Multi-Agents

* Refactored `buildMessages` method to pass memory context to parallel agents, ensuring they share the same user context.
* Improved handling of memory context when no existing instructions are present for parallel agents.
* Added comprehensive tests to verify memory context propagation and behavior under various scenarios, including cases with no memory available and empty agent configurations.
* Enhanced logging for better traceability of memory context additions to agents.

* chore: Memory Context Documentation for Parallel Agents

* Updated documentation in the `AgentClient` class to clarify the in-place mutation of agentConfig objects when passing memory context to parallel agents.
* Added notes on the implications of mutating objects directly to ensure all parallel agents receive the correct memory context before execution.

* chore: UsageMetadata Interface docs for Token Spending

* Expanded the UsageMetadata interface to support both OpenAI and Anthropic cache token formats.
* Added detailed documentation for cache token properties, including mutually exclusive fields for different model types.
* Improved clarity on how to access cache token details for accurate token spending tracking.

* fix: Enhance Token Spending Logic in Abort Middleware

* Refactored `spendCollectedUsage` function to utilize Promise.all for concurrent token spending, improving performance and ensuring all operations complete before clearing the collectedUsage array.
* Added documentation to clarify the importance of clearing the collectedUsage array to prevent double-spending in abort scenarios.
* Updated tests to verify the correct behavior of the spending logic and the clearing of the array after spending operations.
2026-01-20 14:43:19 -05:00