Commit graph

138 commits

Author SHA1 Message Date
Marco Beretta
6f4e36665e
🗄️ perf: Cut Serial and Redundant Database Reads (#16711)
* perf: Cut Serial and Redundant Database Reads

* perf: Pick the Earliest Balance per User in MongoDB
2026-10-03 10:41:47 -04:00
Marco Beretta
92432e95d5
🔐 feat: Enforce Required Two-Factor Authentication (#14731)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
* feat: enforce required two-factor authentication

Require local and LDAP accounts to enroll in TOTP before they can use the
app when ENFORCE_TWO_FACTOR_AUTHENTICATION is set. Sign-in, refresh, bearer
verification, optional share auth and image authorization all route an
unenrolled account into a staged setup flow, and every bearer credential
minted before enrollment or a password reset is retired.

Password reset retirement uses the existing credentialsChangedAt stamp, and
commitPasswordReset clears any staged enrollment in the same write. Passkey
sign-in follows the same enrollment gate as password sign-in.

* test: cover passkey sign-in handing off to required enrollment

* refactor: keep Mongoose user types out of two-factor auth contracts

* fix: withdraw a minted session when its retirement recheck fails

* fix: hold Agent API keys and the reset CLI to the required two-factor policy

* fix: adopt a setup token handed off after the setup screen mounted

* fix: move every tab that receives an enrollment hand-off into setup

* fix: read the enrollment gate's retirement cutoff in system scope

* fix: clear minted session cookies before the fallible session deletion

* fix: adopt a setup token persisted before the setup screen subscribed
2026-10-01 23:39:51 +02:00
Marco Beretta
26b6988355 🧯 fix: Outlast Failed Auth Cache Eviction on Credential Changes (#16435)
* fix: outlast failed auth cache eviction on credential changes

A credential change committed to MongoDB could report success while a
cached auth user document without the new credentialsChangedAt stayed
readable, so an old OpenID access token kept authorizing until the
cache TTL. The Redis-backed auth cache swallowed every failure (Keyv and
KeyvRedis resolve a failed GET as a miss and a failed DEL as a no-op), and
the eviction helpers stopped at the first error.

- The AUTH_USER_DOC namespace opts into throwOnErrors so a failed read or
  delete is visible; every caller of that namespace already catches.
- evictAuthUserDocs attempts every delete even when another fails and
  reports whether a document may still be served; the data-schemas and
  packages/api invalidation paths share it.
- updateUser waits out AUTH_USER_DOC_CACHE_TTL_MS before returning a
  credential change whose eviction was incomplete, so the change is only
  confirmed once no pre-change document can be served. Other updates stay
  best effort, and later revocations are delayed, never skipped.

* fix: record the reverse index before caching an auth user document

A document written before its reverse-index entry was cached without an
index naming it when the index write failed, so eviction could not find
it. Write the index first with a TTL that outlives the documents it names,
and skip the document when the index write fails.

* test: keep real data-schemas helpers in the pinned order spec

* fix: settle the auth cache after revoking reset credentials

Waiting out the cache TTL inside updateUser held the reset before its
session and passkey revocations, so a live refresh token or passkey could
mint an access token issued after credentialsChangedAt during the wait.
updateUser is best effort again; awaitAuthUserDocEviction retries the
eviction and waits one TTL only when it is still incomplete, and both the
web reset controller and the reset CLI call it after revoking sessions and
passkeys, before confirming the reset.

* fix: settle the credential barrier with a margin and on CLI revocation errors

Timers can fire a millisecond early, so the eviction barrier now waits the
cache TTL plus a 100 ms margin. The reset CLI also settles the barrier when
revoking passkeys or sessions throws, since the password is already changed.

* fix: treat an auth cache that cannot delete as not evicted

* fix: keep the reverse index until every document it names is evicted

A failed eviction still deleted the reverse index, so the credential
barrier's retry found no index and reported the cache clear while the
documents an unreadable index or a failed delete left behind stayed
cached. The index is now deleted only after it was read and every document
it names is gone, so a retry can still find what remains.

* fix: treat a malformed auth reverse index as unreadable

* fix: treat a reverse index with a non-string entry as unreadable

(cherry picked from commit 18b6937da3)

Original-PR: https://github.com/LibreChat-AI/LibreChat/pull/16435
2026-10-01 03:46:18 +00:00
Marco Beretta
2be35b2f91 💬 refactor: ChatForm UI redesign with unified palette (#14473)
* 💬 refactor: ChatForm UI redesign with unified palette

Replace the scattered composer controls (badge row, tools dropdown, attach
menu, floating steer chips) with one system: a searchable palette with
favourites, a composer bar holding active tools plus effort and speech
controls, a tray staging files and manual context, and a queue rail that
buffers messages while a reply is running.

Collapses the 94 development and review-fix commits of PR #14473 into one
commit so the branch could be rebased onto dev once; the upstream merge of
this pull request is a squash merge either way. The pre-rebase history is
kept locally at tag prerebase-backup (afde3e4b9a).

* fix: finish the dev rebase of the composer redesign

The rebase onto dev left four defects the merge itself produced, plus one
review finding the branch had never answered:

- ChatForm's `min-w-0` JSX comment lost its closing brace, so the file did
  not parse at all.
- `useSteering` carried both dev's local draft consumer and the host-injected
  one under the same name; the host's `consumeDraft` owns the storage key
  autosave actually parked on, so the local copy is gone.
- `useUploadOptions` kept dev's `awaitingAgentProvider` gate after the merge
  replaced the block that derived `isSavedAgent`; the predicate is back, from
  the same `isEphemeralAgentId` the shared allowance helper uses.
- `MessagesView` still reserved a band for the in-flight steer overlay and
  read `steerOverlayHeightFamily`, which nothing writes now that pending
  steers render inside the streaming reply. The reservation is gone and
  `ScrollButton`'s lift is optional; composer panels keep standing the button
  down through `composerOverlayCountFamily`.
- The aggregate MCP chip keyed on `mcp:pinned`, which a server literally
  named `pinned` also produces (`mcp:${serverName}`), so both entries shared
  one React key and one packing slot. It uses a key outside that namespace.

Also drops the locale keys whose only consumers the redesign deleted, ports
dev's extracted-text delivery path into `useAttachExisting` (every
attach-existing surface routes through it now), repoints the mock e2e
built-in helpers at the palette, and repairs test fixtures the merge left
stale (a query mock that blanked agent metadata, a Jotai store mismatch, an
ambiguous tray query, an ephemeral-agent fixture standing in for a saved one).

* test: cover the composer redesign's acceptance scenarios end to end

Twelve mock-harness scenarios, one `@scenario:` tagged test each, over the
behaviours this change is accountable for: the single searchable palette
(open, search, favourite across a reload, keyboard-only operation), the
composer bar and tray (active tool chip, a staged file riding with the
message, a per-turn thinking level that applies to the next turn only), the
queue rail (Enter buffers during a run and drains after it, keyboard reorder
changes the send order, the draft survives run completion), and the composer's
portaled surfaces (Stop reaches the run from inside a portal, a pending tool
approval in the bar resumes the run).

Two contract scenarios are deliberately absent, with reasons: split-pane Stop
isolation is not runtime-reachable because `ChatRoute` mounts one pane, and
code-approval mode mid-run needs an attached stateful code environment the
mock lab does not configure, so dev's `ChatForm.approvalMode` unit spec stays
its coverage.

* fix: make the composer's acceptance run and the whole client suite green

The first verification run of the twelve scenarios passed six and failed six,
and the full client jest suite (which CI runs sharded, unlike the diff-scoped
local precheck) failed fourteen tests. One of each group was a real defect; the
rest were queries and fixtures still aimed at surfaces the redesign replaced.

Defects:

- `useQueueDrain` held its send lock after a revealed queued turn settled, so
  the next server-owned row could never be selected. The reveal path now owns
  its own lock and releases it when the revealed turn clears.
- `ChatForm` recomputed the during-run slot without `consumeComposer` in its
  dependency list, and still carried a landing flag the host prop replaced.

Tests and lab:

- The five `ChatForm.attachments` focus cases now drive the palette dialog and
  the tray's quote chips instead of the deleted attach menu and quote popup,
  with the same strict focus assertions; the harness gains the `main` landmark
  the palette portals into, and the touch fixture a complete `matchMedia`.
- `Palette.spec` no longer closes a hoisted `jest.mock` factory over a later
  `const`; `useSteering.spec` follows dev's toast wording.
- The six failing scenarios were repointed at what this lab actually renders:
  uploads go through the palette's local-computer row, the reasoning control
  needs `interface.parameters`, the queued-row contract is the row and its
  drain rather than a control the submit consumes, the palette's keyboard path
  uses a built-in the lab grants, the slow-reply wait uses the fixture's chunk
  marker, and the approval scenario selects its agent through the builder flow
  the passing approval specs use.

* fix: close the eight findings Codex raised on the rebased head

Codex reviewed the rebased heads and reported eight issues; all eight were
real. Two of them concern code this session added.

Behaviour:

- A custom or OpenRouter endpoint that declares neither `paramDefinitions`
  nor a `reasoningFormat` no longer gets a reasoning control inferred from the
  generic settings list. Capability now has to be declared (a reasoning-key
  parameter definition, a non-disabled `reasoningFormat`, or a known reasoning
  provider as the default params endpoint); a `disabled` format always
  resolves nothing, and every consumer — the composer control, the pre-send
  validation and the server — asks the same resolver, so a gateway backed by a
  non-reasoning model can no longer offer a level the turn would fail on.
- Restoring a queued attachment keeps its `llmDeliveryPath`, so an extracted
  document is routed as its text rather than as raw provider content.
- The placeholder-to-durable reasoning migration now requires this pane's own
  live submission, so opening another conversation from a new chat no longer
  carries the pending level into it. The created/sync case it was written for
  still carries.
- The queue rail treats an acknowledged server-owned row as an ordering
  boundary: local rows still reorder among themselves, a server-owned row
  cannot move, and a move that would cross one is refused out loud — the
  queued-turn API has enqueue, list and cancel only, so a client-side reorder
  across that boundary could not be made authoritative.
- The palette resolves an agent's skill scope with `resolveAgentSkillsScope`
  instead of treating an empty list as an open catalog, and the attach target
  resolves `useResponsesApi` with `resolveUseResponsesApi`, matching what the
  run will actually do.

Policy:

- The reasoning-override behaviour moved out of the CJS middleware into
  `packages/api/src/modelSpecs/reasoningOverride.ts`, which returns a
  discriminated result; the middleware is wiring again, and the behavioural
  cases moved with it.
- `showComposerTips` and `showFilesDialog` are Jotai atoms now, the tips
  preference keeping its storage key and reaching the composer as a prop.

* fix: make the redesigned composer reachable from the mock e2e helpers

CI's Playwright shard failed four `file-provisioning` tests on this branch
because the shared upload helpers still clicked `#attach-file-menu-button` and
`#attach-file-button`, entry points the redesign removed — attaching now goes
through the palette, which also owns the file input. Both helpers drive the
palette now, `uploadViaLegacyOption` mapping its two destination names onto the
palette rows that route to the code environment and the vector store, and the
specs that drove the old surfaces directly (`code-upload`, `thread-renderers`,
`unified-upload`) follow.

Also closes the last client/server disagreement the acceptance run exposed: a
custom endpoint that declares a reasoning key in `paramDefinitions` had that
declaration merged into the provider defaults and then discarded by model-name
inference, so the composer offered a level for `mock-model-a` that the server
rejected with "Invalid reasoning override". An explicitly declared key now
selects the merged setting before inference, while an undeclared generic custom
or OpenRouter target still resolves nothing.

The twelfth acceptance scenario is rescoped: a paused approval keeps the
composer usable and resumes to completion, but the composer-hosted approval
panel and chip are not asserted, because the pending action is keyed by the
stream's conversation id while the composer stays bound to the new-chat
sentinel — in code byte-identical to dev, and mirroring the write onto the
sentinel did not surface the panel. Tracked at berry-13/LibreChat#39.

* test: finish the acceptance run and keep the client typecheck green

- The thinking scenario read `reasoningOverride` from a `userMessage` wrapper
  the chat request does not have; the trace shows the request carrying
  `{ key: 'effort', value: 'max' }` at the root, so the assertion reads it
  there. Source behaviour was already correct.
- The approval scenario timed out on mobile because the Agent Builder drawer
  stayed open after selecting the agent and swallowed the pointer events meant
  for the composer; it now closes the sidebar and waits for the drawer to go
  inert before sending.
- Two specs typed a named custom endpoint's label as an `EModelEndpoint`, which
  CI's client typecheck rejects (the label is exactly what the case is about).

All twelve acceptance scenarios now pass in desktop light, desktop dark and
mobile emulation.

* fix: close the nine findings Codex raised on the verified head

Two were defects in code this session added while closing earlier findings:

- A reasoning key declared in `paramDefinitions` but absent from the generic
  settings list was accepted as capability and then never inserted, so the
  resolver exposed and validated the generic `reasoning_effort` instead. A
  declared key is now appended to the effective settings and resolved before
  any fallback, for `effort`, `thinkingLevel` and `thinkingBudget` alike.
- The queue's reorder guard validated from the destination row's own index, so
  `target === index` rejected every pointer drag and only keyboard reordering
  worked. It validates the dragged row's identity now; the server-owned
  boundary, its announcement and the disabled grips are unchanged.

The rest:

- Stop no longer falls through to the document-wide control once a focused
  pane has been resolved: focus in an idle pane while another generates used
  to abort the other pane's run.
- `PendingSteers` is mounted once in `MessagesView`, above the renderer
  switch, so a sent or failed steer keeps its in-flight, retry and
  send-as-new surface with `FLAT_THREAD` off, where the recursive renderer
  had none and the composer no longer carries a fallback.
- The file-list policy moved out of the CJS route into
  `packages/api/src/files/list.ts`, and its cap is `interface`-level config
  (`fileListLimit`, default 100) instead of a literal in the route.
- The steer-arm confirmation window is `interface.steerArmConfirmationTimeoutMs`
  with its current 10s as the default, travelling the same path as the other
  client-consumed timings.
- The pause predicates judge the active branch's tail rather than the last
  assistant in array order, so an inactive sibling branch can no longer
  enable or disable the inline escalation control.

* fix: move persisted reasoning validation into api package

* fix: restore composer entry points and defaults

* style: normalize reasoning helper formatting

* test: open mobile sidebar before account menu

* test: use settled sidebar layout for mobile account menu

* test: handle empty mobile sidebar in account scenario

* test: target account menu in mobile drawer

* fix: keep composer surfaces and mobile drawer reachable

* fix: let thread approvals receive pointer events

* fix: stabilize composer steering and shortcuts

* fix: commit queue drags and centralize file modal

* fix: return a canceled steer's text to the composer

Canceling a pending steer dropped the words the user had written: the
bubble disappeared and nothing carried the text back, so a reclaim that
the server confirmed still lost the message.

Hand the reclaimed text to `pendingComposerTextByConvoId`, which the
composer already drains on its first render, and append rather than
overwrite when the composer holds a draft the user typed while the cancel
request was in flight.

* fix: close the six findings Codex raised on the pre-rebase head

Pending steers keyed off the Recoil conversation id, which lags the route
during warm-cache navigation, so the surface could sit at the destination
thread's tail while its Cancel and Escalate acted on the source run. It
now keys off the rendered tree, as the mount window already does.

Arming a steer removes the control that was activated, dropping keyboard
focus on the document body. `useSteerEscalate` already exposes `onArmed`
for exactly that; the bubble now passes one that moves focus to the row's
Cancel action.

Opening the file manager from the account menu left its dialog with no
opener: the modal rejects an active element inside a menu, and nothing
supplied a fallback, so closing it returned focus to the body. The opener
is published alongside the open flag and cleared on close, which keeps the
shortcut path's composer-focus behaviour intact.

A steer the server rejected was swept into the queue with no marking, and
the run-end drain then sent those words as a fresh turn without the user
touching Retry or Send as new. Converted failures now carry
`needsExplicitSend`, which the drain skips the same way it skips a row the
rail is mid-edit on.

The queued-turn reconciliation window was a hard-coded 60s, unreachable by
an operator behind a slow proxy or a delayed read replica. It ships as
`interface.queuedTurnReconciliationTimeoutMs` with that value as default,
beside the steer arm timeout it belongs with.

Reasoning-override shape validation moved to
`parseReasoningOverrideRequest` in packages/api, leaving the middleware to
map a rejection onto its own error response.

* fix: keep the composer's reasoning level in step with the parameters panel

`stableConversation` memoized the conversation without any reasoning field
in its key, so editing the level in the Parameters panel produced a new
conversation object that every listed dependency ignored. The composer
went on showing the old level while the next ordinary send already used
the new one.

* test: stop the sidebar helper waiting for a drawer desktop never renders

`ensureSidebarOnScreen` measured `#mobile-drawer` whenever the sidebar had
no Pinned region. Desktop renders no drawer, and `boundingBox()` on a
locator that matches nothing waits for it, so every scenario that reached
that branch at a desktop viewport timed out instead of continuing. Check
for the drawer's absence rather than measuring it.

* test: open the mobile drawer from its inert state, not its geometry

A closed drawer stays mounted and keeps its layout box, so measuring the
Pinned region reported it on screen while every control inside was still
untappable. `inert` is the state the component actually sets when it
closes, so the helper reads that and clicks the header opener before
anything else measures.

* fix: return a canceled steer whole, through one recovery boundary

A steer that stops being a steer was handed back three different ways,
and each one lost something. The immediate cancel wrote `steer.text`
into the composer handoff atom, dropping the attachments, quoted
excerpts, manual skill picks and reasoning override that `settleReclaimed`
then deleted with the pending object. The cancel that won its race
against its own POST cleared the pending marker and restored nothing at
all, so those words disappeared. A definitively rejected steer could
only be resent unchanged, because the row offering to edit it was gone.

Give the recovery one boundary. `useSteerRehome` hands the whole steer
to the composer through the guarded restore the queue rail already uses,
and converts it into an ordinary queued follow-up when the composer
refuses: a draft in the box, another chat on screen, a paused question,
or a composer that unmounted while the reclaim was in flight. The chip
goes either way, since a pending row claiming the server still holds the
words is a lie once they live somewhere else.

The composer is a sibling of the message list rather than its ancestor,
so the thread had no way to reach that restore. The pane's host now
holds the reference and both sides read it there, which also means a
reclaim resolving after the composer is gone refuses instead of writing
into a dead form.

Because the restore refuses rather than overwrites, a draft typed while
a cancel is in flight is no longer glued to the returned steer; the
steer becomes its own queued message and the draft is left as typed.

* test: drive the surfaces these scenarios actually reach

Four scenarios were failing against surfaces the user never touches.

The approval scenario clicked Approve on the thread's copy of the
pending card. The composer's review panel floats over the tail of the
thread and owns the batch submit, so that copy sits behind it and the
click retried until the test timed out. Drive the panel instead; the
thread card stays asserted as the record of the request.

The palette scenario counted rows the moment the dialog painted, then
expected clearing the filter to restore that number. The catalog arrives
in waves: the MCP server list is released by the app-wide warmup timer
and the skills catalog only starts fetching once the dialog's open
effect has run, so the count kept climbing under the test. Wait for the
last wave and for the count to hold still, then snapshot.

The sidebar helper opened the mobile drawer twice through two signals
that disagree. It waited for `inert` to clear, which happens on a state
commit several frames ahead of the slide, then fell through to a
bounding box still reading off-screen and clicked the opener again, into
a machine already opening. One open, through `inert`, then wait for the
drawer to settle in the viewport.

Splitting the cancel scenario follows the recovery change: an empty
composer takes the steer back, and a staged draft keeps the box while
the steer becomes a queued message of its own.

* fix: hold a server-rejected steer for an explicit send

Editing a definitively failed steer falls back to the queue when the
composer will not take it. Queued like any other follow-up, the run-end
drain would send it the moment the reply finished, which turns the
server's refusal into an automatic new turn carrying the payload it
refused.

Mark that fallback the way the rest of the rejected-steer path is
marked, so the row waits for the user to send it.

* test: restore two assertions this branch broke while rewriting specs

Both specs were updated for the redesigned surfaces and both lost a step
they needed, and both fail deterministically in CI as a result.

thread-fold pressed Enter mid-run and waited for a queued row, under a
comment claiming Enter queues by default. It does not: the default
during-run action is steer, so Enter produced a pending steer and the
queued row the next line waits for could never appear. Ctrl/Cmd+Enter is
what takes the other action, which is the one this test needs.

mcp-oauth-readiness asserted on the server card's Cancel button straight
after clicking Connect. Connect opens the OAuth dialog over the panel,
and it is modal, so while it is open the card underneath is marked
aria-hidden and Cancel is not in the accessibility tree at all, which is
why the failure reads "element(s) not found" rather than "not visible".
The rewrite dropped the dismissal the previous flow performed; restore it
as a helper both call sites share.

* test: reach the account menu without the pinned section

The two account-menu tests borrowed `ensureSidebarOnScreen`, which also
settles the Pinned section's placement for the pinned specs. They never
touch that region, and on a run where an earlier test left a pinned
conversation the section exists but has not been laid out yet, so both
tests spent fifteen seconds polling it and failed on the wait rather than
on anything they assert.

Give them a helper that does only what they need: open the mobile drawer
from `inert`, the one signal that distinguishes a closed drawer from an
open one while it stays mounted, then wait for the slide to reach the
viewport. Desktop renders no drawer and needs nothing.

* test: open the drawer by what it shows, not one early read

The helper decided once, from a single read of `inert` taken right after
navigation. On a fresh mobile load the drawer can still be mounting, so
that read reports a state the app has not settled into, the helper
returns believing the drawer is open, and the account menu inside it is
never shown.

Retry against the thing the caller actually needs, the account menu being
visible, re-checking before each attempt. While the drawer is open its
control is labelled "Close sidebar", so the retry cannot toggle it shut.

* fix: keep the composer observing MCP authorization where its tools apply

The old badge row mounted its provider with observeToolAuthorization set
whenever the composer showed ephemeral tools, so on a plain endpoint the
chat kept MCP connection status and discovered tools in the query cache.
The redesigned composer mounts the same provider without the prop, so it
defaults to off and nothing on the chat side loads that state any more.

The Agent Builder's tool library reads that cache. With it empty, every
server reads as Unknown with no tools, and clicking one opens its detail
dialog offering to connect instead of adding it to the agent.

Pass the same gate the badge row used: showTools is the old
showEphemeralBadges expression.

* test: let the palette close before reopening it, and name what is asserted

enableBuiltinTool pressed Escape and returned while the palette was still
animating out. A caller that reopened it straight away resumed the same
popover, scroll offset included, so the Attach rows at the top sat outside
the virtualized window and upload helpers timed out looking for them. Wait
for the dialog to be gone before returning.

The staged-file assertions matched the bare file name, which the palette
now also lists under recent uploads while it closes; scope them to the
tray they are about.

The OAuth dialog dismissal waited for a title built from the server's
display name, but the dialog is titled with the server name.

* feat: make the queued-send lock expiry configurable

The per-pane lock that stops the run-end drain and an explicit Send now
from submitting the same queued message twice gives up a broken claim
after a fixed 60 seconds. Until then every drain and every Send now for
that pane is refused, and a deployment had no way to tune that window.

Expose it as interface.queuedSendLockTimeoutMs beside the queued-turn
reconciliation timeout, defaulting to the same 60 seconds so nothing
changes unless an operator sets it. The interface loader copies each
field explicitly, so the value is added there too; without that line a
configured value would never reach the client.

* fix: decline interrupt-and-steer while reasoning is staged

With a reasoning choice staged, the Interrupt & steer row is disabled,
but its keyboard chord still reached interruptSteer, which fell back to
interruptAndSend: it aborted the run and threw away the partial answer
the action promises to keep.

A staged reasoning choice belongs to a new generation that a steer into
the live one cannot carry, so decline the chord exactly as the row does,
the same way the function already declines while paused on an approval.
The words stay in the composer, where Enter queues them as a turn of
their own.

* fix: wake the queue drain when a late steer retry is queued

Retry marks a failed steer sending, and a sending steer is kept out of
the run-end sweep so its own POST settles it. When the run finishes
while that POST is in flight, the drain spends its one-shot signal on a
queue the steer is not in yet; the late ack or NO_ACTIVE_RUN then moves
it into the queue, where it sat until the user sent it by hand or
another run ended.

Every other late conversion re-posts the last completed run end through
the composer's rewakeDrain. The retry lives in the thread, so the
composer now publishes that function through the same host it already
uses to hand a reclaimed steer back to the draft, and the retry calls it
after converting. It stays a no-op unless the run completed cleanly and
no newer signal is armed, or when no composer is mounted.

* fix: honour thinking-budget ranges wider than 200,000

The request-scoped reasoning override capped thinkingBudget at 200,000
in the shared schema. An operator's paramDefinitions may advertise a
wider range, and the composer renders it, but every value above the cap
was dropped by the control's parse and rejected by the server's before
the range-aware check could run.

That check already exists: resolveReasoningOverride validates the value
against the endpoint's resolved range. Drop the fixed ceiling so the
resolved range is the bound, keeping the integer and lower-bound rules.
The new case covers both directions on a custom endpoint whose range
goes to 500,000.

* refactor: persist the preliminary reasoning override through the shared helper

The agents controller validated the request's reasoning override with
the schema itself before writing the preliminary user message. That is
behavior in the CJS layer, and it duplicated resolvePersistedReasoningOverride,
which @librechat/api already exports and BaseClient already uses for the
same field.

Call the helper instead. A preliminary message is always a fresh turn,
so it is never an edit or a compaction. The two controller specs that
replace @librechat/api with a hand-built mock now take the real helper.

* fix: clear the file manager's state at the session boundary

The file manager's open flag and its opener live in Jotai's default
store, which outlives the authenticated route. A file manager left open
at logout reopened for the next account, still holding a ref to the
previous session's unmounted opener. Reset both in useClearStates, next
to the other composer state this branch already clears there.

* fix: decline interrupt-and-steer on the first turn while reasoning is staged

* test: read the phone layout from the viewport and keep the drawer open

* fix: name both reasons a queued message cannot be edited yet

* test: reuse a published file through the composer palette

* style: move the composer onto the design-system rules

* style: move the file row's new layout styles into classes

* fix: keep the composer's reused thumbnails, focus ring and disabled dimming intact

* fix: bring a steer back when its deferred cancel does not remove it

* fix: treat a run as over only when every pane holding it is idle

* refactor: pass enter-to-send into the composer from its host

* feat: make the palette's recent-file count configurable

* fix: let a file already in the composer be reattached at the limit

* test: hide the chip before replaying a deferred cancel

* refactor: restore the resume request through one packages/api facade

* fix: keep a conversation's staged reasoning across a visit to another model

* fix: keep a connected MCP server's variables configurable from the palette

* fix: resolve every pane-scoped shortcut through the pane holding focus

* refactor: keep every reasoning-override decision out of the /api wiring

* perf: skip paging the skill catalog for an agent that can show no skill

* test: type the reasoning snapshot the override helper records

* test: pin the applied budget and the untouched parameters in the override specs

* style: give the stop button the send button's shared submit recipe

* fix: name only the newline key when no chord sends

* fix: name the stop key only once the stop control can act

* fix: clear every staged reasoning selection at sign-out

* fix: stop advertising send-now while staged reasoning forces a queue

* fix: resolve the reasoning target of an ephemeral agent in the composer and the request

* test: type the staged reasoning the sign-out spec seeds

* fix: check the composer will take a queued row before discarding its parked copy

* style: carry the composer's classes onto Tailwind v4

* style: sort the composer's classes in Tailwind v4 order

* fix: return a refused queued row to the durable queue instead of memory

* i18n: localize the whole attachment-limit message

* refactor: validate the reasoning override inside the packages/api facade

* style: give the composer's submit slot its own IconButton variant

* test: pin the submit slot to IconButton's submit variant

* test: stub the seen-state hook the rebased MessagesView now calls

* fix: turn file search on for an embedded file attached through the shared hook

* chore: record the composer's design-rule counts against the rebased base

* fix: keep the composer's container query inert under Tailwind v4

* test: point the v4 focus scenarios at the redesigned composer's controls

* fix: keep a re-queued row on its own lineage after the run ends

* test: assert a re-queued row carries its lineage

* refactor: pass the dictation preferences into the composer from its host

* test: read the usage atom from the store the ask hook renders under

* fix: read a pending steer's pause from its own pane's branch

* a11y: keep focus on the quote chips until the last one is removed

* fix: keep staged reasoning for the next message across a manual compaction

* test: expect focus on the remaining quote chip after removing one of two

* style: sort the imports the canary rebase merged

* refactor: remove the side-panel files table the account menu replaced

* fix: settle a retried steer's late ack against a racing cancel

A cancel clicked while the retry POST was in flight left the optimistic
marker unread: the ack re-minted a live chip and injected words the user
had asked back, and a failed retry after that cancel lost them entirely,
because the failure path maps chips the optimistic cancel had removed.
The ack path now defers to the marker the way the composer's does, the
failure path re-adds the chip before settling it, and the queued row a
late ack produces binds its recovery identity only under protocol v2,
the same default the composer's degradation path trusts.

* fix: strip a replayed reasoning override that no longer validates on resume

A paused turn's reasoning override is replayed onto its resume request
and then validated as if it were fresh client input, so an operator
changing the endpoint's reasoning config between pause and approve made
the checkpoint permanently unresumable with a 400. The replay now marks
the request, and a replayed override that fails validation is stripped
so the resume proceeds on defaults; a fresh override that fails is still
rejected.

* refactor: derive the enforced spec fields inside the modelSpecs facade

The set of keys an enforced model spec locks is the input contract of
the reasoning-override refusal rule, and deriving it in the route
middleware left the policy untestable from packages/api and free to
diverge from the facade that owns the rest of the preset application.
applyModelSpecPreset now returns the enforced fields beside the private
ones it applies, and the middleware consumes them.

* fix: restore the pre-override parameters when a resume degrades

Stripping a replayed reasoning override that no longer validates removed
only the metadata field, while the resume had already replayed the paused
turn's model parameters with the override applied into the endpoint
option, so the provider still received what validation had refused. The
strip now inverts the application from the replayed base: the recorded
pre-override value returns and the thinking flag the override forced on
comes off.

* fix: resolve the effective responses route for upload targets

The attach target returned the conversation or agent's saved
useResponsesApi flag, so an Azure chat auto-routed to the Responses API
by responsesApiRouting preflighted its uploads against the chat
completions route and lost provider-document uploads. The resolver now
applies resolveEffectiveUseResponsesApi with the endpoint, model,
web-search state and routing policy, the same resolution the during-run
upload target makes.

* fix: name the interrupt chord in the hint only while it still interrupts

The during-run hint advertised Alt+Enter Interrupt & send even when the
chord had been claimed: a submitMessage rebound to Alt+Enter, a chord
yielded to a global shortcut, or shortcuts disabled altogether each make
the keydown resolver return something else. The hint now reads the same
resolver verdict the during-run send button does and omits the chord it
no longer owns.

* test: assert the unseen count the scenario adds rather than a zero baseline

The tab title counts every unseen conversation, and a shared
verification run leaves legitimately-unseen conversations from earlier
scenarios behind, so the three title-count scenarios failed on other
scenarios' leftovers instead of the behavior they seed. Each now reads
its baseline and asserts the count it adds and clears.

* test: hold the unseen count to its direction on the mobile project

The away poll that lights a dot also discovers conversations the client
had not counted yet, so on the mobile project the title count can jump
past any baseline taken mid-test. Opening still has to clear this
conversation's contribution and an away reply still has to raise the
count, which is what the assertions now hold.

* test: enable the MCP Apps server through the redesigned palette

The MCP Apps integration spec still drove the old composer's MCP
Servers menu, which the redesign replaced with toggleable rows in the
attach-and-tools palette; the lane failed on the missing menu on every
run. The helper now opens the palette, waits for the server wave, and
flips the row's pressed state.

* fix: preserve numeric reasoning defaults on blur

* test: reopen persisted code cards before checking highlights

* test: follow the current file manager navigation

* fix: offer only reasoning options a request override can carry

* fix: keep the favorited copy when skills share a name

* fix: restore composer focus by the interaction that closed the palette

* test: expect a keyboard close to refocus the composer on every project

* chore: re-record the design-rule backlog on the rebased composer

* chore: drop the files table and label the rebase brought back

* test: tolerate other projects' unseen conversations when marking one unread

* test: open the file table header scenario from the account menu

* fix: let the stop shortcut reach a hidden stop control from a dialog

* fix: keep staged reasoning out of tool-limit recovery and MCP App turns

* fix: degrade a queued turn's stale reasoning override instead of dead-lettering it

* fix: refresh the attach target when web search changes the route

* fix: re-arm the run-end drain after moving a steer to the queue

* test: hover the MCP row before reaching its revealed Connect action

* test: pick the parts scenario's MCP server from the tools palette

* fix: cancel a recovered row's parked steer only after the composer takes it

A recovered row's only durable copy is the steer parked on the server, and
once its run has ended nothing can re-create it, so a composer that refused
the row after the cancel left the words in memory until the next reload.
The composer now takes the words first; a cancel that then fails holds the
row out of the run-end drain instead of sending it a second time.

* fix: name only the idle composer affordances that act

* fix: keep steer escalation off until the pane owns the rendered conversation

The branch tail that decides whether a run is paused is keyed by the pane's
conversation, which can still name the previous chat while a cached one
renders. Until the pane catches up, a pause on any cached branch now keeps
Escalate disabled instead of reading a missing or foreign tail as unpaused.

* fix: route the host's Enter and auto-send preferences into the composer's nested hooks

ChatForm takes enterToSend and autoSendText from its host, but useTextarea,
DuringRunSendButton and both speech engines still read the store atoms, so a
host whose values differ advertised one behavior while the handlers ran
another. Pass the host values through instead.

* fix: settle a browser dictation take only after its final transcript lands

stopListening resolves on the recognizer's end event, after the final result
reached useSpeechRecognition's reducer but before React committed it, so the
take settled while the composer still held the interim text. Settle from an
effect declared after the transcript effect, off a state update queued behind
the result.

* fix: drop a restored reasoning override the current setting no longer offers

The staged override was only checked for a matching parameter key, so a value
restored after a capability change on the same target stayed staged, blocked
live steering, and was dropped at send time. Validate it against the setting's
options and range.

* test: scan the attach and tools palette for accessibility

The a11y suite still opened the removed tools dropdown. Open the palette
instead, and add a scenario that runs Axe over the open palette.

* fix: make the palette's virtualized list the grid that owns its rows

The rows sat under a wrapper grid behind react-virtualized's scroller, which
always renders aria-label and aria-readonly and so could not be made
presentational. Axe reported both the disallowed attribute and the grid's
invalid child. The scroller is now the labelled grid, with a rowgroup holding
the rows.

* fix: state the palette grid's row total and each row's position

Only the rows in view are mounted, so without aria-rowcount and aria-rowindex
assistive technology reads the rendered window as the whole grid.

* fix: key an empty destination's pending steers to the route

An empty tree has no message naming its conversation, so the pending-steer
surface fell back to the chat context, which still names the source chat
during warm navigation, and its controls acted on the source run. Fall back
to the chat's messages key, which follows the route.

* fix: keep assistant uploads available when the file config fails

Assistants upload directly and never needed the destination policy, which is
why the replaced AttachFileChat rendered their uploader outside the policy
gate. Restore that exception so a failed or paused file config request does
not remove every attachment entry.

(cherry picked from commit f568a4ba30)

Original-PR: https://github.com/LibreChat-AI/LibreChat/pull/14473
2026-10-01 03:46:18 +00:00
Marco Beretta
b0783e8085 🔑 fix: Invalidate the Auth User Cache After a CLI Password Reset (#16257)
* fix: invalidate the auth user cache after a CLI password reset

The reset CLI wrote the new password and credentialsChangedAt with
User.updateOne, which bypasses the auth user document cache
invalidation the data-schemas user methods perform. With
AUTH_USER_CACHE_MODE=on a cached document without the new stamp could
keep pre-reset access tokens verifying for up to the cache TTL after
the reset. Route the write through updateUser with the shared cache
factory, the same wiring config/delete-user.js uses, so the cached
entries for the user are deleted as part of the reset.

* test: cover CLI password reset token revocation in the mock harness

* test: verify CLI reset evicts the shared Redis auth cache

* fix: preserve pending account expiry during CLI resets

* test: assert pending account verification after CLI reset

(cherry picked from commit 6f903fc9e5)

Original-PR: https://github.com/LibreChat-AI/LibreChat/pull/16257
2026-10-01 03:46:17 +00:00
lia-by-librechat[bot]
55f3187296 🥢 fix: Fit MCP App Snapshots Within Message Documents (#16299)
* 🥢 fix: Fit MCP App Snapshots Within Message Documents

* 🥢 fix: Resolve Message Model Only at App Write Time

* 🥢 style: Sort MCP App Snapshot Persistence Imports

* 🥢 fix: Centralize MCP App Message Admission Across Writers

* 🥢 fix: Inject the App Budget Reader Without a Config Cycle

* test: Cover App Budget Initialization Across Mocked Hosts

---------

Co-authored-by: Lia <lia@librechat.ai>
(cherry picked from commit 411bb5444f)

Original-PR: https://github.com/LibreChat-AI/LibreChat/pull/16299
2026-10-01 03:46:17 +00:00
Marco Beretta
4617302116 🔑 feat: Passkey Authentication with WebAuthn Sign-In and Management (#14715)
* feat: passkey authentication with WebAuthn sign-in and management

* refactor: tighten passkey account protections

Require password confirmation before adding or removing a passkey, and rate
limit those confirmations per user in their own namespace. Restrict passkey
enrollment and sign-in to local accounts, and re-evaluate account status once
the credential owner is resolved. Require user verification on the sign-in
ceremony. Invalidate outstanding access tokens on password reset. Tighten
redirect target validation and sanitize ceremony failure logs.

Adds coverage across the passkey controller, the WebAuthn helpers, the step-up
limiter, both JWT strategies, the redirect utility and the passkey settings UI.

* fix: localize passkey default name and respect router basename

Send a localized default credential label at enrollment so non-English
locales no longer persist the server's English fallback, and route the
passkey post-login and 2FA navigations through the deployment base path
so subdirectory installs stay inside the router.

Also mock syncStaticTools in the server boot specs; without it the
post-listen MCP initialization calls undefined and exits the worker.

* fix: harden passkey counter checks and reach stranded credentials

Make the signature counter write a compare-and-swap so two assertions
that verified against the same stored counter cannot both commit, and
reject the losing one; a cloned authenticator is otherwise accepted
inside that window. Counterless authenticators keep restamping at 0.

Keep the passkey panel reachable for identity-provider accounts, which
already get passwordless removal server-side but had no way in, while
hiding the add control they cannot use. Cascade passkey deletion from
the admin user-deletion path so credential IDs are not stranded.

* fix: reject stale 2FA temp tokens and surface passkey load failures

A temporary 2FA token proves only the first factor, so honouring one
minted before a password reset let anyone holding the second factor
survive account recovery. Apply the same credentials-changed cutoff the
JWT strategy uses.

Also distinguish a failed passkey query from an empty list; the error
state previously rendered as "no passkeys yet", which misreports what
is protecting the account.

* fix: fail closed when the passkey counter write cannot commit

Returning success on a storage error bypassed the compare-and-swap it
exists to enforce, so a cloned authenticator could be accepted while the
stored counter stayed stale. A database that cannot take this write
cannot mint the session either, so little is gained by failing open.
Counterless credentials carry no clone signal and keep their old path.

* fix: keep the passkey 2FA handoff inside the router

A full-page navigation to the 2FA screen remounted AuthContextProvider,
which silent-refreshes on mount; no refresh session exists yet at that
point, so it bounced back to login before the code could be entered.
Route through the router like the password flow does.

Also reveal the removal step-up field when the server refuses a
passwordless attempt, so an account migrated to an identity provider
without its old hash cleared can still clear its credentials; drop the
staged password when the dialog closes; and revoke passkeys in the CLI
password reset, which bypassed the HTTP controller's cleanup.

* fix: apply one email-verification policy across login methods

Accounts predating mandatory verification are marked verified as they
sign in when no email is configured, but only the password strategy did
it, so the same legacy account was admitted by password and refused by
passkey. Extract the rule into a shared helper and call it from both.

* fix: refuse a passkey sign-in when the ban check cannot complete

checkBan defaults next to a no-op, so calling it bare discarded any
internal failure and left req.banned unset, letting the request fall
through and be issued tokens. Capture the error the middleware chain
would have propagated and reject instead.

* fix: refine passkey management dialogs

* test: stabilize passkey tooltip assertions

* test: reconcile passkey specs with the canary reset and deletion flows

The credentialsChangedAt stamp now lives in commitPasswordReset, so its
assertion moves to the passwordReset unit spec. Restores the describe
blocks the rebase merged together.

* test: mock passkey routes in the cross-site login suite

* refactor: move passkey handlers and credential checks into packages/api

The passkey controller, the credentialsChangedAt predicate and the legacy
email-verification grandfathering now live in TypeScript with injected
dependencies; /api keeps only the wiring that builds the handlers.

* style: align passkey UI with Tailwind v4 order and design-rule lint

* fix: revoke earlier access tokens on a CLI password reset

passwordVersion is not a user schema field, so the write was dropped and
tokens issued before the reset kept verifying.

* test: cover passkey sign-in and credential revocation end to end

* fix: hide passkey sign-in when email login is disabled

The passkey endpoints already refuse under ALLOW_EMAIL_LOGIN=false, so
advertising the button only led to a 403.

* fix: let only one concurrent verification consume a passkey challenge

Without a native getDel the challenge was read then deleted, so two
concurrent verifications could both use it. The delete now arbitrates:
only the caller whose delete removed the key gets the challenge.

* test: run passkey ceremonies on the localhost relying party

WebAuthn refuses an IP address as the RP ID, and the harness serves the
app on 127.0.0.1. The reset scenario runs once per project, so the mock
harness lifts the per-IP reset request limit as it does for forks.

* fix: keep passkey management when email login is disabled

Gate only the sign-in button on emailLoginEnabled, since only the sign-in
routes sit behind that gate; list, rename and remove stay reachable.

* fix: build the passkey credential index before the first registration

With MONGO_AUTO_INDEX=false the unique credentialId index was never
created, so concurrent registrations could store the same credential.

* test: enable password reset in the mock harness and cover email-login-off passkeys

* fix: stop re-exporting the passkey cap from data-schemas

The bundler left every import of methods/passkey.ts unlinked while that
module re-exported MAX_PASSKEYS_PER_USER from librechat-data-provider, so
the index build and the counter-failure log threw ReferenceError at
runtime. Consumers now import the cap from librechat-data-provider.

* test: close nested settings dialogs before logging out in passkey scenarios

* fix: reject cross-site passkey assertions

The verify route accepted a form posted from another site, which let an
attacker sign a victim into the attacker's account. It now runs the same
requireSameOrigin guard as password login and the 2FA step.

* fix: end refresh sessions on a CLI password reset

A surviving refresh session could mint an access token issued after the
credentialsChangedAt stamp, so the CLI now deletes sessions like the web
reset does.

* fix: keep the passkey button locked when it supersedes autofill

Starting the modal ceremony aborts the pending autofill one, and the
autofill error path cleared the shared lock and busy state while the
modal ceremony was still running.

(cherry picked from commit 471abaa64d)

Original-PR: https://github.com/LibreChat-AI/LibreChat/pull/14715
2026-10-01 03:46:16 +00:00
Danny Avila
8ec632ff0e
📡 fix: Explain Model Streams That Die Mid-Response (#16415)
* 📡 fix: Explain Model Streams That Die Mid-Response

* 📡 fix: Bound Model Stream Failures to Response Bodies

---------

Co-authored-by: Lia <lia@librechat.ai>
2026-09-27 15:21:20 -04:00
Danny Avila
f13b0eaef4
🚚 chore: Point Repository and Image References at LibreChat-AI (#16005)
Some checks failed
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Docker Dev Images Build / publish (push) Waiting to run
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
Publish `@librechat/client` to NPM / pack (push) Has been cancelled
Publish `librechat-data-provider` to NPM / pack (push) Has been cancelled
Publish `@librechat/data-schemas` to NPM / pack (push) Has been cancelled
Publish `@librechat/client` to NPM / publish-npm (push) Has been cancelled
Publish `librechat-data-provider` to NPM / publish-npm (push) Has been cancelled
Publish `@librechat/data-schemas` to NPM / publish-npm (push) Has been cancelled
Update package repository and bugs URLs, GitHub links in docs, templates and comments, and image references (compose files, deployed-update, Helm values and renovate hints) from danny-avila to LibreChat-AI / librechat-ai. Covers the rag-api repository and its images, which moved to the organization. Sponsor links and the sealed star-history chart are unchanged.
2026-09-23 23:08:36 -04:00
Marco Beretta
f077e8c967
🗯️ fix: Give Failed Turns Actionable Guidance (#15896)
* 🧯 refactor: State Failed Turns in Copy a Reader Can Act On

Every client-facing error code now has authored copy, so no failed turn renders
raw JSON, an unresolved `com_error_*` key, or a provider message cut at 512
characters. `client/src/components/Messages/Content/Error.tsx` becomes a module
directory: a dispatcher resolving `json.code ?? json.type`, a registry splitting
one-sentence copy from payload-aware renderers, and shared parts that resolve the
provider, model, user-key ownership and compaction capability behind the failure.

Message-limit violations persist `resetAt`/`retryAfterSeconds`, so the row counts
down to the window's reset instead of naming a window length alone.

* 📸 chore: pin before/after error copy screenshots for the pull request

* 🧹 chore: drop the pinned pull request screenshots from the tree

* 🔻 style: Trail the error disclosure chevron after its label

* ⏱️ refactor: Derive Limiter Resets in packages/api, Stamp Key Expiry as ISO

The message limiter's reset fallback and retry arithmetic move out of the
CJS middleware into `getRateLimitReset` in packages/api, which both the
message and agent-event handlers now call, so /api keeps only the wiring.

`checkUserKeyExpiry` persists `expiredAt` with `toISOString()` instead of
the server's `toLocaleString()`, which a client in another locale could
parse with day and month swapped.

* 🪪 fix: Attribute Failed Turns to the Row, Agent and Payload That Failed

Error content parts render through `Part` without their message, so their
copy named the conversation's current endpoint and model rather than the
ones that failed. The chat, flat-thread and search row renderers now supply
the row's identity through `ErrorSourceProvider`, and the dispatcher reads
it whenever no row is passed directly.

A saved agent's row carries the `agents` endpoint and the agent id as its
model, so identity, key ownership and the key dialog resolve against the
agent's own provider and model; an endpoint named by the payload (an
expired key) outranks both. The agent builder is only suggested where the
side panel would offer it: agents configured, builder not disabled, USE and
CREATE held, and an EDIT grant or administrator role.

Also in the renderers:
- structured refusals read Anthropic's `stop_details.explanation`
- provider bodies nesting their message under `error` keep that message
- upstream model failures name the provider when it is known
- `INVALID_USER_KEY` offers the key dialog on a user-provided endpoint
- an oversized message is never told to compact the conversation
- the countdown is bounded by the limiter's window, not a fixed day, and its
  ticking text is hidden from the assertive live region
- an expiry persisted in a server locale is shown as written
- search rows that failed render through the dispatcher, not as markdown
- the payload type is an explicit JSON value type

* 🧪 chore: Seed Real Refusal, Legacy Expiry and Nested Provider Error Shapes

The gallery's refusal case used a string `info` no producer writes; it now
carries Anthropic's stop metadata and a Bedrock content filter. It also
seeds an expiry stamped in a server locale, an unreadable key on a
user-provided endpoint, and a provider body nesting its message under
`error`, and removes whatever it inserted when a write fails.

* 🧵 fix: Keep the Reason From a Provider Body Embedded in Failed-Run Text

A failed run persists a provider error as `<base message>: <SDK message>`,
and Anthropic's SDK message embeds the response body, so the persisted text
reads `…request: 400 {"type":"error","error":{"message":…}}`. The
fallback renderer dropped that body and kept only the prefix, losing the
reason. It now swaps the body for the message it carries.

* 🔀 fix: Resolve Handoff Agents, URL-Owned Credentials and Nested Error Codes

An error part after an `AGENT_UPDATE` belongs to the agent the run was
handed to. `ErrorSourceProvider` now records where a row's handoffs sit,
and an error part resolves the agent active at its own `partIndex`; search
and shared rows give error parts that position too. A handoff the agents
map cannot resolve names no provider instead of the agent that handed off.

A user-provided base URL makes the reader's own record hold the key as
well, so key failures on such an endpoint offer the key dialog. `NO_BASE_URL`
and `INVALID_BASE_URL`, which only a user-provided URL produces, render
through the same renderer with named copy and an add or update action.

OpenAI-compatible bodies nest `code` and `type` under `error`; when the top
level names neither, the dispatcher reads the envelope, so a nested
`invalid_api_key` reaches the key renderer.

* 🕰️ fix: Keep Unknown Key Ownership Generic, Measure Resets From the Row

A shared link's viewer has no endpoint configuration, so whether the reader
or the deployment owns a key is unknown; key copy there read that as
deployment-owned and told the viewer to ask an administrator. Ownership is
now undefined without a configuration, and the copy stays generic.

The countdown's window bound used this device's clock, so a device running
behind the server could drop a valid reset. The window now runs from the
row's own timestamp, which the server stamped alongside `resetAt`.

* 🗜️ fix: Suggest Compaction Only Where the Compact Action Is Offered

Manual compaction is only reachable from the context usage popover, which
`interface.contextUsage: false` removes. Context errors on such a deployment
suggested compacting anyway; they now fall back to the standard next steps.

* 🧭 fix: Keep Row Identity Authoritative and Every Generation Charge Visible

A row's own endpoint and model now stand even where it lacks them, so a
restored failure no longer borrows whatever the conversation selected since;
only an error rendered with no row reads the conversation, and an agents row
with no stored agent id (the live row of a turn that just failed) still
resolves the conversation's agent.

Also:
- a token-balance row lists every generation instead of the first 20
- search rows re-render when their `error` flag changes
- `AUTH_CROSS_ORIGIN`, added since this branch began, renders the login
  page's copy and has a gallery case, so the seed's completeness check runs;
  a table test now fails for any enum member without authored copy
- upstream model failures keep the provider-neutral sentence the server
  persists as their text, which the e2e suite reads the row by

* 🪞 fix: Resolve Ephemeral Agent Ids and Keep an Agents Row's Model Name

An agents row whose model is a model name, as an ephemeral agent's is,
lost both its provider and its model whenever no saved agent resolved.
Agent ids now resolve the way sibling headers resolve them: a saved agent
through the map without the parallel `____N` suffix, an ephemeral agent
through the endpoint and model its id encodes. A row that still resolves
nothing keeps the model name it recorded; an agent id is never shown.

* 🛤️ fix: Attribute Lane Errors to Their Agent, Count Live Retries Relatively

A parallel run stamps each part with the agent of its lane, and an error
part in one lane was attributed to whichever handoff last preceded it. The
row now records error parts' own agents alongside its handoff positions,
and an error part's own agent comes first.

A live countdown subtracted this device's clock from the server's
`resetAt`, so a device running fast could report the limit already over.
`retryAfterSeconds` is now counted from the row's own timestamp, which the
device stamps on a live row and the server on a restored one.

Search results read a cached startup config but have no compact control, so
compaction guidance now also requires the chat surface.

* 🧷 fix: Resolve an Agents Row From Its Own Model, Never the Current Agent

An agents row that stored a model name, as an ephemeral agent's does, fell
back to the conversation's current agent when no saved agent resolved, so
switching the conversation to a saved agent later rewrote who produced the
old failure. The fallback served no real row: the client's live placeholder
for an agents turn stores the conversation's agent id as `model`, as
persisted rows do. The row's `model` now decides on its own — a saved or
encoded ephemeral id resolves, a model name is kept, and the `ephemeral`
placeholder names nothing; only an error with no row reads the conversation.

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-09-14 10:37:22 -04:00
JOJO
7fe9a45894
🧵 ci: Keep a Floor of Two Jest Workers in CI (#15819)
Some checks are pending
Docker Dev Images Build / publish (push) Waiting to run
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
`maxWorkers: '50%'` resolves to a single worker on the 2-vCPU GitHub-hosted
runners that private repositories get, and Jest then runs in band. Any test
file that leaves a handle open (a listening `http.Server`, a timer, a DB
connection) keeps the main process alive after "Ran all test suites" until the
job's `timeout-minutes` cancels it, so a run with every test passing still
fails. Public runners have 4 vCPUs, where `'50%'` is already two workers and
the leaked handle dies with the worker process.

Move the worker count into `config/jest.workers.cjs`, shared by all six
workspace configs: local runs keep `'50%'`, CI uses `max(2, floor(cpus / 2))`.
On 4-vCPU runners the value is unchanged; on 2-vCPU runners Jest now uses
worker processes the same way it does upstream.

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-09-14 00:32:20 -04:00
Danny Avila
696bcb6c1e
🏦 fix: Reserve Credits for In-Flight Requests (#15879)
* 💳 fix: Reserve Balance for In-Flight Requests

Admit each balance-checked request against the credits that other
in-flight requests have not already reserved, and hold its prompt cost
on the balance record until the turn settles. Apply a due auto-refill in
the same fenced write so one refill window credits exactly once.

- data-schemas: reserveBalance / releaseBalanceReservation replace
  createAutoRefillTransaction; the write is a compare-and-swap on
  tokenCredits, reservations and lastRefill (no pipeline updates)
- balance.reservations is select:false so existing reads never see it
- checkBalance returns a reservation handle; BaseClient and the
  assistants controllers release it once the turn settles
- balance.reservationTtlMs bounds how long an unreleased reservation
  counts (default 30 minutes)

* 🧮 refactor: Admit Reservations Without Contention and Record Refills Durably

- admission is a conditional $push/$inc against a reservedCredits total,
  so concurrent admissions of a funded balance commit on their first write
- expired reservations are pruned per element with their credits
- auto-refill is fenced on every setting its decision reads and leaves a
  pendingRefill ledger marker in the same write; any later read records the
  transaction under the fixed id and clears the marker
- a missing record is created with $setOnInsert inside reserveBalance
- reservation lifecycle moves to packages/api (createBalanceReservations,
  withBalanceReservations); BaseClient releases once usage is recorded

* 🪪 fix: Resolve Each User to One Balance Record

- balance records a creator inserts are keyed by the user id, so creators
  racing on a missing record converge on one document; every balance read
  and write resolves a user to their oldest record, so existing duplicates
  stay inert
- upsertBalanceFields takes insert-only fields; the balance-config
  middleware, schedules and lazy initialization pass the starting credit
  through them so it never overwrites a record another writer created
- the auto-refill write is also fenced on reservedCredits
- the scheduled-chat balance pre-skip counts reserved credits as unavailable

* ⏳ fix: Count Only Unexpired Reservations in Scheduled Balance Checks

findBalanceByUser with includeReservedCredits now totals the reservations
that have not expired, so a reservation left by a crashed request stops
pre-skipping scheduled chats at its expiry instead of when a later
reservation write prunes it.

* 🔁 fix: Renew Balance Reservations While Their Request Runs

A reservation is renewed every half TTL until it is released, so only a
reservation whose process stopped expires. Pruning removes a reservation
only while it is still expired at the write, so one renewed after the
pruning read survives.

* 🛟 fix: Keep Reservations Alive Through Renewal Failures and Lost Acks

- renewal reschedules itself: every half TTL, capped to the timer range,
  and a failed renewal retries within a tenth of that window
- an admission write that errors removes its possibly committed
  reservation before rethrowing

* 🛑 fix: Hold a Stopped Turn's Reservation Until Its Charge Lands

- an aborted turn lets its reservations lapse after a minute (or the TTL)
  instead of releasing them, so the Stop request's charge lands before the
  credits are admitted again
- balance.reservationTtlMs has a 10 second minimum, enforced in the schema
  and at runtime, so renewal stays bounded

* ↩️ revert: Release a Stopped Turn's Reservation When the Turn Settles

An aborted turn releases its reservations when it settles, the same as
every other exit, instead of holding them for a fixed lapse window that
refused funded requests after every Stop. The minimum reservation TTL
stays.

* ⏸️ fix: Hold Assistants Reservations Through Background Run Retries

An in-progress Assistants run continues in the background after the
handler responds; the turn's reservations now stay held until that run
settles (createBalanceReservations().holdUntil).

* 🔁 fix: Apply an Auto-Refill Once per Admission

A refill interval that is due again the moment it is applied, such as a zero
interval, let one admission refill repeatedly until the retry bound. Each
admission now applies at most one refill, as the previous check did.

* 🧹 fix: Prune Expired Reservations in One Write

An admission that found expired reservations issued one write per hold, so a
backlog left by a stopped process fanned out into as many concurrent writes.
The prune is now a single write fenced on every hold it removes; a hold renewed
after the read makes the write miss and the admission re-read.

* ⏱️ fix: Renew Reservations From Their Stored Expiry

The next renewal was scheduled half a TTL after the previous write settled,
and the first one half a TTL after the admission returned, so a slow write
could let a live hold expire before its renewal. Renewals are now scheduled
from the expiry the last write stored.

Expired holds are pruned as one bulk write of per-hold fenced updates, so a
hold that changed after the read is skipped while the rest are removed, and
the admission no longer spends a retry on the whole prune.
2026-09-13 12:53:19 -04:00
Danny Avila
dad20b115e
🛗 fix: Add Tenant Index Upgrade Migration (#15767)
* fix: expose tenant index upgrade migration

* fix: retry tenant migration index builds

* docs: run tenant migration from the image root
2026-09-09 00:10:40 -04:00
Danny Avila
164f146bdf
🛞 feat: Manage Personal Code Environment Lifecycles (#15369)
* feat: add self-service code worker pairing

* fix: enforce code worker principal boundaries

* fix: separate pairing control planes from execution

* fix: preserve principal worker routing

* fix: pin artifact traffic to code workers

* fix: authenticate code worker selection

* test: mock worker-aware artifact headers

* fix: harden self-service code worker routing

* fix: fail closed for principal code pairing

* feat: manage personal code environments

* fix: harden personal code environment lifecycle

* fix: satisfy backend lifecycle types

* fix: harden principal environment lifecycle and quotas

* fix: fence environment lifecycle races

* fix: reconcile code environment stack

* style: format code execution headers

* fix: reconcile code environment lifecycle

* fix: close self-service pairing gaps

* fix: close code environment reconciliation gaps

* fix: close code environment UI review findings

* fix: satisfy code environment type checks

* fix: route code priming to selected worker

* fix: harden code environment lifecycle fences

* fix: type lease renewal interval

* fix: harden environment lifecycle recovery

* fix: close lifecycle recovery races

* fix: type compensated agent timestamps

* fix: fence lifecycle cleanup by tenant and revision

* fix: scope environment revision fencing
2026-09-01 21:10:05 -04:00
Danny Avila
dc77b78d3e
🔇 refactor: Quiet Framework Logging in Unit Tests (#15363)
* 🔇 refactor: Quiet Framework Logging in Unit Tests

Unit test output was overwhelmingly framework log lines rather than test
results. A `packages/api` run printed 9,076 lines for 434 suites; 1,596 of
them were the identical `at Console.log (winston/transports/console.js:87)`
frame that Jest staples onto every winston write.

Three causes:

- Each winston logger built its Console transport with a hard-coded
  `level: 'info'`. Winston resolves a transport's explicit level ahead of
  its parent's, so the logger's own `NODE_ENV`-derived `warn` never applied
  — in tests or in production. `CONSOLE_LOG_LEVEL` now drives it, defaulting
  to the previous `info` so deployments are unaffected. `meiliLogger` had
  the same hard-coded level and is routed through the same helper.
- `LOG_TO_FILE` defaults on, so every Jest worker opened DailyRotateFile
  transports. A run left `packages/api/logs/` holding a 107KB error log, a
  gzipped rotation, audit JSON, and — from fake-timer suites — files dated
  1969 and 2023.
- Client suites carried leftover debug `console.log` in shipped source
  (`Test mode. Skipping silent refresh.` alone accounted for 109 blocks)
  plus three test-harness defects whose React error dumps buried everything
  else: a markdown suite rendering Mermaid with no Router, a `~/Providers`
  mock missing `useSearchContext` (the component's own catch swallowed the
  throw, so the sources path was dead while the suite passed), and
  `getBoundingClientRect` stubs without `left`, which made a computed
  `right` NaN.

A shared `config/jest.setup.logging.cjs` sets the two env vars for both
backend workspaces. It sets env only and requires nothing: eagerly requiring
the logger froze `CREDS_KEY` into `encryptV3` before specs could set it and
broke five suites. `TEST_VERBOSE_LOGS=true` restores the logs for debugging.

Output per full run:

  packages/api           9,076 -> 770 lines   (winston lines 1,596 -> 0)
  packages/data-schemas  2,521 -> 946 lines   (winston lines   290 -> 0)
  client                10,384 -> 3,153 lines (console blocks  285 -> 74)

Test counts are unchanged: client 476 suites / 5,827 tests green, and the
Ariakit `act(...)` warnings the selector suites emitted are gone rather than
suppressed.

* 🔧 fix: Settle Shutdown Timers Before Counting Them

`destroy()` finishes cancelling its fenced retirement timers on the tick
after it resolves. The count only returned to the baseline because a console
write from the logger happened to yield first — silencing that logging left
four timers still pending at the assertion, deterministically failing the
suite in CI's sharded, coverage-enabled run.

Advance the fake clock by zero before counting, so the assertion proves the
timers were cancelled rather than that something incidental yielded.

* 🛡️ fix: Validate CONSOLE_LOG_LEVEL Instead of Trusting It

Winston resolves a level name against the logger's level map, so a name
outside it resolves to `undefined` and the transport drops every message.
Passing `CONSOLE_LOG_LEVEL` straight through meant a typo (`warning`) or a
stray space silently muted all console output — a misconfiguration that reads
as a dead deployment.

Validate against the level set, normalize case and whitespace, and fall back
to the default with a warning on the way past, since the logger cannot report
its own misconfiguration.

The level map itself was duplicated verbatim in both loggers; it now lives in
`utils` beside the validation that depends on it.

* 🎚️ fix: Let CONSOLE_LOG_LEVEL Outrank DEBUG_CONSOLE

`DEBUG_CONSOLE=true` forced the console transport to `debug`, so on those
deployments every non-`silent` value of `CONSOLE_LOG_LEVEL` was discarded —
`CONSOLE_LOG_LEVEL=error` still emitted debug and info, defeating the control
it advertises.

`DEBUG_CONSOLE` now only moves the *default* level and keeps choosing the debug
format, so an explicit level stays in charge of verbosity. With no explicit
level the resolved default is still `debug`, unchanged for existing setups.

Covers the wiring with a spec that re-imports the logger per environment,
rather than only testing the resolver in isolation.
2026-08-30 15:26:30 -04:00
Danny Avila
d369c649ed
🪥 chore: Run CI's Static Checks on Each Commit's Diff (#15303)
* 🪝 chore: Run Static Checks on Every Commit

Adds `scripts/static-checks.mts`, a local port of the Static Checks CI job
(.github/workflows/static-checks.yml) scoped to the files in a diff. It
resolves the changed-file list, applies the same `dorny/paths-filter` groups
the job uses, and runs whichever checks those paths activate — ESLint,
Prettier, import order, ESLint config validation, package.json validation, and
(behind `--full`) config migration tests, unused i18n keys and depcheck. Like
the job, every selected check runs even after one fails and the failures are
summarized at the end.

The pre-commit hook keeps lint-staged for the per-file layer, which verifies
the exact staged content of partially staged files, then runs the script for
everything lint-staged cannot cover. lint-staged now uses the job's ESLint
invocation, so warnings fail locally the way they fail CI.

The slow gates stay opt-in (`npm run static-checks:full`, or
`STATIC_CHECKS_FULL=1`) to keep commit latency unchanged.

Hooks were never installed: `config/prepare.js` existed but no `prepare` script
called it, so the hook only ran where `core.hooksPath` had been set by hand.
Replaces it with an inline `prepare` — both Dockerfiles run `npm ci` after
copying only the manifests, so a `node config/prepare.js` step would fail the
image build, and husky is absent from `--omit=dev` installs.

The i18n scan is a single pass over the source identifiers rather than one grep
per key, verified to flag exactly the same keys as the CI loop (including the
substring and dynamic-key cases) in 0.5s instead of 14s.

* 🩹 fix: Address Codex Round 1 on the Static Checks Runner

Activate gates from the unfiltered changed-path list. `dorny/paths-filter`
matches deleted paths too, so gating on the `--diff-filter=ACMRTUXB` list the
per-file steps use let a delete-only commit — the last reference to a
translation key, say — slip past the i18n and depcheck gates. The two lists are
now derived separately, the way CI derives them.

Pass `-m` to `git diff-tree` in `--commit` mode. Without it a merge commit
emits no paths at all, so `--commit <merge-sha>` reported "Nothing to check";
a real merge in this repo's history goes from 0 to 97 files.

Build the workspaces the config suite imports instead of skipping when `dist`
is absent. The three `dist` directories are gitignored and `npm ci` does not
produce them, so a fresh checkout reported a pass for a gate that never ran —
and an existing `dist` could be stale. Each is a sub-second tsdown build.

Resolve a global depcheck through a shell on Windows, where npm exposes it as
`depcheck.cmd` and `spawnSync` cannot see the shim.

Skip dot directories when walking for imports. `.claude/worktrees/` can hold a
full checkout per branch — 117 on this machine — and the root-wide scan behind
the depcheck gate walked every one of them.

Records the remaining boundary in the header: the per-file checks see exact
staged content via lint-staged, while the tree-wide gates read the working
tree, as running them by hand would.

* 🩹 fix: Address Codex Round 2 on the Static Checks Runner

Treat an unresolvable checker as a failure. ESLint and Prettier missing meant
the runner printed "All affected static checks passed" without having linted
anything; only depcheck, which CI installs globally and this documents as
optional, may still skip.

Catch per-check exceptions. The runner promises that every selected check runs
even after a failure, but a throw — a malformed translation JSON, say —
escaped and cancelled the checks after it. Each is now recorded as that
check's failure; verified that depcheck still runs after i18n throws.

Restrict `--commit` to the checked-out commit. Paths came from the named
commit while contents came from the working tree, so an older revision was
scored against the wrong file contents: a file added then deleted vanished,
and one modified since was read at its newer contents. It now fails with a
pointer to `--against`.

Reject unknown options. `--ful` silently ran the fast tier and `--commmit HEAD`
treated `HEAD` as a file path, both exiting 0 and implying gates had run.

Cover the runner in CI. `scripts/**` was absent from the workflow's trigger
paths, so a PR touching only the script the pre-commit hook now depends on got
no Static Checks run — and ESLint has no flat-config match for
`scripts/**/*.mts`, so nothing else loads it either. Adds the trigger path, a
`runner` filter group and a step that runs the script against the PR's own diff.

* 🔗 feat: Add Circular Dependency and TypeScript Gates

Both already run in CI as jobs of the Backend Unit Tests workflow; this brings
them to the local runner so they land before a push rather than after.

Circular dependencies (`node config/circular-deps.mjs`) is fast enough at 0.9s
to sit in the per-commit tier, gated on the same paths that trigger the CI job.

TypeScript stays opt-in behind `--full`: the five projects cost between 2.3s
and 20.9s each, which is too much per commit. Each project declares the paths
that can affect it — its own sources plus its upstream packages — so an edit to
data-provider still typechecks data-schemas, api, packages/client and client,
while an `api/**`-only change runs none of them, since no typechecked project
includes that directory. The builds a project's imports resolve through are
made first, mirroring the CI jobs' dependency on the build artifacts, through a
helper the config suite now shares.

Also addresses codex round 3:

Reject conflicting target selectors. `--against origin/dev package.json`
silently checked only the file, and `--against <bad-ref> --commit HEAD` never
resolved the bad base, so a caller could believe a range had been checked.

Require a clean worktree in `--commit` mode. The HEAD-only restriction was not
enough: contents still come from the working tree, so an uncommitted edit was
scored against the named commit — an invalid uncommitted package.json failing a
valid HEAD, or an uncommitted fix masking a defect in it.

The summary now names how many checks were skipped rather than reporting a
bare pass, and a typecheck failure carries the stale-workspace-build hint —
inside a git worktree `librechat-data-provider` resolves to the main checkout,
whose dist can predate the branch and shows up as missing properties.

* 🩹 fix: Address Codex Round 4 on the Static Checks Runner

Diff `--against` from the merge base. A two-dot diff reports the base branch's
own commits in reverse once it advances, so `--against origin/dev` scored 64
files for a branch that changed 7, activating gates for files the branch never
touched. Three dots makes the documented PR-style command mean what it says.

Typecheck on root manifest changes. Both review workflows trigger their
TypeScript jobs on package.json and package-lock.json, because a dependency or
@types bump breaks compilation on its own; the local filter ignored them, so
`static-checks:full` passed where CI would fail.

Validate every workspace manifest. The list mirrored the four the CI step
happens to name, so a malformed packages/api, data-provider or data-schemas
manifest passed validation in the revision modes, which have no lint-staged
pass behind them. Both lists now cover all seven.

Make the runner smoke execute a check. `--list` never runs one, and a
script-only PR activates no group, so the CI coverage added for exactly that
case could pass with the execution path untouched. It now runs against an
explicit target.

* 🩹 fix: Address Codex Round 5 on the Static Checks Runner

Activate the JSON gate for every manifest it validates. Round 4 added the four
workspace manifests to the validation list but not to the filter that turns the
gate on, so a malformed packages/data-provider or data-schemas manifest still
passed when it was the only changed file — the list grew and the trigger did
not. The same two entries also feed the unused-package calculation, reached
through api/package.json's @librechat/data-schemas dependency.

Include the owning workflows in the imported gates' filters. Circular
dependencies and TypeScript come from the review workflows, both of which list
their own YAML in `on.paths` and therefore rerun those jobs when the workflow
changes; locally the gates stayed inactive, so a change to how they are built
or invoked could bypass the local equivalent. Added to the group filters and to
the per-project predicates, since a workflow-only change would otherwise
activate the group and then select no project.

Bound command batches by characters rather than file count. Windows caps a
command line at 32767 characters, far below POSIX ARG_MAX, and a count does not
bound that: 400 of this repository's longer paths already come to 30176
characters before the executable and fixed arguments. Verified that a list
spanning several batches still reports a defect in its final file.
2026-08-28 10:08:37 -04:00
Danny Avila
c5276fc63d
⏱️ feat: Run Scheduled Chats Through Durable Agent Triggers (#14939)
* feat: Scheduled Chats — agent-centric scheduled runs creating real conversations

Squash of the full review-hardened branch (PR 14540, supersedes 14373) onto
latest dev, preserving the exact verified tree. History prior to this commit
lived on the pre-squash branch; every invariant below survived 25 Codex review
rounds plus two external audit rounds (R26) with regression tests that fail
without their fixes.

Feature:
- Schedules CRUD + side-panel UI (cadence dialog, run cards, Run Now), roles/
  permissions (SCHEDULES:USE), interface.schedules availability, per-user limits
  and capacity slots, timezone-aware cadence with DST-conservative floors and
  misfire grace.
- Engine: single-process claim/fire loop with leases, loopback POST dispatch
  (signed schedule-fire JWT claims, per-occurrence idempotency key inside the
  route's clientRequestId charset), overlap/balance/capacity/duplicate skip
  policies with auto-disable streaks (too_many_failures, insufficient_balance),
  reconciliation from retained terminal-job evidence, erasure sweep.
- Scheduled runs create real conversations through the resumable agents chat
  path: HITL pauses surface on the card (requires_action), resumes re-apply the
  fire boundary's admission policy (revision fence, enabled, global kill switch,
  SCHEDULES:USE, availability) before continuing a billed generation.

Correctness invariants (the audit surface):
- Settlement discipline: every persistence-producing write happens-before a
  run's terminal outcome write; Stop/complete/pause race through single-winner
  terminal CAS claims (dev's TerminalJobClaim substrate) with retained,
  completedAt-less evidence for scheduled fires plus an owner-intended outcome
  stamp (scheduleOutcome) the reconciler prefers over re-derived success —
  round-tripped through the Redis hash mapper.
- Swallowed generation failures (client error content parts) classify to
  error/skipped_balance instead of success on both initial and resumed paths;
  stale stamps are refreshed evidence-first when persistence plus the Mongo
  outcome write both fail.
- Abort honesty: delivery judged by generation ownership and the CAS's actual
  from-status; republication escalates to the transport's acknowledged variant;
  the Stop route settles only genuinely paused runs.
- Account deletion: one-way barrier (deletionRequestedAt) with auth-cache
  tombstone-before-stamp, boundary rechecks across all auth strategies, quiesce
  of scheduled + interactive work with durable per-stream abort fences
  (positive-evidence acknowledgement only), owner-side finalization markers for
  post-terminal billed writes, deferred-deletion sweep (explicitly ensured
  partial index) that completes cascades autonomously. Remote OpenAI-compatible/
  Responses requests are documented as outside the quiesce and tracked in issue
  14594.
- Store compatibility: finalization markers optional on the legacy IJobStore
  contract with coherent degradation (registration fails -> synchronous title
  fallback; count reads 0).

* fix: fence terminal response persistence from deletion; retain stale-pause evidence

Two blockers from the third external review round.

Terminal persistence visible to account deletion:

- The finalization marker was registered only when post-terminal TITLE work was
  possible, but every persistence-owning terminal CAS opens the same window: the
  claim drops the job out of the active set BEFORE the response save (and the
  background user-message/convo saves), so a deletion quiesce landing there saw
  neither an active job nor a marker and could cascade while the admitted request
  could still recreate messages. Both controllers now register the marker before
  every persistence-owning claim — the fresh-turn path and the HITL resume — and
  release it only after their pending saves (and any post-terminal title) have
  landed, on success and failure paths alike. The TTL bounds a crash.
- settleAbortFence no longer clears a complete/error fence while
  `terminalPersistencePending` is set: terminal at the CAS is not settled while
  the owner is still persisting. The job facade now surfaces the flag.
- The marker trio is REQUIRED by the runtime store contract (assertJobStoreV2
  refuses a store without it at configure time, keeping the failure loud and
  deterministic) while remaining optional on the legacy public IJobStore type for
  source compatibility. The silent degrade path from the previous round is gone —
  it was not deletion-safe.

Stale-pause recovery retains scheduled evidence:

- Three crash/timeout recovery paths — ApprovalLifecycle.failStalePausePersistence
  and the InMemory/Redis stale-pause cleanups — unconditionally stamped
  `completedAt`, putting a scheduled fire's failed-pause error terminal on the
  short completed TTL. A Mongo outage longer than that TTL erased the evidence and
  the reconciler recovered the run as `interrupted` instead of `error`. All three
  now follow the controller-observed path from the previous round: scheduled jobs
  omit `completedAt` (retained-evidence TTL) and stamp the error outcome.

The PR description now explicitly narrows the deletion guarantee for the remote
OpenAI-compatible/Responses paths (tracked in issue 14594).

* fix: deletion-fence marker protocol — generation-scoped, atomic, fail-closed, all terminal paths

One consolidated pass over the finalization-marker mechanism, per the fourth
external review round. The invariant it establishes: NO persistence-owning
terminal CAS runs without a durable, generation-qualified marker covering the
window it opens, and every consumer treats a pending terminal as unsettled.

- Generation-scoped markers. Entries were keyed (userId, streamId), so a
  Stop-superseded generation finishing late could clear the marker its
  replacement registered on the same conversation. Marker fields are now
  qualified by the generation's createdAt; clears must present the same
  identity, and an unqualified legacy clear cannot drop a qualified entry.
- Atomic Redis registration. HSET-then-EXPIRE loses the fresh marker when the
  user's existing hash expires between the two commands (or the process dies
  there); registration is now a single Lua script carrying both.
- Fail closed everywhere. Registration failure (after one retry) now REFUSES
  the terminal CAS instead of proceeding uncovered: the completion claim throws
  into the error path, the error path skips completeJob and leaves the job
  ACTIVE — deletion-visible by itself, recovered by the stale-running reaper —
  and abortJob returns a new retryable `fence_unavailable` failure the Stop
  route answers with 503 and the deletion quiesce treats as an unacknowledged
  stop (fence kept). The previous round's log-and-proceed is gone.
- Every terminal path enrolled. abortJob now owns its window (register before
  the abort CAS, clear in its finally — the Stop route's checkpoint prune and
  partial save run inside beforePublish, between CAS and publication); the
  interactive and background generation-error paths register before their
  completeJob; the resume controller's error finalization registers before its
  completeJob. A lost or thrown claim releases the marker after pending saves
  flush instead of holding the user's deletion behind the TTL.
- Every pending terminal unsettled. settleAbortFence defers on
  terminalPersistencePending for ALL statuses — including `aborted`, whose
  route-side persistence the previous guard missed.

Also: a direct Redis regression for scheduled stale-pause retention (the P2
test gap), and the PR description no longer claims to carry every commit.

* fix: lease-token lifecycle fences — same-generation isolation, admission fence, undelivered-Stop retention, legacy abort enrollment

Fifth external review round; four P1s, handled as the requested consolidated
lifecycle-fence pass.

- Lease tokens. Marker fields were (streamId, createdAt), shared by every
  contender on the same generation — completion and Stop, or two racing Stops —
  so a losing contender's cleanup erased the winner's still-live marker. Every
  registrant now carries a unique lease token in the field and may only ever
  clear its own lease; unqualified legacy clears cannot touch qualified entries.

- Admission fence. Authentication can pass before the deletion barrier goes up,
  and the durable createJob is several async steps later — a deletion quiesce in
  that window saw neither an active job nor a marker and could cascade before
  the admitted request created its job. The controller now registers an
  admission lease and THEN rereads the deletion barrier: the ordering guarantees
  either this request observes the barrier (403, lease released, slot/claim
  cleanup) or the quiesce observes the lease and defers. Held until createJob is
  durable; released on every refusal and initialization-error path. Fail closed
  when the lease itself cannot be registered (503 retryable).

- Undelivered-Stop retention. abortJob released its lease in a finally even when
  delivery AND publication had provably failed — the job reads terminal
  (invisible to active-set scans), a user Stop writes no durable abort fence,
  and the remote owner keeps generating and will persist its abort-catch writes
  whenever the signal finally lands. The lease is now retained in exactly that
  case, and each resignal attempt heartbeats a fresh lease so the fence outlives
  the TTL for as long as delivery is still being driven. The abort-winning
  turn's own loser-side pending saves are additionally fenced in the controller
  catch (best-effort — those writes are already in flight).

- Legacy abort enrollment. abortMiddleware (assistants abort route fallback for
  non-assistants endpoints) awaited abortJob and then spent usage and saved the
  stopped response AFTER the abort's lease was released. Both writes now run
  inside `beforePublish`, between the abort CAS and publication, covered by the
  same lease as every other abort.

Barrier tests, each verified to fail without its fix: same-generation lease
isolation (store), racing two-Stop loser cleanup (manager, stale-read forced
CAS race), undelivered-Stop lease retention, resignal heartbeat, admission
refusal with lease-before-reread ordering plus release-on-durable-create, and
legacy-abort persistence inside beforePublish.

* fix: heartbeat-backed owner-lifecycle leases close the settlement handoff races

Sixth external review round: the remaining P1 interleavings were one structural
problem — lease handoffs that were not atomic — resolved as the requested
consolidated lifecycle-lease pass.

- Quiesce reads leases BEFORE the active-job scan. The admission-lease -> durable
  -job handoff is only atomic against a reader in the OPPOSITE order of the
  writer: writers hold the lease strictly until the job is active-set visible,
  so leases-first shows every interleaving either the lease or the job.
  Jobs-first allowed a request to create its job after the scan and release its
  lease before the count — hiding both, cascading, and letting the new
  generation persist into a deleted account.

- The abort acknowledgement is fenced by the owner-lifecycle lease. Redis ACKed
  the moment the owner's AbortController tripped; the stopping side released its
  lease on that ACK while the owner's asynchronous abort-catch persistence was
  still ahead. The transport now awaits a manager-installed pre-ACK hook that
  registers a DETERMINISTIC owner lease (exactly one owner exists per
  generation, and determinism is what lets the signal-time registrant and the
  owner's catch-side release agree across processes) before the acknowledgement
  is persisted or published; an owned same-replica abort bridges to the same
  lease before tripping its local controller. Both generation-owner catches
  (fresh turn, resume) release it once their writes land.

- A failed replacement handoff no longer orphans the predecessor. The atomic
  replacement removes it from active storage, and an unconfirmed handoff
  terminalizes the replacement too — leaving nothing a quiesce could discover
  while the predecessor's provider may still be generating. Its owner lease is
  now retained at the point the receipt fails delivery; the owner replica renews
  it through the pre-ACK fence when the signal finally lands.

- Leases HEARTBEAT while held. The five-minute store TTL only bounds a crashed
  holder; live persistence — a stalled save, a long deferred title — must never
  outlive its own fence. holdUserFinalization registers and renews every minute
  until released; the controllers' completion/error/admission leases all hold.
  The undelivered-Stop retention moved to a deterministic `stop` lease that
  every resignal attempt renews and the first successful one clears (no more
  opaque leases accumulating to TTL), and a THROWN abort transition releases the
  contender lease instead of leaking it.

- The user-document abort-fence mutations now invalidate the auth user-doc
  cache, matching every other user-doc write.

Barrier tests, each verified to fail without its fix: quiesce lease-scan
ordering (plus the observed-lease defer), pre-ACK fence ordering at the
transport, owned-abort owner-lease bridging, replacement-handoff predecessor
retention, held-lease heartbeat past the TTL, and failed-then-successful
resignal reaping the retained stop lease.

* fix: one manager-owned owner-lease span across every abort delivery path

Seventh external review round; four lifecycle-fence gaps, closed by making the
owner-lifecycle lease a single manager-owned, heartbeat-held span.

- Fail-closed acknowledgements. The pre-ACK hook registered a one-shot lease and
  the transport ACKed even when it failed; a same-replica owned abort likewise
  swallowed registration failure. The hook now acquires a HELD owner lease
  (heartbeat until the owner's catch releases it via releaseOwnerLease) and a
  rejection SUPPRESSES the acknowledgement — the stopping side stays retryable
  behind its retention lease, and every resignal re-drives the handler. The hook
  also stopped gating on `job.createdAt === generationId`: during a replacement
  handoff the store holds the replacement while the abort targets the
  predecessor, and that gate silently skipped exactly the generation being
  acknowledged (the store job is owner identity, never a generation gate). A
  local owned abort acquires the same held lease before tripping its provider;
  post-CAS the trip cannot be withheld, so acquisition failure downgrades
  delivery and the retention handoff keeps the user fenced.

- Committed-but-lost-reply disambiguation. A thrown abort transition released
  the contender lease as if nothing had happened, but a Lua CAS can commit and
  lose its reply — an aborted job invisible to active-set scans whose provider
  was never signalled, with no fence left. The throw path now re-reads the exact
  generation: only a job still live under the caller's identity proves no
  commit; committed or ambiguous outcomes hand the fence to the deterministic
  stop lease (kept on the contender lease if even that fails) before rethrowing.

- Replacement handoff covered end to end. A LOCAL replacement abort acquires the
  predecessor's held owner lease before the trip (failure reports the receipt
  undelivered, engaging retention). Failed-handoff retention is no longer a
  swallowed one-shot: it heartbeats with the durable acknowledgement proof as
  its renewal predicate — acquisition failures keep retrying for as long as the
  fence is needed, and the retainer stands down (without clearing the shared
  field) once the remote owner ACKs and thereby holds its own lease.

- A LOCAL resignal delivery hands off to the owner lease BEFORE clearing the
  retained stop lease, and keeps the stop lease when that handoff fails.

Barrier tests: hook rejection suppressing the ACK, commit-then-lost-reply
retention with its provably-uncommitted counterpart, local-resignal owner
handoff ordering, pre-ACK owner lease held past the store TTL until release
(and provably stopped after), and failed-handoff retention retrying on its
heartbeat — verified fail-before/pass-after by stashing the fixes.

* fix: finish scheduled chat lifecycle hardening

* fix: close scheduled chat review follow-ups

* fix: generation-fence abort recovery evidence

* test: wait for settled approval tool output

* test: preserve scheduled init reconciliation option

* refactor: rebuild scheduled chats on durable agent triggers

* test: reset MCP cache mock between cases

* test: isolate scheduler startup in server specs

* fix: harden scheduled run lifecycle

* fix: normalize schedule capacity conflicts

* test: type schedule collision fixture

* fix: re-fence scheduled resume and expiry

* fix: fence scheduled resume handoffs

* fix: release superseded manual schedule leases

* fix: release failed run-now claims

* fix: release superseded engine claims

* fix: repair schedule dialog interaction and rework its form

The agent picker was unusable: ControlCombobox portals its popover to the
body by default, which lands it outside the dialog's Radix focus trap. Clicks
passed through it, it could not be tabbed into, and the trap fighting Ariakit
for focus locked the page up on selection. The prop is documented for exactly
this case — pass `portal={false}` and give the dialog `overflow-visible`, as
ProjectButton already does. The time and day dropdowns defaulted the same way.

Alongside that:

- Extract the agent builder's instructions editor (special-variable menu plus
  expand-to-fullscreen) into a controlled `VariableEditor` and use it for the
  schedule prompt. Insertions now route through `onChange`, so react-hook-form
  sees them — the schedule PATCH is built from `dirtyFields`, and a `setValue`
  that skipped dirty tracking would have dropped an inserted variable silently.
- Replace the hand-rolled frequency buttons with the shared `Radio`. They marked
  the selection with `bg-surface-hover` on an outline button whose hover is the
  same token, so the selected option was indistinguishable from a hovered one;
  `Radio` is also a real radiogroup rather than four `aria-pressed` toggles.
- Wrap the fields in a real `<form>` and associate the footer button by id, so
  Enter submits. Group the frequency, day and time controls in fieldsets.
- Add placeholders for name and prompt, match the textarea fill to the other
  fields, and label the hourly case as minutes past the hour.
- Widen the dialog to `md:max-w-3xl` and pair name with agent so the form fits
  without scrolling on desktop.
- Move scheduled chats below skills and above prompts in the side nav.

The new dialog spec fails when the portal fix is reverted.

* test: cover scheduled and subagent deletion drains

* refactor: own the form-control appearance in the client primitives

Addresses the codex finding on ScheduleDialog: a feature-local `FIELD_CLASS`
restated the `Input` primitive's border, radius, height and background so it
could be pasted onto the schedule dropdowns, leaving those controls with no
connection to the primitive they were imitating.

Move that appearance into `packages/client/src/components/Field.ts` as the
single source `Input` and `Textarea` now compose, and give `Dropdown` and
`ControlCombobox` a `variant="field"` that applies it. The schedule dialog
passes the variant and carries no class strings of its own.

This also repairs a break the dev merge would otherwise have introduced: the
newer `Dropdown` splits `className` (wrapper) from `triggerClassName`, so the
old pasted classes would have landed on the wrapper and left the triggers
unstyled.

The semantic-token guard now watches the shared module and asserts each
primitive still composes it, which covers more than the two files it read
before.

* fix: keep an explicit schedules disable from becoming an opt-in

`use` is two things at once for a dual-purpose runtime interface field: a
permission bit, which DB overrides strip, and the runtime disable signal that
`getLimits` reads. Stripping it from `{ use: false, maxPerUser: 2 }` leaves an
object, and `getLimits` treats any object without `use: false` as enabled — so
an admin override written to stop scheduled billing for a role or user started
it instead.

Collapse an explicit disable to the boolean form before the strip, on both
paths that accept it: the `interface.schedules` field patch, which admitted the
object wholesale because bare runtime paths deliberately bypass the permission
gate, and the overrides merge, which reached the composite-field branch and
kept `maxPerUser`. Objects that only narrow limits are untouched, so a
principal can still be given a smaller cap.

Both regressions fail without the normalizer.

* fix(schedules): Wave A — null-balance CAS, atomic paused-card clear, clustered erasure sweep

Slice 1 (thread r3804518381): route the existing-null balance initialization
through a { user, tokenCredits: null } compare-and-set instead of a blind $set,
so a concurrent initializer/charge landing between the preflight read and the
write is never handed back its spent starting balance. On a CAS miss the
preflight re-reads the winner. The absent-record $setOnInsert path and the
credited-record refill-config sync are unchanged. Adds the initializeNullBalance
adapter (no upsert) and regression coverage for winner/miss/sync cases.

Slice 2 (thread r3804518388): updateScheduleById now drops a `requires_action`
lastRun projection atomically with the configRevision bump. Any pause present at
edit time was projected under the pre-edit revision and can never be replaced by
its own revision-fenced terminal outcome — a disabling edit would strand the
card on "Needs approval" forever. Implemented as classic-operator CAS branches
(DocumentDB rules out a conditional pipeline $unset), fenced on the card STILL
being the pause so a terminal outcome or newer occurrence that races in is
preserved. Terminal history survives untouched.

Slice 4 (thread r3803826204): expose initializeScheduleErasureSweep from the
schedule runtime facade and start it in every clustered (experimental) worker
after Mongo is up. It re-drives eraseScheduleIfDrained for soft-deleted rows so
a hidden prompt cannot outlive its drain when the delete/erase-on-settle
attempts miss. It arms nothing else and never infers owner death from a
process-local missing job (isTopologySafeToArm gates that).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

* fix(schedules): Wave B.1 — reversible account-deletion schedule suspension

Thread r3804518383. Account-deletion quiesce marked every schedule `deleting`,
disabled it, and cleared nextRunAt destructively. When a later cascade step (or
the drain itself) failed, the controller cancelled the user-deletion fence —
restoring the user — but the schedules stayed `deleting` and were erased by the
sweep, silently losing all of a live user's scheduled prompts.

Replace the destructive marking with a REVERSIBLE, token-fenced suspension:

- suspendUserSchedulesForDeletion(userId, token) snapshots each schedule's prior
  enabled/nextRunAt under a per-attempt token, then fences firing (disable, clear
  nextRunAt, rotate claimToken). It never sets `deleting`, so a suspended row is
  not erasure-eligible. Snapshotting reads then bulkWrites (a classic update
  cannot copy field values under DocumentDB), fenced per row so an already-
  suspended/soft-deleted/edited row is left alone; idempotent per token.
- restoreUserSchedulesFromDeletion(userId, token) reverses it, re-enabling and
  re-arming only rows still carrying the exact attempt token and not independently
  deleted — so an owner-deleted or newer-attempt-suspended schedule is never
  resurrected.
- deleteUserController generates the attempt token, passes it to quiesce, and on
  any failure that cancels the user-deletion fence restores the suspended rows. A
  successful deletion hard-deletes them (and their snapshots) via the existing
  cascade and never restores.

Adds a `deletionSuspension` embedded field (excluded from the wire schedule),
data-method regression tests (suspend/restore/fence/idempotency/no-resurrect),
and controller tests (restore on drain-false and post-quiesce cascade failure,
no restore on success).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

* fix(schedules): Wave B.2 — wire the interactive Stop persistence protocol

Thread r3804255932. The durable Stop primitives (requestRunAbort 'stop',
getScheduleRunAbortState, markRunAbortPersisted) already existed but production
only used requestRunAbort(..., 'deletion'). An interactive Stop flipped the job
to `aborted` and then persisted its partial message + checkpoint inside
`beforePublish`, without ever stamping the schedule Stop or acknowledging it —
so reconciliation, the generation owner, or a concurrent schedule/account
deletion could terminalize the run and release its capacity (and erase data)
mid-write.

Wire the request -> persist -> acknowledge -> settle barrier through the
schedule runtime (the route never touches raw Mongo):

- Expose beginScheduledStop / acknowledgeScheduledStopPersistence on the service.
- The abort route stamps the Stop BEFORE signalling abortJob (a serialized
  'in_progress' loser returns 409 STOP_IN_PROGRESS without a second abort),
  acknowledges only after beforePublish persistence succeeds, and on a
  persistence failure leaves the barrier unresolved so the run stays preserved
  (client retries; stale-owner timeout is the bounded recovery). A failed abort
  releases the stamp it placed so a replacement/retry is never blocked through
  its predecessor.
- recordScheduleOutcome (the owner settlement path) now waits, bounded, for the
  Stop acknowledgement before terminalizing; a resolved/non-stop/stale marker
  proceeds immediately. The paused Stop settles only after its own ack.

Adds service-layer barrier tests and route-level ordering/persistence-failure/
in-progress tests; the data-layer serialization is already covered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

* fix(schedules): Wave C.1 — reconcile durable trigger delivery with the reservation

Threads r3803826192 (manual limiter) and r3804255924 (PII/moderation), plus the
independently-found long-Retry-After race. fireSchedule reserves a `started` run
and a global capacity slot BEFORE the durable trigger delivery reaches the chat
route, where an interactive limiter (manual Run Now), PII, or moderation can
reject it before any generation job exists — dead-lettering the delivery while
the run sat `started` until the 30-minute orphan sweep mislabeled it interrupted.
And a valid delivery deferred by Retry-After (up to 24h) could be orphan-settled
and have its capacity released, then fire anyway.

Translate durable delivery state into the schedule outcome:

- Store the deterministic trigger deliveryKey on the ScheduleRun reservation
  (computed from the envelope BEFORE enqueue, so an ambiguous commit still has it).
- Add a getTriggerDelivery engine dep (wired to the merged trigger service's
  getDelivery) that reads the durable delivery by key.
- Schedule reconciliation, for a jobless `started` run: staging/pending/leased →
  admission is live, never orphan; dead → record `error` from the durable
  lastError and release capacity promptly (no 30-minute wait), through the
  ordinary outcome/auto-disable path; succeeded or no record → the existing
  legacy orphan policy (interrupted only past the cutoff); a delivery lookup
  failure defers rather than orphaning a possibly-live delivery. Limiter/PII/
  moderation middleware writes no schedule state.

Adds reconcile state-mapping tests (dead/pending/leased/staging/succeeded/none/
lookup-failure) and a fire test that the reservation's deliveryKey equals the
enqueued delivery's idempotency key.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

* fix(stream): Wave C.2 — durable retry/ack for terminal host lifecycle actions

Thread r3804518375. Approval expiry won the `requires_action → aborted` CAS and
then invoked the host hook best-effort: `runApprovalExpiredHandler` swallowed a
failure, and because later sweeps enumerate only `requires_action` jobs, the now-
aborted job was never offered again. In the clustered entrypoint (no schedule
reconciler) the ScheduleRun stayed `requires_action` and its retained job
persisted indefinitely.

Make the host lifecycle work durable rather than schedule-specific:

- Add a generic `terminalHostActionPending` marker, set ATOMICALLY in the same
  terminal transition (ApprovalLifecycle.expireWithIdentity), only when a host
  adapter is installed.
- Retain and index such jobs: both stores keep them out of terminal reaping and
  expose getTerminalHostActionJobs(); Redis adds a set + extended (24h-bounded)
  TTL, in-memory a bounded 24h retention so a permanently-failing hook cannot leak.
- The manager clears the marker only after the adapter acknowledges success,
  fenced by generation identity (clearTerminalHostAction), so a replacement
  generation can neither clear nor execute its predecessor's action.
- cleanup()/expireStaleApprovals() enumerates unacknowledged terminal host actions
  across restarts and replicas and retries the idempotent hook; the relay only
  re-invokes while the marker is unacknowledged, so a successful ack prevents
  duplicate work. Store-won expiry marks it too, so a loser-replica relay still
  crosses the hook.
- Terminal SSE notification continues regardless of host-hook outcome.

Covers in-memory behavior (retry after failure, restart/other-replica retry, ack
prevents duplicates, identity fence, terminal notification on failure, no marker
accumulation for non-scheduled jobs) and updates the Redis cluster-membership
contract test for the new index.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

* style(schedules): satisfy import sorting in fire.ts and fire.spec.ts

CI "Static checks" failed on IMPORT_SORT for the two files Wave C.1 added imports
to (the AgentTriggerEnvelope type import and getAgentTriggerIdempotencyKey).
Applied scripts/sort-imports.mts to exactly those files — imports-only reordering,
no behavior change. Other files reported by a repo-wide check are pre-existing on
dev and deliberately left untouched so this PR is not widened.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

* fix(schedules): repair the deletion CLI and order restore before the fence release

Addresses three findings from the fresh Codex review.

P1 — config/delete-user.js called methods.disableUserSchedulesForDeletion, which
Wave B removed in favor of suspendUserSchedulesForDeletion. The file is
@ts-nocheck and its spec mocked the removed name, so neither typecheck nor tests
caught it; the real CLI would throw a TypeError before deleting anything and then
only unwind the fence. The CLI now uses the tokenized protocol: it mints a
suspension token, suspends with it, and restores that exact attempt's rows in its
finally block when the deletion does not commit. Its spec mocks the real methods,
so the breakage can no longer hide.

P2 — both the HTTP controller and the CLI released the user-deletion fence BEFORE
restoring schedules. That fence is what refuses new schedule writes/claims, so the
gap let an owner PATCH edit a still-suspended row and have its enabled/next-run
state overwritten by the older snapshot, and let a second deletion attempt
re-suspend under a new token — making the first restore a no-op and stranding the
disabled snapshot permanently. Restore now runs first, while writes are still
fenced.

Hardening for the same defect class across a crash: suspendUserSchedulesForDeletion
now ADOPTS an existing suspension's snapshot when re-suspending a row abandoned by
an earlier attempt, instead of re-capturing the row's current (already-suspended)
state. Without this, an attempt that died before restoring would have its
successor snapshot "disabled, no next run" and permanently strand the schedule.

Tests: CLI restore-before-fence ordering and no-restore-on-success; the same
ordering assertion on both controller post-quiesce failure paths; a data-method
regression that a second attempt adopts the abandoned snapshot and restores the
original enabled/next-run state.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

* fix(schedules): converge dead deliveries in topology-safe maintenance

Codex finding: the `dead` delivery mapping added in Wave C.1 lives only inside
startScheduleEngine's reconciler, but the clustered entrypoint arms no engine — it
runs erasure-only maintenance. A delivery queued before a restart into clustered
mode and then rejected before generation creation (interactive limiter, PII,
moderation) dead-letters while its ScheduleRun stays `started`, holding a global
capacity slot indefinitely for an ordinary non-deleting schedule.

Add a dead-delivery convergence pass to the erasure sweep, so every topology that
runs schedule maintenance settles it. The pass is POSITIVE-EVIDENCE-ONLY and is
therefore safe where absence-based reconciliation is not: a `dead` delivery is
durable shared state proving no generation owns the reservation. It settles only
when the job is confirmed absent or identity-mismatched (an identity-matched job
still owns the run), defers on an unknown job lookup, on an in-flight abort, and
on an in-flight resume hand-off, ignores legacy reservations with no deliveryKey,
and applies a short grace so an accepted delivery still creating its generation is
never settled mid-handoff. Auto-disable policy is deliberately left to the armed
engine; this path records the failure and frees the slot.

Deliberately does NOT touch api/server/experimental.js — the clustered entrypoint
already starts this sweep, so the convergence arrives through the existing
initializer and the shared-file footprint stays as-is.

Tests: settles a dead delivery as error under an explicitly UNSAFE topology,
leaves live deliveries alone, never settles under an identity-matched running
generation (delivery is not even consulted), defers an in-flight abort, and
ignores a reservation with no deliveryKey.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

* fix(stream): refresh host-action retention on each retry attempt

Codex finding: unacknowledged terminal host-action evidence was capped at the
24h pause TTL, so a host dependency (Mongo) unreachable for longer than that let
the Redis key — and its pending marker — expire with no generation-fenced
acknowledgement, stranding the ScheduleRun where no reconciler is armed.

Measure retention from the LAST retry rather than from the terminal transition:
enumerating a pending host action IS the retry attempt, so both stores refresh
its retention as they hand it to the hook (Redis re-EXPIREs the job key; the
in-memory store stamps terminalHostActionRefreshedAt and bounds from it). Evidence
therefore survives as long as some replica is still actively retrying, while a
deployment that stops sweeping entirely still lets it age out — so this does not
reintroduce the unbounded leak the cap existed to prevent.

Test: after a failed hook, a later cleanup pass keeps the marker pending and moves
its retention basis forward.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

* fix(schedules): defer settlement when the Stop barrier times out

Codex finding: waitForStopPersistence returned after its 5s poll budget even when
the Stop was still fresh and unacknowledged, and recordScheduleOutcome then went
straight on to terminalize the run — releasing its capacity, deletion, and erasure
barriers while beforePublish may still have been writing. Slow checkpoint cleanup
is indistinguishable from a dead route on that signal, so the timeout was being
treated as if the barrier had been satisfied.

The poll budget now means "undecided", not "clear". On timeout with a fresh,
unacknowledged Stop the barrier DEFERS: recordScheduleOutcome returns false
without recording, leaving the run active/preserved. Settlement then happens
either when the route acknowledges, or once the existing stale-owner cutoff
(ABORT_OWNER_PRESUMED_ALIVE_MS) authorizes a later attempt — which the loop
already treats as clear-to-settle. Callers with durable retry (the approval-expiry
host action, reconciliation) re-drive it, so a deferral converges rather than
stranding the run.

Test: a fresh Stop that never acknowledges within the budget reports not-settled
and records no outcome.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

* fix(schedules): require definite delivery failure, extract its message, converge deferred Stops

Third Codex round. Three findings in code from this closeout, plus one pre-existing
P1 that is a one-line operator-safety fix.

P1 — dead-delivery settlement demanded too little evidence. `dead` does not prove a
request was rejected: the trigger host marks response timeouts and invalid success
responses `certainty: 'ambiguous'`, and the engine dead-letters those once retries
are exhausted. The erasure sweep treated every dead letter as positive evidence, so
an ambiguous one sitting over a generation a peer had accepted could terminalize the
run and release its capacity mid-flight. It now settles only on a DEFINITE rejection,
unless job absence is deployment-authoritative (safe topology), where the
confirmed-absent job is itself the evidence.

P1 — `lastError` is an `AgentTriggerDeliveryFailure` object, not a string. A
duplicated local interface declared it `string` (against CLAUDE.md's no-duplicate-
types rule), so both the sweep and the engine reconciler passed the object into the
String-typed run/schedule `error` fields; Mongoose would reject the cast, the per-row
catch would swallow it, and the run would keep its global capacity slot. The dep type
now reuses the canonical `AgentTriggerDeliveryFailure` and both call sites pass
`.message`. Re-typing immediately surfaced a stale test that had asserted a string.

P1 (pre-existing) — the base-config global stop is honored in `getLimits` via
`isRuntimeDisabled` rather than a literal `=== false`. The stop has two shapes, and
deepMerge turns base `{ use: false }` plus a principal override of `true` into
`{ use: true }`, so the literal check reported the feature enabled and Run Now
dispatched straight through fireSchedule, bypassing the operator's emergency stop.
Now the same predicate the engine gate already uses.

P2 — a Stop whose settlement DEFERRED past the poll budget had no convergence path
where no reconciler is armed. `acknowledgeScheduledStopPersistence` now optionally
re-drives the terminal outcome once the barrier clears; `recordRunOutcome` is
match-guarded and idempotent, so an owner that already settled makes it a no-op. The
abort route passes it for a running generation; a paused job still settles explicitly.

Tests: ambiguous dead letters refused under unsafe topology but settled when absence
is authoritative, definite rejections settled either way, the failure message carried
through, and the abort route's re-drive present for running / absent for paused.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

* fix(schedules): converge terminal runs in clustered workers, retry suspension restore

Fourth Codex round on 74f7af8. Both findings are in code from this closeout.

P2 - a clustered worker never settled a run whose generation finished but whose
outcome write failed. `recordScheduleOutcome` retries three times and then returns
false; the owner honors that by PRESERVING the terminal job as the only surviving
evidence, and the armed engine's reconciler replays exactly that. The clustered
entrypoint arms no engine, and its sweep only covered deleting schedules
(settleAbandonedRuns) and dead deliveries - an identity-matched job was skipped
outright. So an ordinary live schedule's run stayed `started`, held its GLOBAL
capacity slot, and kept a preserved job that carries no `completedAt` and is
therefore invisible to the store's finished-job sweep: both leaked until store
expiry.

The live-schedule pass now also converges from a retained terminal job, mirroring
the reconciler branch it stands in for: honor the owner's stamped outcome over the
generic status (so a balance refusal still walks its streak rather than resetting
it), clear the reserved conversationId when the generation never emitted its
created event, and delete the retained job only AFTER the outcome write is durable.

This stays positive-evidence-only and safe in every topology. Presence, not
absence, is the evidence: an identity-matched job is authoritative wherever it is
observed - a shared store shows the real generation, a process-local store can
only be showing this process's own - which is why it needs no
canInferOwnerDeathFromMissingJob fence, unlike the absence-based paths. The
in-flight abort/resume fences still defer, so an `aborted` job cannot settle a run
whose owner may still be persisting. Both cases now share one pass over one window
rather than two, and `retainedOutcome` moved to types.ts so the sweep reuses the
engine's mapping instead of duplicating it (and stays independent of the engine).

P2 - a failed restore stranded a live account's schedules. Cancelling an account
deletion restores the suspended rows while the deletion fence is still armed and
then releases that fence; nothing re-drives the restore afterwards, so one
transient write failure left the user with silently disabled, next-run-less
schedules. The restore is now retried at the single choke point both the HTTP
controller and the CLI share. Retrying is safe because each attempt re-reads only
the rows STILL carrying the token: a partially-applied unordered write converges
on exactly the stragglers, and a fully-applied one finds nothing.

The fence is still released when every attempt fails, deliberately: retaining it
would refuse the live account's schedule writes AND make beginAgentTriggerUserDeletion
report `in_progress` forever, blocking the retry that is the convergence path (a
later attempt adopts this snapshot, so its cancel restores these exact rows). Both
callers now log the user id and suspension token so the state stays recoverable by
hand if that never happens.

Tests: retained terminal job settled and its evidence released, stamped outcome
preferred over the generic status, stamped failure reason carried, reserved
conversationId cleared for a never-created conversation, identity-mismatched
terminal job ignored, abort-in-flight deferred; restore retried past a transient
write failure and converging on a partially applied restore.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

* fix(schedules): converge unprojected pauses and release replayed bookkeeping jobs

Two of the four findings I flagged as pre-existing to my closeout commits. Both are
in code this PR introduces, so merging would have shipped them; both are the same
capacity/evidence-leak class the last four rounds have been closing. The remaining
two (slotless rows escaping the per-user cap, and the non-rotating `started`
reconciliation bucket) are genuinely latent and deliberately left alone.

An UNPROJECTED PAUSE held a global capacity slot forever. The pause projection is
what moves a run row off `started`; `recordScheduleOutcome` already retries it, but
the request controller discarded the result, so three failed attempts were dropped
silently. The armed engine's reconciler replays that state, but the clustered sweep
did not: a paused job is not terminal, so the retained-job path returned without
settling, and the dead-delivery path never inspects an identity-matched job at all.
The row stayed `started` with no cutoff that would ever clear it.

The call site now surfaces the failure, and the sweep converges it, mirroring the
reconciler's pause branch: project `requires_action` (which frees the slot) but do
NOT release the job's evidence — unlike a terminal job it is still live, awaiting an
approval. The resume hand-off fence still defers, so re-projecting cannot release a
slot a continuation just claimed.

The BOOKKEEPING REPLAY pass leaked its retained job. A run reaches that pass only
because its owner crashed before bookkeeping — which is also before it could release
the job it retained for exactly this recovery. The active-run pass clears its own;
this one never did, and a preserved job is deliberately kept WITHOUT `completedAt`
so the store's finished-job sweep cannot reap it early, so nothing else ever would.
It now clears after `finalizeBookkeeping` succeeds — identity-guarded, a no-op when
no job is retained, and skipped entirely when the replay itself failed, since the
retained job is the only surviving evidence in that case.

Tests: pause projected and the slot freed while the live job's evidence is kept,
paused job deferred during a resume hand-off, retained job released once replayed
bookkeeping is durable, and retained job kept when the replay fails.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

* fix(schedules): fence the clustered pause replay against a concurrent resume claim

Codex round 5, on code I added in f735341. The finding is correct and the exposure
is one I introduced.

The pause replay lives in a sweep that runs in EVERY clustered replica, so several
sweepers can observe the same unprojected pause. Its guards were all derived from an
in-memory row SNAPSHOT: hasResumeHandoffInFlight reads the snapshot's
resumeClaimedAt, and recordRunOutcome's pause branch matched any row currently in
`started`/`requires_action` with no fence of its own. The race: sweeper A projects
the pause and frees the slot; the owner's approval then claims a fresh one
(markRunResumeClaimed takes the row to `started` WITH resumeClaimedAt in one write);
sweeper B, still holding the pre-projection snapshot, passes its hand-off check and
replays — `$unset: { capacitySlot, resumeClaimedAt }` under a continuation that is
already running. The run reverts to `requires_action` while its generation proceeds
outside global capacity.

The engine's reconciler makes the same call and has the same snapshot-derived guard,
but v1 arms exactly one engine, so it has no concurrent racer; the sweep is the first
thing to run this transition in parallel. Left the engine alone rather than widening
the change: its `requires_action` re-affirmation is deliberate and single-writer.

Fixed where the race is, in the write itself. `recordRunOutcome` takes an optional
`requireNoResumeClaim`, which adds `resumeClaimedAt: { $exists: false }` to the pause
filter, and the sweep sets it. Because the stamp is written in the SAME update that
moves the row to `started`, its absence is atomic proof no resume owns the row. The
flag is deliberately NOT set by the generation owner: its own re-pause legitimately
clears the stamp as the hand-off's completion signal.

The fence cannot block the recovery it exists to enable: markRunResumeClaimed only
matches `requires_action`, so a genuinely stuck `started` row can carry no resume
claim — it is in fact blocking its own approval until this replay frees it.

Tests: a replay from a stale snapshot leaves a resume-claimed row's status, slot, and
claim stamp intact, while a stuck row with no claim still recovers; and the sweep is
asserted to send the fence.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

* fix(schedules): fence stale resume claims by age, not existence

Codex round 6, on the fence I added in ee00c60. Correct again, and the defect is
the mirror image of the one it fixed.

The fence was an EXISTENCE check (`resumeClaimedAt: { $exists: false }`) while its
caller's guard is a FRESHNESS check (hasResumeHandoffInFlight, bounded by
RESUME_HANDOFF_STALE_MS). They agree while a claim is fresh and disagree once it is
abandoned: a worker that dies after markRunResumeClaimed takes the row to `started`
and stamps resumeClaimedAt — but before the continuation resumes or
releaseRunResumeClaim rolls it back — leaves the stamp set forever. Past the bound
the sweep correctly stops deferring and tries to recover the row, but the write
rejected it purely because the field still existed. The row stayed `started` holding
its global capacity slot, and its approval was unresumable for good, since
markRunResumeClaimed only matches `requires_action`. That is exactly the stuck state
this replay exists to clear, so the fence had reintroduced it for the crashed-resume
case.

`requireNoResumeClaim: boolean` becomes `resumeClaimStaleBefore: Date`, and the
filter matches a row with no claim OR a claim older than that cutoff. The sweep
passes the SAME bound its in-flight check uses, so the two can no longer disagree.
A genuinely racing claim is by construction fresh — it is created after the sweeper's
snapshot — so the race from round 5 stays closed.

Tests: a row whose resume claim was abandoned by a dead worker now recovers (status,
slot and stamp all cleared), alongside the existing two — a fresh claim still repels
a stale-snapshot replay, and an unclaimed stuck row still recovers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CZwZPDBgxy4mCWWg6zQSjq

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-20 11:51:30 -04:00
Jackson Riding
389cfebea1
🔒 fix: Hide Password Input in Reset CLI (#14923)
Prevent reset-password prompts from echoing sensitive input and add regression coverage for terminal output.
2026-08-18 07:50:47 -04:00
Danny Avila
aa35cd42b1
📬 feat: Add Durable Agent Trigger Delivery (#14925)
* feat: wire trusted agent trigger dispatch

* feat: add durable agent trigger delivery

* fix: annotate trigger envelope byte limit

* test: isolate trigger startup in server specs

* fix: fence trigger delivery during account deletion

* test: isolate trigger service in user controller specs

* fix: close trigger deletion admission race

* fix: harden account deletion fences

* fix: close durable trigger review gaps

* fix: require offline stale-fence recovery

* fix: type trigger lane sequence ids

* fix: fence admin user deletion triggers

* fix: make trigger deletion recovery durable

* fix: harden offline user deletion

* fix: serialize trigger lane publication

* style: sort trigger delivery imports

* fix: recover orphaned trigger publications

* fix: preserve trigger recovery ordering

* fix: fence trigger publication during purge

* fix: defer remote trigger deletion fences

* fix: close durable delivery cleanup races

* fix: drain CLI generation owners before deletion
2026-08-17 09:25:08 -04:00
Danny Avila
3a3a8dcad0
🖍️ refactor: Typed Console Colors and Hardened Script Helpers (#14778)
* 🛠️ refactor: Enhance console color handling and improve deleteNodeModules function

* refactor: Use coloredConsole for consistent console output in invite-user.js

* refactor: Convert year to string format in invite user payload
2026-08-12 22:48:36 -04:00
Danny Avila
2d606a9783
🧹 chore: Migrate Legacy Duplicate Code Files Blocking Dedupe Index (#14593)
Atomic file claiming (#11675) added a unique partial index on
(filename, conversationId, context, tenantId) for execute_code outputs.
Records written before it inserted a new document per regeneration, so
any deployment that re-ran a cell producing the same filename carries
duplicates the index cannot span: Mongo aborts the build with E11000 and
the constraint is silently absent — the claim path still works, but
without its database-level guard against concurrent inserts.

Adds config/migrate-code-file-duplicates.js to normalize that legacy
data, following the existing migration conventions (dry-run default,
--batch-size, runAsSystem for cross-tenant scans).

Renames rather than deletes: each duplicate is a distinct stored object,
typically still referenced by a message attachment, so removing one
would strip a real artifact from a user's history. The newest record
keeps the canonical name — matching the claim path's latest-write-wins
behavior — and older copies gain a ' (n)' suffix that skips names
already taken in the conversation. Attachments embed their own filename,
so rendered history is unchanged.

After a successful apply the script builds the index directly (targeted
createIndex, not syncIndexes) so the operator learns immediately whether
the constraint is now in place.
2026-08-02 06:41:21 -04:00
Danny Avila
ad0f72dede
🌀 ci: Deterministic Circular Dependency Checks (#14579)
* 🌀 ci: Deterministic Circular Dependency Checks

* 🌀 ci: Enforce Type-Level Edges in Circular Dependency Scan

* 🌀 ci: Materialize Import-Type Expression Edges in Cycle Scan

* 🌀 ci: Collect Inline Type-Only Specifier Edges in Cycle Scan
2026-08-01 14:43:26 -04:00
unsnow-iac
8374b8416a
📧 fix: Restore invite-user CLI (#14436)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
* fix: restore invite-user CLI

* fix: guard createInvite failure shape in invite-user CLI

createInvite resolves to { message } on error rather than throwing, so the
CLI interpolated it into the register link as [object Object] and emailed
an unusable invite.

* fix: normalize invite email before token creation

findToken lowercases its email query but the Token schema has no lowercase
setter, so a mixed-case address passed to the CLI was stored verbatim and
the resulting invite could never be redeemed.

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-07-26 23:35:50 -04:00
Danny Avila
481e2a9257
🧹 chore: Prune Dangling Images After deployed-update Pull (#14269) 2026-07-14 18:02:38 -04:00
Marco Beretta
b84e26671e
🕒 feat: Track Terms Acceptance Timestamp (#10810)
* feat: add terms acceptance timestamp tracking and migration script

* feat: update migration script to use countUsers method for user count

* Update config/migrate-terms-timestamp.js

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* feat: enhance terms acceptance response to include acceptance timestamp

* fix: make terms acceptance idempotent and fail migration on partial errors

Preserve the original termsAcceptedAt on repeat accepts within a terms
cycle so retried or duplicate requests no longer overwrite the first
acceptance time. Exit the migration script with a non-zero status when
any per-user update fails so partial failures are not reported as
successful.

* style: fix import ordering in data-provider mutations

* refactor: record terms acceptance atomically to preserve first-accept time

Replace the read-then-write in acceptTermsController with a single
atomic acceptTerms method that conditionally stamps termsAcceptedAt via
an $ifNull aggregation update. This removes the TOCTOU window where two
concurrent first-time accepts could overwrite the earlier acceptance
timestamp, while still preserving an existing timestamp and backfilling
legacy accepted users.

* fix: run terms timestamp migration under system tenant context

Wrap the count, cursor scan, and per-user updates in runAsSystem so the
tenant isolation plugin does not throw under TENANT_ISOLATION_STRICT or
scope the cross-tenant migration to a non-existent tenant, matching the
other maintenance migrations.

* fix: guard terms backfill against concurrent acceptances

Add the missing-timestamp predicate to the per-user updateOne filter so
a user who accepts through the API between the cursor read and the write
keeps their real acceptance time instead of being overwritten with
createdAt. Track modified vs skipped so the summary reflects skips.

* fix: scope terms backfill to still-accepted users

Add termsAccepted: true to the per-user updateOne filter so a reset that
clears acceptance between the cursor read and the write is not re-stamped
with createdAt, which would otherwise poison the next acceptance cycle
through the $ifNull preserve in acceptTerms.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-06-24 16:26:42 -04:00
Atef Bellaaj
86fe79c37d
🔗 feat: Add Granular Access Control to Shared Links via ACL System (#13051)
* feat: Add granular access control to shared links via ACL system

* fix(shared-links): preserve isPublic on failed migration grants

Transient ACL failures during auto-migration permanently stranded
links — $unset ran unconditionally, removing the legacy flag that
triggers retry. Now only $unset isPublic after all grants succeed.

* fix(config): skip isPublic unset for failed ACL grants

Bulk migration unconditionally removed isPublic from all links,
even those whose ACL writes failed. Failed links then lost the
legacy marker needed for auto-migration retry. Now tracks failed
link IDs per-batch and excludes them from the $unset step.

Also adds sharedLink to AccessRole resourceType schema enum —
was missing, only worked because seedDefaultRoles uses
findOneAndUpdate which bypasses validation.

* ci(config): add jest config and PR workflow for migration tests

config/__tests__/ specs depend on api/jest.config.js module
mappings but had no dedicated runner. Adds config/jest.config.js
extending api config with absolutized paths, npm test:config
script, and a GitHub Actions workflow triggered by changes to
config/, api/models/, api/db/, or packages/ ACL code.

* fix(permissions): honor boolean sharedLinks config

SHARED_LINKS has no USE permission, so boolean config produced
an empty update payload — gate conditions only matched object
form, making `sharedLinks: false` a no-op on existing perms.

* fix(share): resolve role before creating shared link

Role lookup between create and grant left an orphaned link
without ACL entries if getRoleByName threw — retry then hit "Share already exists" with no recovery path.

* fix: Restore Public ACL Access Checks

* fix: Type Public ACL Lookup

* fix: Preserve Private Legacy Shared Links

* chore: Promote Shared Link Permission Migration

* fix: Address Shared Link Review Findings

* fix: Repair Shared Link CI Follow-Up

* fix: Narrow Shared Link Mongoose Test Mock

* fix: Address Shared Link Review Follow-Ups

* fix: Close Shared Link Review Gaps

* fix: Guard Missing Shared Link Permission Backfill

* test: Add Shared Link Mock E2E

* test: Stabilize Shared Link Mock E2E

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-06-03 14:17:17 -04:00
Danny Avila
820bc3bf23
🔧 chore: Drop duplicate getAppConfig import in set-balance.js (#13417)
#12669 added `const { getAppConfig } = require('~/server/services/Config');` near the top of `config/set-balance.js` but the same import already existed lower in the file, producing:

  config/set-balance.js
    9:9  error  'getAppConfig' is already defined  no-redeclare

The fork's sync CI surfaced this when its pre-commit hook ran eslint on the merged file. The upstream `eslint-ci.yml` is path-filtered on `api/**`, `client/**`, and `packages/**` — none of which match `config/**`, which is why CI didn't catch it upstream.

Drop the second declaration. Functionally identical, lint clean. No other changes.
2026-05-30 09:12:58 -07:00
Lionel Ringenbach
3487672a32
💰 fix: Load app config in set-balance script to respect balance settings (#12669)
The `set-balance` script called `getBalanceConfig()` without the app
config, so it always reported balance as disabled regardless of the
librechat.yaml configuration. Mirror the working `add-balance` script
by loading the app config first and passing it into `getBalanceConfig`.

Fixes #12413

Co-authored-by: Claude <noreply@anthropic.com>
2026-05-30 10:36:14 -04:00
Danny Avila
b01d34abe2
🪪 fix: Preserve Trusted Registration Provider Overrides (#13307) 2026-05-30 10:04:50 -04:00
Odrec
0f59cd79dd
🪙 fix: Pass appConfig to getBalanceConfig in set-balance script (#13070)
`config/set-balance.js` calls `getBalanceConfig()` without an argument,
so it cannot read the parsed `balance` section from librechat.yaml.
As a result the script always errors with "Balance is not enabled.
Use librechat.yaml to enable it" — even when balance is enabled.

`config/add-balance.js` already follows the correct pattern (fetch
`appConfig` via `getAppConfig()` and pass it into `getBalanceConfig`),
introduced in #9234. This applies the same pattern to set-balance.js.

Co-authored-by: Odrec <Odrec@users.noreply.github.com>
2026-05-11 15:08:23 -04:00
Danny Avila
40a05bbf83
📦 chore: npm audit fixes and Mongoose 8.23 TypeScript follow-ups (#12996)
* chore: Update axios dependency to version 1.16.0 across multiple package files

* chore: Update express-rate-limit and ip-address dependencies to versions 8.5.1 and 10.2.0 in package-lock.json and package.json

* chore: Update mongoose and hono dependencies to versions 8.23.1 and 4.12.18 across multiple package files

* fix: Add type parameters to mongoose lean queries in accessRole and aclEntry methods

* fix: Add type parameters to mongoose lean queries in action, agent, and agentCategory methods

* chore: Update moduleResolution to 'bundler' in tsconfig.json for api and data-schemas packages

* fix: Update mongoose lean queries to include type parameters across various methods for improved type safety
2026-05-07 09:47:40 -04:00
Danny Avila
f7d59d3285
📦 chore: Update TypeScript Config for TS v7 (#12794)
- Enabled `esModuleInterop` in `client/tsconfig.json` for better module compatibility.
- Changed `moduleResolution` from `node` to `bundler` in `client/tsconfig.json`.
- Set `noEmit` to `true` in several `tsconfig.json` files to prevent output generation.
- Removed `baseUrl` from various `tsconfig.json` files to simplify path resolution.
- Updated path mappings in multiple packages to reflect new directory structures.

These changes aim to streamline TypeScript configurations and improve module resolution across the project.
2026-04-23 12:51:03 -04:00
Danny Avila
181d705579
🧹 fix: Clean Up Orphaned Agent File Stubs After Deletion (#12781)
* 🧹 fix: Prune Orphaned File References on File Deletion

Deleting a file via the Manage Files tab left its file_id in every agent's
tool_resources.*.file_ids. Stubs accumulate until the frontend dedupe keys
them as duplicates and blocks all new uploads (issue #12776).

- Add removeAgentResourceFilesFromAllAgents in packages/data-schemas: a
  single updateMany/$pullAll across every EToolResources category.
- Invoke it from processDeleteRequest after db.deleteFiles so every
  referencing agent is cleaned up, not just the one passed in req.body.
- Wrap the cleanup in try/catch so a stale agent update cannot mask a
  successful file deletion.

* 🧼 fix: Prune Orphaned File References on Agent Update

Already-affected agents would stay broken even after the delete-time fix:
the stubs sit on the agent document until something strips them. Heal them
on the next save (issue #12776).

- Add collectToolResourceFileIds + stripFileIdsFromToolResources helpers
  in @librechat/api — centralizing the tool_resources traversal used by
  the controller and the follow-up migration script.
- In updateAgentHandler, check the effective tool_resources against the
  files collection. When orphans are found, either strip them from the
  incoming tool_resources (if the update sets them) or run the bulk
  cleanup (if the update leaves tool_resources untouched).

* 🧰 chore: Add Migration to Clean Up Orphaned Agent File References

Complements the delete-time and save-time fixes by healing agents that
already accumulated orphan stubs before the upgrade (issue #12776). The
script is idempotent — re-running it on a clean database is a no-op.

- Add config/migrate-orphaned-agent-files.js following the existing
  migrate-*.js convention: --dry-run by default omitted (writes by
  default) and --batch-size= tuning knob. Streams agents via cursor.
- Register migrate:orphaned-agent-files and :dry-run npm scripts.
- Reuse collectToolResourceFileIds from @librechat/api so migration and
  runtime share the same traversal logic.

* 🩹 fix: Address Codex/Copilot Review on Orphaned Agent File Cleanup

Refines the #12776 fix series based on automated review feedback.

- Scope save-time pruning to the current agent only. When a PATCH
  carries tool_resources, strip orphans from the incoming payload and
  pay the DB round-trip only then. Removes the collection-wide
  updateMany previously triggered when tool_resources was absent
  (Codex P2 / Copilot).
- Wrap the orphan check in try/catch so a transient db.getFiles
  failure can't turn a good save into a 500 (comprehensive review #1).
- Replace Object.values(EToolResources) casts with an explicit list of
  agent-side categories in both orphans.ts and agent.ts. code_interpreter
  belongs to the Assistants API and isn't a key of AgentToolResources —
  including it was a type lie and generated dead MongoDB clauses
  (comprehensive review #3, #8).
- Export TOOL_RESOURCE_KEYS from @librechat/api and consume it in the
  migration script, dropping one duplicated definition (#4).
- Cap migration results.details at 50 sample entries so the memory
  footprint stays bounded on deployments with thousands of corrupted
  agents (Codex P3).
- Add migrate:orphaned-agent-files:batch npm script to match the
  convention set by migrate-agent-permissions / migrate-prompt-permissions
  (#7).
- Add controller-level tests covering the three orphan-pruning paths:
  strip from incoming tool_resources, leave alone when tool_resources
  is absent, swallow db.getFiles errors and still save (#6).
- Back pre-existing "should validate tool_resources in updates" test's
  file_ids with real File docs — the new pruning would otherwise strip
  them, and that test is about OCR conversion / schema filtering, not
  file existence. Register the File model in beforeAll so the fixture
  works.

* 🩹 fix: Tighten TOOL_RESOURCE_KEYS Type and Align Migration Sample Output

Two follow-ups from the second review pass.

- Type data-schemas' TOOL_RESOURCE_KEYS as ReadonlyArray<keyof
  AgentToolResources> instead of readonly string[]. Data-schemas depends
  on data-provider, so the import is clean. Catches typos and aligns
  with the matching export in @librechat/api — doesn't guarantee
  exhaustiveness, but that's a TypeScript limitation, not a workspace
  one.
- Align the migration's console output with DETAIL_SAMPLE_LIMIT: print
  every collected detail (up to 50) and, when more agents were affected
  than the sample size allowed, show a truncation notice. The old hard
  cap of 25 meant affected agents in the 26-50 range were collected
  but never shown.

* ✅ test: Add Integration Coverage for Orphan Cleanup Paths (#12776)

Exercise the delete-time and migration paths end-to-end against a real
in-memory Mongo. Catches integration bugs the isolated unit tests on
each layer couldn't.

- api/server/services/Files/process.integration.spec.js — the primary
  repro: seed an Agent + File, call processDeleteRequest, assert the
  file_id disappears from every referencing agent's tool_resources
  while unrelated agents stay untouched. Also covers the no-op case
  and confirms a failure in the new cleanup step cannot roll back the
  file deletion itself.
- api/test/migrate-orphaned-agent-files.spec.js — drives the migration
  module: --dry-run reports without writing, apply mode prunes across
  every tool_resource category, re-running is idempotent, and
  DETAIL_SAMPLE_LIMIT caps the in-memory sample on wide corruption.
  Mocks only the connect helper (the spec owns the mongoose instance)
  so the real migration code path — cursor, $pullAll, reduce — runs.

* 🔒 fix: Run Orphan Cleanup Migration in System Tenant Context

Codex P2 catch: under TENANT_ISOLATION_STRICT=true, the migration
throws on the very first Agent.countDocuments() because the tenant
isolation plugin fail-closes on queries without tenant context — which
makes migrate:orphaned-agent-files unusable on the exact deployments
most likely to have accumulated corruption.

- Wrap the scan/prune body in runAsSystem so queries bypass the tenant
  filter (SYSTEM_TENANT_ID sentinel). The migration legitimately needs
  cross-tenant visibility — this is the same pattern seedDatabase and
  the S3 refresh job already use.
- Add a regression test that spies on Agent.countDocuments() and
  asserts the active tenantStorage context is SYSTEM_TENANT_ID during
  the call. Pins the wrap against future regressions without the
  brittleness of toggling the strict-mode env var (which caches on
  first read).

Note: the delete-time and save-time paths already run inside an
authenticated HTTP request where tenantStorage.run is set by auth
middleware, so the cleanup naturally scopes to the current tenant —
which is the correct behavior there since file ownership is
tenant-scoped.

* 🧹 chore: Drop Unused path Import From Process Integration Spec

Leftover from an earlier iteration that resolved the migration path
via path.resolve before I switched to a relative require. The import
does nothing now — removing it.
2026-04-22 11:35:48 -07:00
Danny Avila
8ba2bde5c1
📦 refactor: Consolidate DB models, encapsulating Mongoose usage in data-schemas (#11830)
* chore: move database model methods to /packages/data-schemas

* chore: add TypeScript ESLint rule to warn on unused variables

* refactor: model imports to streamline access

- Consolidated model imports across various files to improve code organization and reduce redundancy.
- Updated imports for models such as Assistant, Message, Conversation, and others to a unified import path.
- Adjusted middleware and service files to reflect the new import structure, ensuring functionality remains intact.
- Enhanced test files to align with the new import paths, maintaining test coverage and integrity.

* chore: migrate database models to packages/data-schemas and refactor all direct Mongoose Model usage outside of data-schemas

* test: update agent model mocks in unit tests

- Added `getAgent` mock to `client.test.js` to enhance test coverage for agent-related functionality.
- Removed redundant `getAgent` and `getAgents` mocks from `openai.spec.js` and `responses.unit.spec.js` to streamline test setup and reduce duplication.
- Ensured consistency in agent mock implementations across test files.

* fix: update types in data-schemas

* refactor: enhance type definitions in transaction and spending methods

- Updated type definitions in `checkBalance.ts` to use specific request and response types.
- Refined `spendTokens.ts` to utilize a new `SpendTxData` interface for better clarity and type safety.
- Improved transaction handling in `transaction.ts` by introducing `TransactionResult` and `TxData` interfaces, ensuring consistent data structures across methods.
- Adjusted unit tests in `transaction.spec.ts` to accommodate new type definitions and enhance robustness.

* refactor: streamline model imports and enhance code organization

- Consolidated model imports across various controllers and services to a unified import path, improving code clarity and reducing redundancy.
- Updated multiple files to reflect the new import structure, ensuring all functionalities remain intact.
- Enhanced overall code organization by removing duplicate import statements and optimizing the usage of model methods.

* feat: implement loadAddedAgent and refactor agent loading logic

- Introduced `loadAddedAgent` function to handle loading agents from added conversations, supporting multi-convo parallel execution.
- Created a new `load.ts` file to encapsulate agent loading functionalities, including `loadEphemeralAgent` and `loadAgent`.
- Updated the `index.ts` file to export the new `load` module instead of the deprecated `loadAgent`.
- Enhanced type definitions and improved error handling in the agent loading process.
- Adjusted unit tests to reflect changes in the agent loading structure and ensure comprehensive coverage.

* refactor: enhance balance handling with new update interface

- Introduced `IBalanceUpdate` interface to streamline balance update operations across the codebase.
- Updated `upsertBalanceFields` method signatures in `balance.ts`, `transaction.ts`, and related tests to utilize the new interface for improved type safety.
- Adjusted type imports in `balance.spec.ts` to include `IBalanceUpdate`, ensuring consistency in balance management functionalities.
- Enhanced overall code clarity and maintainability by refining type definitions related to balance operations.

* feat: add unit tests for loadAgent functionality and enhance agent loading logic

- Introduced comprehensive unit tests for the `loadAgent` function, covering various scenarios including null and empty agent IDs, loading of ephemeral agents, and permission checks.
- Enhanced the `initializeClient` function by moving `getConvoFiles` to the correct position in the database method exports, ensuring proper functionality.
- Improved test coverage for agent loading, including handling of non-existent agents and user permissions.

* chore: reorder memory method exports for consistency

- Moved `deleteAllUserMemories` to the correct position in the exported memory methods, ensuring a consistent and logical order of method exports in `memory.ts`.
2026-03-21 14:28:53 -04:00
Danny Avila
58f128bee7
🗑️ chore: Remove Deprecated Project Model and Associated Fields (#11773)
* chore: remove projects and projectIds usage

* chore: empty line linting

* chore: remove isCollaborative property across agent models and related tests

- Removed the isCollaborative property from agent models, controllers, and tests, as it is deprecated in favor of ACL permissions.
- Updated related validation schemas and data provider types to reflect this change.
- Ensured all references to isCollaborative were stripped from the codebase to maintain consistency and clarity.
2026-03-21 14:28:53 -04:00
Danny Avila
38521381f4
🐘 feat: FerretDB Compatibility (#11769)
* feat: replace unsupported MongoDB aggregation operators for FerretDB compatibility

Replace $lookup, $unwind, $sample, $replaceRoot, and $addFields aggregation
stages which are unsupported on FerretDB v2.x (postgres-documentdb backend).

- Prompt.js: Replace $lookup/$unwind/$project pipelines with find().select().lean()
  + attachProductionPrompts() batch helper. Replace $group/$replaceRoot/$sample
  in getRandomPromptGroups with distinct() + Fisher-Yates shuffle.
- Agent/Prompt migration scripts: Replace $lookup anti-join pattern with
  distinct() + $nin two-step queries for finding un-migrated resources.

All replacement patterns verified against FerretDB v2.7.0.

* fix: use $pullAll for simple array removals, fix memberIds type mismatches

Replace $pull with $pullAll for exact-value scalar array removals. Both
operators work on MongoDB and FerretDB, but $pullAll is more explicit for
exact matching (no condition expressions).

Fix critical type mismatch bugs where ObjectId values were used against
String[] memberIds arrays in Group queries:
- config/delete-user.js: use string uid instead of ObjectId user._id
- e2e/setup/cleanupUser.ts: convert userId.toString() before query

Harden PermissionService.bulkUpdateResourcePermissions abort handling to
prevent crash when abortTransaction is called after commitTransaction.

All changes verified against FerretDB v2.7.0 and MongoDB Memory Server.

* fix: harden transaction support probe for FerretDB compatibility

Commit the transaction before aborting in supportsTransactions probe, and
wrap abortTransaction in try-catch to prevent crashes when abort is called
after a successful commit (observed behavior on FerretDB).

* feat: add FerretDB compatibility test suite, retry utilities, and CI config

Add comprehensive FerretDB integration test suite covering:
- $pullAll scalar array operations
- $pull with subdocument conditions
- $lookup replacement (find + manual join)
- $sample replacement (distinct + Fisher-Yates)
- $bit and $bitsAllSet operations
- Migration anti-join pattern
- Multi-tenancy (useDb, scaling, write amplification)
- Sharding proof-of-concept
- Production operations (backup/restore, schema migration, deadlock retry)

Add production retryWithBackoff utility for deadlock recovery during
concurrent index creation on FerretDB/DocumentDB backends.

Add UserController.spec.js tests for deleteUserController (runs in CI).

Configure jest and eslint to isolate FerretDB tests from CI pipelines:
- packages/data-schemas/jest.config.mjs: ignore misc/ directory
- eslint.config.mjs: ignore packages/data-schemas/misc/

Include Docker Compose config for local FerretDB v2.7 + postgres-documentdb,
dedicated jest/tsconfig for the test files, and multi-tenancy findings doc.

* style: brace formatting in aclEntry.ts modifyPermissionBits

* refactor: reorganize retry utilities and update imports

- Moved retryWithBackoff utility to a new file `retry.ts` for better structure.
- Updated imports in `orgOperations.ferretdb.spec.ts` to reflect the new location of retry utilities.
- Removed old import statement for retryWithBackoff from index.ts to streamline exports.

* test: add $pullAll coverage for ConversationTag and PermissionService

Add integration tests for deleteConversationTag verifying $pullAll
removes tags from conversations correctly, and for
syncUserEntraGroupMemberships verifying $pullAll removes user from
non-matching Entra groups while preserving local group membership.

---------
2026-03-21 14:28:49 -04:00
Danny Avila
8b18a16446
🏷️ chore: Remove Docker Images by Named Tag in deployed-update.js (#12138)
* fix: remove docker images by named tag instead of image ID

* refactor: Simplify rebase logic and enhance error handling in deployed-update script

- Removed unnecessary condition for rebasing, streamlining the update process.
- Renamed variable for clarity when fetching Docker image tags.
- Added error handling to catch and log failures during the update process, ensuring better visibility of issues.
2026-03-08 21:48:22 -04:00
Danny Avila
9956a72694
🧭 fix: Subdirectory Deployment Auth Redirect Path Doubling (#12077)
* fix: subdirectory redirects

* fix: use path-segment boundary check when stripping BASE_URL prefix

A bare `startsWith(BASE_URL)` matches on character prefix, not path
segments. With BASE_URL="/chat", a path like "/chatroom/c/abc" would
incorrectly strip to "room/c/abc" (no leading slash). Guard with an
exact-match-or-slash check: `p === BASE_URL || p.startsWith(BASE_URL + '/')`.

Also removes the dead `BASE_URL !== '/'` guard — module init already
converts '/' to ''.

* test: add path-segment boundary tests and clarify subdirectory coverage

- Add /chatroom, /chatbot, /app/chatroom regression tests to verify
  BASE_URL stripping only matches on segment boundaries
- Clarify useAuthRedirect subdirectory test documents React Router
  basename behavior (BASE_URL stripping tested in api-endpoints-subdir)
- Use `delete proc.browser` instead of undefined assignment for cleanup
- Add rationale to eslint-disable comment for isolateModules require

* fix: use relative path and correct instructions in subdirectory test script

- Replace hardcoded /home/danny/LibreChat/.env with repo-root-relative
  path so the script works from any checkout location
- Update instructions to use production build (npm run build && npm run
  backend) since nginx proxies to :3080 which only serves the SPA after
  a full build, not during frontend:dev on :3090

* fix: skip pointless redirect_to=/ for root path and fix jsdom 26+ compat

buildLoginRedirectUrl now returns plain /login when the resolved path
is root — redirect_to=/ adds no value since / immediately redirects
to /c/new after login anyway.

Also rewrites api-endpoints.spec.ts to use window.history.replaceState
instead of Object.defineProperty(window, 'location', ...) which jsdom
26+ no longer allows.

* test: fix request-interceptor.spec.ts for jsdom 26+ compatibility

Switch from jsdom to happy-dom environment which allows
Object.defineProperty on window.location. jsdom 26+ made
location non-configurable, breaking all 8 tests in this file.

* chore: update browser property handling in api-endpoints-subdir test

Changed the handling of the `proc.browser` property from deletion to setting it to false, ensuring compatibility with the current testing environment.

* chore: update backend restart instructions in test subdirectory setup script

Changed the instruction for restarting the backend from "npm run backend:dev" to "npm run backend" to reflect the correct command for the current setup.

* refactor: ensure proper cleanup in loadModuleWithBase function

Wrapped the module loading logic in a try-finally block to guarantee that the `proc.browser` property is reset to false and the base element is removed, improving reliability in the testing environment.

* refactor: improve browser property handling in loadModuleWithBase function

Revised the management of the `proc.browser` property to store the original value before modification, ensuring it is restored correctly after module loading. This enhances the reliability of the testing environment.
2026-03-05 01:38:44 -05:00
Danny Avila
9b3152807b
🐳 chore: Update image registry references in Docker/Helm configurations (#12026)
- Changed image references from `ghcr.io` to `registry.librechat.ai` across multiple Docker and Helm files, ensuring consistency in image sourcing.
- Updated `deploy-compose.yml`, `docker-compose.override.yml.example`, `docker-compose.yml`, `rag.yml`, and various Helm chart files to reflect the new registry.
2026-03-02 22:14:50 -05:00
Danny Avila
c3da148fa0
📝 docs: Add AGENTS.md for Project Structure and Coding Standards (#11866)
* 📝 docs: Add AGENTS.md for project structure and coding standards

- Introduced AGENTS.md to outline project workspaces, coding standards, and development commands.
- Defined workspace boundaries for backend and frontend development, emphasizing TypeScript usage.
- Established guidelines for code style, iteration performance, type safety, and import order.
- Updated CONTRIBUTING.md to reference AGENTS.md for coding standards and project conventions.
- Modified package.json to streamline build commands, consolidating frontend and backend build processes.

* chore: Update build commands and improve smart reinstall process

- Modified AGENTS.md to clarify the purpose of `npm run smart-reinstall` and other build commands, emphasizing Turborepo's role in dependency management and builds.
- Updated package.json to streamline build commands, replacing the legacy frontend build with a Turborepo-based approach for improved performance.
- Enhanced the smart reinstall script to fully delegate build processes to Turborepo, including cache management and dependency checks, ensuring a more efficient build workflow.
2026-02-19 16:33:43 -05:00
Danny Avila
e50f59062f
🏎️ feat: Smart Reinstall with Turborepo Caching for Better DX (#11785)
* chore: Add Turborepo support and smart reinstall script

- Updated .gitignore to include Turborepo cache directory.
- Added Turbo as a dependency in package.json and package-lock.json.
- Introduced turbo.json configuration for build tasks.
- Created smart-reinstall.js script to optimize dependency installation and package builds using Turborepo caching.

* fix: Address PR review feedback for smart reinstall

  - Fix Windows compatibility in hasTurbo() by checking for .cmd/.ps1 shims
  - Remove Unix-specific shell syntax (> /dev/null 2>&1) from cache clearing
  - Split try/catch blocks so daemon stop failure doesn't block cache clear
  - Add actionable tips in error output pointing to --force and --verbose
2026-02-13 14:25:26 -05:00
Danny Avila
6279ea8dd7
🛸 feat: Remote Agent Access with External API Support (#11503)
* 🪪 feat: Microsoft Graph Access Token Placeholder for MCP Servers (#10867)

* feat: MCP Graph Token env var

* Addressing copilot remarks

* Addressed Copilot review remarks

* Fixed graphtokenservice mock in MCP test suite

* fix: remove unnecessary type check and cast in resolveGraphTokensInRecord

* ci: add Graph Token integration tests in MCPManager

* refactor: update user type definitions to use Partial<IUser> in multiple functions

* test: enhance MCP tests for graph token processing and user placeholder resolution

- Added comprehensive tests to validate the interaction between preProcessGraphTokens and processMCPEnv.
- Ensured correct resolution of graph tokens and user placeholders in various configurations.
- Mocked OIDC utilities to facilitate testing of token extraction and validation.
- Verified that original options remain unchanged after processing.

* chore: import order

* chore: imports

---------

Co-authored-by: Danny Avila <danny@librechat.ai>

* WIP: OpenAI-compatible API for LibreChat agents

- Added OpenAIChatCompletionController for handling chat completions.
- Introduced ListModelsController and GetModelController for listing and retrieving agent details.
- Created routes for OpenAI API endpoints, including /v1/chat/completions and /v1/models.
- Developed event handlers for streaming responses in OpenAI format.
- Implemented request validation and error handling for API interactions.
- Integrated content aggregation and response formatting to align with OpenAI specifications.

This commit establishes a foundational API for interacting with LibreChat agents in a manner compatible with OpenAI's chat completion interface.

* refactor: OpenAI-spec content aggregation for improved performance and clarity

* fix: OpenAI chat completion controller with safe user handling for correct tool loading

* refactor: Remove conversation ID from OpenAI response context and related handlers

* refactor: OpenAI chat completion handling with streaming support

- Introduced a lightweight tracker for streaming responses, allowing for efficient tracking of emitted content and usage metadata.
- Updated the OpenAIChatCompletionController to utilize the new tracker, improving the handling of streaming and non-streaming responses.
- Refactored event handlers to accommodate the new streaming logic, ensuring proper management of tool calls and content aggregation.
- Adjusted response handling to streamline error reporting during streaming sessions.

* WIP: Open Responses API with core service, types, and handlers

- Added Open Responses API module with comprehensive types and enums.
- Implemented core service for processing requests, including validation and input conversion.
- Developed event handlers for streaming responses and non-streaming aggregation.
- Established response building logic and error handling mechanisms.
- Created detailed types for input and output content, ensuring compliance with Open Responses specification.

* feat: Implement response storage and retrieval in Open Responses API

- Added functionality to save user input messages and assistant responses to the database when the `store` flag is set to true.
- Introduced a new endpoint to retrieve stored responses by ID, allowing users to access previous interactions.
- Enhanced the response creation process to include database operations for conversation and message storage.
- Implemented tests to validate the storage and retrieval of responses, ensuring correct behavior for both existing and non-existent response IDs.

* refactor: Open Responses API with additional token tracking and validation

- Added support for tracking cached tokens in response usage, improving token management.
- Updated response structure to include new properties for top log probabilities and detailed usage metrics.
- Enhanced tests to validate the presence and types of new properties in API responses, ensuring compliance with updated specifications.
- Refactored response handling to accommodate new fields and improve overall clarity and performance.

* refactor: Update reasoning event handlers and types for consistency

- Renamed reasoning text events to simplify naming conventions, changing `emitReasoningTextDelta` to `emitReasoningDelta` and `emitReasoningTextDone` to `emitReasoningDone`.
- Updated event types in the API to reflect the new naming, ensuring consistency across the codebase.
- Added `logprobs` property to output events for enhanced tracking of log probabilities.

* feat: Add validation for streaming events in Open Responses API tests

* feat: Implement response.created event in Open Responses API

- Added emitResponseCreated function to emit the response.created event as the first event in the streaming sequence, adhering to the Open Responses specification.
- Updated createResponse function to emit response.created followed by response.in_progress.
- Enhanced tests to validate the order of emitted events, ensuring response.created is triggered before response.in_progress.

* feat: Responses API with attachment event handling

- Introduced `createResponsesToolEndCallback` to handle attachment events in the Responses API, emitting `librechat:attachment` events as per the Open Responses extension specification.
- Updated the `createResponse` function to utilize the new callback for processing tool outputs and emitting attachments during streaming.
- Added helper functions for writing attachment events and defined types for attachment data, ensuring compatibility with the Open Responses protocol.
- Enhanced tests to validate the integration of attachment events within the Responses API workflow.

* WIP: remote agent auth

* fix: Improve loading state handling in AgentApiKeys component

- Updated the rendering logic to conditionally display loading spinner and API keys based on the loading state.
- Removed unnecessary imports and streamlined the component for better readability.

* refactor: Update API key access handling in routes

- Replaced `checkAccess` with `generateCheckAccess` for improved access control.
- Consolidated access checks into a single `checkApiKeyAccess` function, enhancing code readability and maintainability.
- Streamlined route definitions for creating, listing, retrieving, and deleting API keys.

* fix: Add permission handling for REMOTE_AGENT resource type

* feat: Enhance permission handling for REMOTE_AGENT resources

- Updated the deleteAgent and deleteUserAgents functions to handle permissions for both AGENT and REMOTE_AGENT resource types.
- Introduced new functions to enrich REMOTE_AGENT principals and backfill permissions for AGENT owners.
- Modified createAgentHandler and duplicateAgentHandler to grant permissions for REMOTE_AGENT alongside AGENT.
- Added utility functions for retrieving effective permissions for REMOTE_AGENT resources, ensuring consistent access control across the application.

* refactor: Rename and update roles for remote agent access

- Changed role name from API User to Editor in translation files for clarity.
- Updated default editor role ID from REMOTE_AGENT_USER to REMOTE_AGENT_EDITOR in resource configurations.
- Adjusted role localization to reflect the new Editor role.
- Modified access permissions to align with the updated role definitions across the application.

* feat: Introduce remote agent permissions and update access handling

- Added support for REMOTE_AGENTS in permission schemas, including use, create, share, and share_public permissions.
- Updated the interface configuration to include remote agent settings.
- Modified middleware and API key access checks to align with the new remote agent permission structure.
- Enhanced role defaults to incorporate remote agent permissions, ensuring consistent access control across the application.

* refactor: Update AgentApiKeys component and permissions handling

- Refactored the AgentApiKeys component to improve structure and readability, including the introduction of ApiKeysContent for better separation of concerns.
- Updated CreateKeyDialog to accept an onKeyCreated callback, enhancing its functionality.
- Adjusted permission checks in Data component to use REMOTE_AGENTS and USE permissions, aligning with recent permission schema changes.
- Enhanced loading state handling and dialog management for a smoother user experience.

* refactor: Update remote agent access checks in API routes

- Replaced existing access checks with `generateCheckAccess` for remote agents in the API keys and agents routes.
- Introduced specific permission checks for creating, listing, retrieving, and deleting API keys, enhancing access control.
- Improved code structure by consolidating permission handling for remote agents across multiple routes.

* fix: Correct query parameters in ApiKeysContent component

- Updated the useGetAgentApiKeysQuery call to include an object for the enabled parameter, ensuring proper functionality when the component is open.
- This change improves the handling of API key retrieval based on the component's open state.

* feat: Implement remote agents permissions and update API routes

- Added new API route for updating remote agents permissions, enhancing role management capabilities.
- Introduced remote agents permissions handling in the AgentApiKeys component, including a dedicated settings dialog.
- Updated localization files to include new remote agents permission labels for better user experience.
- Refactored data provider to support remote agents permissions updates, ensuring consistent access control across the application.

* feat: Add remote agents permissions to role schema and interface

- Introduced new permissions for REMOTE_AGENTS in the role schema, including USE, CREATE, SHARE, and SHARE_PUBLIC.
- Updated the IRole interface to reflect the new remote agents permissions structure, enhancing role management capabilities.

* feat: Add remote agents settings button to API keys dialog

* feat: Update AgentFooter to include remote agent sharing permissions

- Refactored access checks to incorporate permissions for sharing remote agents.
- Enhanced conditional rendering logic to allow sharing by users with remote agent permissions.
- Improved loading state handling for remote agent permissions, ensuring a smoother user experience.

* refactor: Update API key creation access check and localization strings

- Replaced the access check for creating API keys to use the existing remote agents access check.
- Updated localization strings to correct the descriptions for remote agent permissions, ensuring clarity in user interface.

* fix: resource permission mapping to include remote agents

- Changed the resourceToPermissionMap to use a Partial<Record> for better flexibility.
- Added mapping for REMOTE_AGENT permissions, enhancing the sharing capabilities for remote agents.

* feat: Implement remote access checks for agent models

- Enhanced ListModelsController and GetModelController to include checks for user permissions on remote agents.
- Integrated findAccessibleResources to filter agents based on VIEW permission for REMOTE_AGENT.
- Updated response handling to ensure users can only access agents they have permissions for, improving security and access control.

* fix: Update user parameter type in processUserPlaceholders function

- Changed the user parameter type in the processUserPlaceholders function from Partial<Partial<IUser>> to Partial<IUser> for improved type clarity and consistency.

* refactor: Simplify integration test structure by removing conditional describe

- Replaced conditional describeWithApiKey with a standard describe for all integration tests in responses.spec.js.
- This change enhances test clarity and ensures all tests are executed consistently, regardless of the SKIP_INTEGRATION_TESTS flag.

* test: Update AgentFooter tests to reflect new grant access dialog ID

- Changed test IDs for the grant access dialog in AgentFooter tests to include the resource type, ensuring accurate identification in the test cases.
- This update improves test clarity and aligns with recent changes in the component's implementation.

* test: Enhance integration tests for Open Responses API

- Updated integration tests in responses.spec.js to utilize an authRequest helper for consistent authorization handling across all test cases.
- Introduced a test user and API key creation to improve test setup and ensure proper permission checks for remote agents.
- Added checks for existing access roles and created necessary roles if they do not exist, enhancing test reliability and coverage.

* feat: Extend accessRole schema to include remoteAgent resource type

- Updated the accessRole schema to add 'remoteAgent' to the resourceType enum, enhancing the flexibility of role assignments and permissions management.

* test: refactored test setup to create a minimal Express app for responses routes, enhancing test structure and maintainability.

* test: Enhance abort.spec.js by mocking additional modules for improved test isolation

- Updated the test setup in abort.spec.js to include actual implementations of '@librechat/data-schemas' and '@librechat/api' while maintaining mock functionality.
- This change improves test reliability and ensures that the tests are more representative of the actual module behavior.

* refactor: Update conversation ID generation to use UUID

- Replaced the nanoid with uuidv4 for generating conversation IDs in the createResponse function, enhancing uniqueness and consistency in ID generation.

* test: Add remote agent access roles to AccessRole model tests

- Included additional access roles for remote agents (REMOTE_AGENT_EDITOR, REMOTE_AGENT_OWNER, REMOTE_AGENT_VIEWER) in the AccessRole model tests to ensure comprehensive coverage of role assignments and permissions management.

* chore: Add deletion of user agent API keys in user deletion process

- Updated the user deletion process in UserController and delete-user.js to include the removal of user agent API keys, ensuring comprehensive cleanup of user data upon account deletion.

* test: Add remote agents permissions to permissions.spec.ts

- Enhanced the permissions tests by including comprehensive permission settings for remote agents across various scenarios, ensuring accurate validation of access controls for remote agent roles.

* chore: Update remote agents translations for clarity and consistency

- Removed outdated remote agents translation entries and added revised entries to improve clarity on API key creation and sharing permissions for remote agents. This enhances user understanding of the available functionalities.

* feat: Add indexing and TTL for agent API keys

- Introduced an index on the `key` field for improved query performance.
- Added a TTL index on the `expiresAt` field to enable automatic cleanup of expired API keys, ensuring efficient management of stored keys.

* chore: Update API route documentation for clarity

- Revised comments in the agents route file to clarify the handling of API key authentication.
- Removed outdated endpoint listings to streamline the documentation and focus on current functionality.

---------

Co-authored-by: Max Sanna <max@maxsanna.com>
2026-01-28 17:44:33 -05:00
Shahryar Tayeb
639a60cf19
📬 fix: Email Verification Handling in Create-User Command (#11408)
* fix:  email verification handling in create-user command

* set emailVerified to true when the input is 'y'

* normalize email verification input and set emailVerified to true by default
2026-01-21 14:03:49 -05:00
Danny Avila
28270bec58
🌵 chore: Remove deprecated 'prompt-caching' Anthropic header (#11313) 2026-01-12 19:12:36 -05:00
Andrei Blizorukov
7d136edb40
🔧 refactor: batching documents on meili index reset (#11165)
* 🔧 refactor: batching documents on meili index reset

Update on all documents can be very heavy on weak or low-tier instances

🔧 refactor: check if flag is enabled before calling meilisearch

🔧 fix: adding index to query documents to reset meili-search index status

* 🔧 refactor: error handling

🔧 refactor: more unit-test coverage

* 🔧 refactor: edge case error handling & tests
2026-01-02 10:50:06 -05:00
Danny Avila
f9060fa25f
🔧 chore: Update ESLint Config & Run Linter (#10986) 2025-12-15 17:55:25 -05:00
Marco Beretta
5b3cef6d86
📌 feat: Add Support for Persistable (Non-Dismissible) Banners (#10730)
* feat: Add persistable property to banners and update related components

* refactor: Clean up Banner component and improve className handling
2025-12-11 16:37:22 -05:00
Linus Gasser
872dbb4151
🪙 refactor: Remove Use of CHECK_BALANCE in Balance Scripts (#10702)
config/set_balance.js and config/add_balance.js still use the CHECK_BALANCE variable.
This PR makes them use the getBalanceConfig from the api.
2025-11-28 11:11:26 -05:00
WhammyLeaf
846e34b1d7
🗑️ fix: Remove All User Metadata on Deletion (#10534)
* remove all user metadata on deletion

* chore: import order

* fix: Update JSDoc types for deleteMessages function parameters and return value

* fix: Enhance user deletion process by removing associated data and updating group memberships

* fix: Add missing config middleware to user deletion route

* fix: Refactor agent and prompt deletion processes to bulk delete and remove associated ACL entries

* fix: Add deletion of OAuth tokens and ACL entries in user deletion process

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2025-11-21 12:03:26 -05:00
Danny Avila
b49545d916
🪂 refactor: MCP Server Init Fallback (#10608)
* 🌿 refactor: MCP Server Init and Registry with Fallback Configs

* chore: Redis Cache Flushing for Cluster Support
2025-11-20 16:47:00 -05:00