Commit graph

135 commits

Author SHA1 Message Date
Marco Beretta
5b8bb3c31f
⏱️ fix: Bound Remaining Outbound Calls With Timeout Budgets (#17468)
* fix: Bound outbound HTTP calls that had no timeout

Add timeouts.toolsMs, speechMs and imageGenerationMs to the timeouts block and apply the
existing budgets to the RAG delete paths, built-in tool APIs, TTS/STT providers (TTS also
aborts when the client closes), Stable Diffusion, Firebase reads and Mailgun. Calls made
before app config exists (remote config, RAG health, service key URL, Google refresh) get a
fixed 10 second budget.

* fix: Honor configured budgets for RAG deletes and per-chunk TTS deadlines

* refactor: Share one budget helper between speech and image CJS callers

* test: Cover a stalled RAG delete with a mock scenario

* test: Wait for the RAG embed before stalling the delete

* test: Embed at upload so the stalled delete has a vector to remove

* fix: Close a started TTS response on a late chunk timeout and share the service budget helper

* fix: Fail startup when the remote config fetch times out
2026-10-29 09:55:18 +01:00
lia-by-librechat[bot]
68826ec343
🛼 fix: Bound Document Name Collision Processing (#17386)
Co-authored-by: Lia <lia@librechat.ai>
2026-10-23 15:44:18 +02:00
Marco Beretta
5fc4371c6a
🪦 refactor: Remove Dead Legacy API Modules and Root Tooling (#17114) 2026-10-15 12:25:24 +02:00
Danny Avila
b2bcbf9380
🧯 feat: Recover Conversations Whose Attachments Keep Failing (#16965)
* 🧯 feat: Recover Conversations Whose Attachments Keep Failing

Once a branch carries a problem attachment, every turn resends it and
fails the same way. A failed response whose branch carries files now
offers "Retry without files" (re-asks the failed turn with no files as a
new version) and, when earlier turns carry files, "Branch without files"
(a copy up to the last good answer with every upload and code output
left out, with the failed message restored to the composer). File-shaped
errors say so in the card.

The edit view shows a user message's files as removable chips; "Update &
rerun" sends only the kept files and Save is disabled after a removal.
Fork is offered on a failed turn and gains a "Leave out files" option,
backed by a new `excludeFiles` fork option that drops `files`,
`attachments` and the conversation's `files` from the copy.

Bedrock rejects a request whose documents share a name, and history
resends every earlier document, so repeated names are now numbered
before formatting.

* 🧯 fix: Cover Steered Files, Hand Off the Whole Failed Turn, Name Replayed Documents Once

- Excluding files from a fork also strips attachments steered into a
  response (`content[].files` on steer parts), which replay like uploads,
  and the decision lives in `forkFileScope` in packages/api.
- The recovery card counts steered attachments as files, and reads the
  specific code inside a generic `error` envelope, so a rate limit is not
  offered files as the fix.
- "Branch without files" hands the failed turn to the copy's composer the
  way a message is put back for editing: text through the in-memory
  handoff, quotes, manual skills and the reasoning override through their
  compose-time atoms.
- Document names are deduplicated over the formatted payload after the
  steer stamp, so replayed steer documents are numbered too.
- Fork stays withheld on a failed response, whose copy would carry the
  error as an ordinary answer; the card's branch action copies up to the
  turn before.
- Remove-file buttons are natively disabled while the editor is locked.

* 🧯 fix: Keep Search Sources, Scope Root Retries, Skip Legacy Rate Limits

- A fork without files keeps metadata-only attachments (search sources,
  file search results) and drops only file-backed ones; the recovery card
  likewise counts only file-backed attachments as files.
- A turn sent at the root takes none of the conversation's file refs: its
  siblings' uploads (an edited first message, or one retried without its
  files) no longer come back through the unanchored-continuation fallback.
- Rows that carry a legacy LangChain model, key or rate-limit code are not
  offered files as the fix.

* 🧯 fix: Align File Exclusion With Replay's File Shapes, Skip Assistants, Verify the Copy

- Shared `messageCarriesFiles` (with `isFileContentPart`,
  `hasContentPartFiles`, `isFileAttachment`) in data-provider names every
  shape historical replay reads a file by: uploads, file-backed
  attachments, attached `files` on any content part, and `image_file`,
  `file` or direct `file_id` locators. The fork drops all of them and the
  recovery card detects all of them.
- Assistants conversations get no recovery actions: their upstream thread
  keeps every attachment, and those endpoints offer no fork either.
- "Branch without files" checks the returned copy; a server that ignored
  `excludeFiles` gets an error toast instead of a false recovery.

* 🧯 fix: Type Replay's Provider-Native File Locators for the Part Predicates
2026-10-10 08:43:39 -04:00
Lia
c41413ad21 fix: Keep Manual Unread Reminders in Date Groups 2026-10-03 17:56:52 -04:00
lia-by-librechat[bot]
73d704656e
📔 feat: Preserve Owner Text During PII Redaction (#16311)
* 📔 feat: Preserve Owner Text During PII Redaction

* 🧵 fix: Keep Ordinary Chat Views Outside Private Text Provider

* 🧷 fix: Preserve Owner Text Across Copy and Edit Paths

* 🔏 fix: Protect PII on Denial and Abort Paths

* fix: preserve protected text through denial, stop, and owner reload

* test: cover preliminary private metadata in disconnect fixture

* test: carry preliminary metadata in resume controller fixture

* test: preserve Stop prerequisite identity in tenant fixture

* fix: keep protected owner text out of search and exports

* fix: preserve trusted PII placeholders through provider inspection

* fix: coordinate protected turns across Stop and native sharing

* fix: Preserve Private Text Admission and Query Lifecycles

* fix: Track Private Query Effect Dependencies

* fix: Fence Protected Admission and Screenshot Targets

* fix: Share Protected Admission and Preserve Native Token Trust

* test: Wait for Private Message Concurrency Indexes

* fix: Preserve Protected Admission Across Recovery and Titles

* fix: Coordinate Protected Root Admission and Early Stop

---------

Co-authored-by: Lia <lia@librechat.ai>
2026-10-02 11:43:10 -04:00
ethanlaj
90526cdc16
📧 fix: Pass requireTLS So EMAIL_ENCRYPTION=starttls Requires STARTTLS (#16656)
The SMTP transport options set `requireTls`, but nodemailer only reads
`requireTLS`, so the key was ignored. With EMAIL_ENCRYPTION=starttls,
a server that does not offer STARTTLS received the message over an
unencrypted connection instead of the send being refused.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 08:39:04 -04:00
Marco Beretta
5b84c4b57f ⏳ feat: Add Ephemeral Retention Mode for Forced Temporary Chats (#13811)
* feat: add ephemeral retention mode for forced temporary chats

Adds RetentionMode.EPHEMERAL, which forces every newly saved chat
temporary: hidden from history and search, given the temporary-chat
expiration, and with the toggle locked on.

Forces isTemporary and applies the temporary expiration in the normal
conversation and message save methods, routes Assistants/Threads
messages through the normal message save, applies the all-data
retention policy to file uploads and shared links, and stamps imports,
forks, and duplicates. Locks the frontend toggle and keyboard shortcut
behind an administrator-enforced badge.

The default temporary mode and the existing all mode are unchanged.

* fix: enforce ephemeral retention when resuming paused chats

* fix: preserve forced retention across pauses and assistant turns

* test: Use Real Module IDs for Installed Package Mocks

* test: Wait for Resumed Retention Persistence

* test: Control Mongo Index Setup in OpenID Fixture

* test: Close Mobile Drawer Before Retention Resume Scenario

* fix: Store Responses API Messages Under the Retention Write Context

Input and output messages were saved with the raw request, so under
ephemeral retention they missed the forced-temporary classification the
conversation received. Route both through getConversationWriteContext
and announce the stored reply as saved.

* fix: Align Resumed Responses With the Parent Retention Deadline

A resumed response was saved with a fresh forced-temporary window while
the re-stamp kept the conversation's earlier deadline, so a chat that
paused again could leave its response stored after the conversation
expired. Stamp the saved response with the parent's deadline.

* fix: Clear Released Bookmark Tags When Forcing a Chat Temporary

stampForcedRetention released a converted chat's bookmark counts but
kept its tags, so deleting that chat before TTL removal decremented the
same tags again and reduced counts owned by other visible chats. Clear
the tags in the conditional conversion update that releases them.

* fix: Store No Bookmark Tags on Forced-Temporary Imports

An ephemeral import, fork or duplicate skipped its tag count increment
but kept the source tags, so deleting the copy before TTL removal
decremented counts owned by other visible chats. Store the same tags the
import counts, which is none for a forced-temporary copy.

* test: Provide the Forced-Temporary Retention Helper to the Header Stacking Mock

* test: Reuse One Admin Login per Worker in Ephemeral Retention Scenarios

* fix: Keep Legacy Temporary Chats Temporary When Forking Under All-Data Retention

* fix: Convert a Resumed Chat Under Ephemeral Retention Before It Can Re-Pause Empty

* fix: Treat a Reloaded Pre-Policy Run as Temporary Once Retention Is Forced

* fix: Stamp Messages with the Deadline a Concurrent Retention Stamp Stored

* fix: Wait for the Startup Config Before Rebuilding a Resumed Submission

* test: Wait for the Resumed Response Row in the Ephemeral Resume Scenario

* test: Provide the Real Command Executor Stamp to the Callbacks Mock

(cherry picked from commit 02a70e9c58)

Original-PR: https://github.com/LibreChat-AI/LibreChat/pull/13811
2026-10-01 03:46:19 +00:00
lia-by-librechat[bot]
55f3187296 🥢 fix: Fit MCP App Snapshots Within Message Documents (#16299)
* 🥢 fix: Fit MCP App Snapshots Within Message Documents

* 🥢 fix: Resolve Message Model Only at App Write Time

* 🥢 style: Sort MCP App Snapshot Persistence Imports

* 🥢 fix: Centralize MCP App Message Admission Across Writers

* 🥢 fix: Inject the App Budget Reader Without a Config Cycle

* test: Cover App Budget Initialization Across Mocked Hosts

---------

Co-authored-by: Lia <lia@librechat.ai>
(cherry picked from commit 411bb5444f)

Original-PR: https://github.com/LibreChat-AI/LibreChat/pull/16299
2026-10-01 03:46:17 +00:00
Marco Beretta
cd44344647 🔔 feat: Unseen Reply Indicators, Reply Notifications, and Mark as Unread (#15025)
* feat: add unseen reply indicators, reply notifications, and mark as unread

A reply that lands while the user is elsewhere now leaves a trace. The
conversation carries a durable reply stamp written with the assistant message
that backs it, the sidebar row shows an unread dot until that message is
actually rendered, and an optional tab badge, desktop notification and chime
announce arrivals in a tab that is away. Rows can also be marked unread by hand
from the row menu, in the archived view as well as the main list.

The stamp is written by a compare-and-swap so concurrent replies order on the
database rather than on the application hosts' clocks, and only once output that
a reader can actually see is persisted: a stopped turn with nothing to read, an
empty synchronized row, or a failed message write never lights a dot that cannot
be cleared. Read intent survives archives, metadata edits and preset applies,
and a later read always outranks a delayed unread.

* fix: gate reply alerts on config and announce only readable replies

Three review rounds kept finding the same two shapes, so both are answered once
rather than patched again.

The alert capabilities are now operator-configurable through
`interface.replyNotifications` in `librechat.yaml`: the tab badge, desktop
notifications and the chime each have a gate, and the away poll's page size is a
field rather than a constant, with defaults that reproduce the shipped behavior.
A capability the operator turns off reads as off whatever the device stored, and
its settings toggle is not offered.

Deciding whether a persisted turn may raise an indicator now lives in one place
in `packages/api`, with the Responses API controller, the abort route, the
Assistants thread sync and the client's override path all calling it instead of
carrying their own copy. That closes the last case where a reply nobody can read
lit a dot nothing could clear: a `/v1/responses` completion whose output held
only reasoning or tool calls persists an empty row, and stamping it left an
unread conversation that opening could not settle.

Also: the retention backfill no longer advances `updatedAt` past a reply stamp,
which an away tab reads as a metadata-only promotion and withholds the alert
for; arrival evidence is swept of conversations no cached list still holds, so a
long-lived tab stops growing with its lifetime number of replies; and generated
Lighthouse profile directories are ignored where the runner leaves them.

* test: open the sidebar by what the drawer paints, not by its test ids

The drawer became a mounted, sliding panel on canary, so the controls this
helper read no longer say what it assumed. `close-sidebar-button` answers from
the off-canvas copy before the expanded state commits, which returned the helper
with the conversation list still hidden behind the pane, and the panel's own
`new-chat-button` has a rail copy that a plain locator reaches first. Both
readings passed on the previous drawer, which unmounted when closed.

Settling on the close control's accessible name, and on the first candidate that
is actually painted, is what a reader perceives and survives either drawer. One
click is one toggle and the opener sits in the pane the drawer marks inert for
the whole of its travel, so each attempt is given that travel to itself rather
than reissued into it.

Also serves `interface.replyNotifications` in the operator-gate scenario with a
fresh body instead of the upstream response, since reusing its encoding headers
for a re-serialized payload leaves the client unable to read the config, which
looks exactly like an operator who configured nothing.

* fix: gate the assistant save on renderable output and stop overselling the poll limit

Two gaps this round's own work left behind.

`saveAssistantMessage` still decided to stamp from the message id alone, so an
Assistants run that persisted an empty row raised a dot that opening the
conversation could never clear. It now asks `isAnnounceableReply`, the same
predicate every other persistence path asks, which is the point of having moved
that decision into one place.

`pollLimit` advertised a range up to 1000 while `getConvosByCursor` clamps every
conversation read to a hundred-row page, so an operator who set 500 would have
been told it applied and silently served a hundred. The schema now stops at the
page the server will actually return; covering a deployment whose replies
outpace one page needs the server-side unseen query, not a wider page.

* test: let the mobile scenarios use the surface they are actually asserting on

Three scenarios opened the drawer on a page they only ever send a reply from.
On the mobile project that drawer sits over the composer and marks the pane
inert, so there was nothing left to type in and the run waited out its whole
budget. They ask for the composer now, which is what they were always about.

The two that read a row after leaving the list open it again first. A reply sent
from another tab closes this one's drawer, because `sidebarExpanded` is stored
per browser rather than per tab, and opening a chat closes it the way it does
for a reader. Both are what someone would do to look at the row, and neither
changes what is being asserted.

Mark as unread is dispatched rather than clicked, like the row controls around
it: the menu is portaled, and behind the drawer's scrim it inherits
`pointer-events: none` and goes click-dead.

* test: dispatch the row menu's unread item like the controls around it

The portaled menu sits under the mobile drawer's scrim, which intercepts the
pointer, so a real click retried until the budget ran out. The sibling scenarios
and the row's own controls already dispatch; these two were the last real
clicks.

* fix: hold reply alerts until the deployment answers, and read the resolved temporary state

The capability gate treated a startup config that had not loaded yet the same as
one that set nothing, so a device that stored "on" could badge, poll or notify in
the window before an operator's `false` arrived. Nothing is permitted now until
the config is in hand; only a loaded config that lacks the field reads as the
shipped default, which is the case a backend predating the setting produces.

The Responses API announcement read the temporary flag from the request body
alone, while the conversation save beside it resolves it from the stored
conversation first. A temporary chat restored or resolved on the server without
the flag in the body would have been announced through the unread indicators;
both completion branches now resolve it the same way the save does.

* fix: route the last reply stamps through the shared owner and hand back failed claims

Moving the announce decision into packages/api left three callers still deciding
for themselves. The agents controller re-stamped an unfinished turn whose
terminal persistence was skipped, and the resumed legacy turn stamped its
directly saved row, both from the message id alone, so a preempted or stopped
turn that persisted nothing readable lit a dot that could never be
acknowledged. Both ask isAnnounceableReply now. The non-agents abort path
carried an inline copy of the same predicate; it calls announceReply instead.
Error turns keep their direct stamp: the error card is what renders them.

A notification claim was written to shared storage before the notification
was constructed, so a constructor that throws left the reply claimed in every
tab with nothing ever shown. The claim is handed back when construction fails,
removed only while it still names that exact stamp.

* fix: judge the normal reply save by the shared predicate and hand back failed chimes

The regular BaseClient save still stamped any assistant reply that persisted,
readable or not, because the conversation write took a bare reply id. It takes
the persisted reply now and asks isAnnounceableReply before it stamps, which
leaves the error turns as the only direct stamps, and those are rendered by the
error card. saveTurnConversation is exercised against a real store in both
directions. announce.ts derives its write-context type locally so the two
modules do not import each other.

The chime claimed its replies before building the tones, so an output that
failed while they were being scheduled left them claimed in every tab with
nothing played. Those claims are handed back on failure, the same way a
notification that could not be constructed now releases its own.

* feat: let operators set the reply polling cadence

The away poll and the focused refresh were fixed at 30 seconds and five minutes,
which are request-rate levers on the conversation list and nothing an operator
could tune. Both are fields on `interface.replyNotifications` now, bounded and
defaulting to the values they had, and the watcher reads them through the same
capability hook as the rest of the block.

* fix: keep reply identity through list merges and drop rows the server no longer has

The list merge carried three of the four read-state fields by hand and left out
`lastResponseMessageId`, so an SSE update on an unseen conversation erased which
branch the reply landed on, and a missing identity reads as visible. It uses the
same `preserveReadState` helper as the other two merge paths now, and the test
that covered the other three fields covers this one too.

A local cache merge was only inspected on a list's first page for new arrivals.
A title or oldest-first sort leaves a replied row where it is, so its first
reply on a later page went into the alerts baseline without an arrival and its
chime or notification was lost. Local merges scan every loaded page; server
responses keep the first-page rule, and only rows already known are considered.

An unread request that matched nothing means the conversation is gone: the
server's owner-scoped update returns the row even for a no-op. The mutation
rolled back the read fields and kept the row; it removes it from the caches now,
as a delete does.

* fix: discard a job completion that resolves after the watcher has gone

The fetch that settles a finished job had no teardown guard, unlike the away
poll. Signing out or switching accounts unmounts the watcher, and a response
still in flight for the previous session would then merge into the caches the
next one reads. The result is dropped once the watcher has unmounted, tracked
on unmount rather than in the jobs effect's cleanup, which re-runs on every
change to the running set while its earlier completions are still wanted.

* fix: move error-turn stamps into packages/api and settle overlapping unread no-matches

Error turns were stamped in CJS: the error middleware checked eligibility, called
the stamp and shaped the event's conversation snapshot itself, and the agents
controller stamped its own. Both call announceErrorTurn now, and the
skipped-persistence restamp goes through announceReply, so no stamp is issued
directly from /api any more. The existing sendError tests pass against the real
helper unchanged.

An unread call that no longer owned the row dropped a server no-match with the
rest of its stale answers, so a deleted conversation could be restored by a newer
call's rollback. The no-match now evicts before ownership is consulted. It stays
after the superseded check: a superseded call never reached the server, and its
synthetic `modified: false` is not a no-match.

Also: Settings offers the reply toggles only once the startup config has loaded,
matching the capability hook; a cancelled reply-discovery snapshot is restarted
even though nothing observes it; and the focused refresh is capped at the five
minutes the discovery snapshot stays authoritative, since that refresh is what
renews it.

* fix: settle the assistant final in packages/api, announce attachment-only replies

Both assistant controllers carried the same persistence barrier: write the
response, check the row and the conversation both persisted, and unwrap the
settled conversation for the final event. That is behaviour, and it was
duplicated in CJS; settleAssistantFinal owns it now and the controllers call
it. Their existing tests pass unchanged.

The announce predicate ignored attachments, so a reply made only of files,
such as a resumed workspace artifact with empty content and text, raised no
dot, badge or alert. The message body renders attachments and acknowledgement
checks the body, so such a reply can be cleared as well as announced; every
announce path now passes them through.

The stopped-reply announcement took the storage engine's own id type in its
exported signature. It takes plain string ids now, and saveConvo accepts them
alongside its own, since the update casts them against the schema; the id type
stays inside data-schemas.

* fix: normalize the stopped reply's written ids in packages/api

The abort route filtered and stringified the ids of the rows it had written
before handing them to announceStoppedReply. That is data shaping, and it sat
in CJS; the announcement takes the ids as the caller holds them and drops the
unwritten ones itself.

* style: order the conversation row classes for Tailwind v4

The v4 upgrade on canary changed the class order Prettier enforces; these
three rows carried the v3 order.

* fix: replay replies a dead tab's focus lease held back, keep the focused refresh at a full page

A tab killed without firing blur or pagehide leaves its focus lease for up to
a minute. A reply arriving under it was baselined and then suppressed, and
nothing reran when the lease lapsed, so its chime and notification were lost.
Arrivals held back only by another tab's lease are kept and rechecked when
that lease expires; a lease the focused tab keeps refreshing still holds them.

The focused refresh reused the away-poll limit, so an operator who lowered it
also shrank the discovery page that keeps the unseen count complete behind a
filtered list. The limit now bounds only the away poll.

* fix: announce a held reply as soon as the focused tab releases its lease

A tab that blurs normally clears its lease long before the lapse, and a reply
held back for it is due as soon as nobody is looking. Lease changes from other
tabs now trigger the recheck, not only the expiry timer.

(cherry picked from commit b947738d0d)

Original-PR: https://github.com/LibreChat-AI/LibreChat/pull/15025
2026-10-01 03:46:16 +00:00
Marco Beretta
4ffe3825ec 📧 feat: add email address change (#14597)
(cherry picked from commit 9ff251dd96)

Original-PR: https://github.com/LibreChat-AI/LibreChat/pull/14597
2026-10-01 03:46:16 +00:00
Danny Avila
e43fed2ed4
🚄 perf: Linearize Conversation Import Lineage (#15881)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
* 🧬 fix: Resolve Large Conversation Import Lineage in Linear Time

Parent lookups, timestamp ordering, skipped-ancestor resolution and
citation linking during conversation import each rescanned the batch or
the text per message, so a large export spent minutes on the request.
They now run in a single pass, lifted into packages/api as
createChatGptLineage, linkChatGptCitations and orderMessageLineage.
Fork cloning and branch collection use id maps for the same lookups.

* 🧬 refactor: Lift Fork Clone Lineage Into packages/api

cloneLineage assigns clone ids, re-links parents and orders timestamps,
and getAllMessagesUpToParent moves over whole with its id map, so
fork.js only supplies the id generator and persists the clones.
2026-09-13 03:48:22 -04:00
Danny Avila
f43301f818
🔭 feat: Conversation Trace Viewer (#15869)
* 🔭 feat: Conversation Trace Viewer

Adds a trace view for a user's own conversations: a summary, a pinned
waterfall overview with interval focus, a hierarchical record ledger and
a record inspector, opened from the chat header (or the mobile overflow
menu) and read through a provider-neutral seam with Langfuse as the
first reader.

- interface.traceViewer (enabled, showInputOutput, maxRecords,
  maxContentLength, requestsPerMinute), off by default
- /api/traces/:conversationId availability, records and record routes,
  re-proving conversation ownership and narrowing the Langfuse session
  to traces the user's own sampled responses produced
- Langfuse reader over the v2 Observations API with cursor paging, time
  bounds, typed failures and server-side input/output withholding
- forks no longer copy the source messages' trace sampling record

* 🛂 fix: Address Trace Viewer Review Round 1

- interface.traceViewer.requestTimeoutMs bounds each Langfuse round trip
- availability is one existence query across every sampled response
  instead of the oldest one, run alongside the ownership check
- tree items report their position among visible siblings
- point-in-time Langfuse events read as completed, not running
- reads prefer the project holding the most responses when a tenant
  connection arrived mid-conversation

* 🎯 fix: Leave Focus With a Navigation That Closes the Trace

* 🧭 fix: Keep Trace Reads on One Project and Cancel Abandoned Reads

- page cursors and record details stay on the project that served the
  first page; pages report an opaque sourceId
- Open in Langfuse only appears when the linked project served every
  loaded page (the session link now reports its destinationId)
- client and server abort Langfuse reads when the viewer closes or the
  client disconnects
- reads never wait on the central project lookup, which the request
  timeout does not govern
- the cost total is withheld when any model call with usage is unpriced

* 🕰️ fix: Refresh Trace Pages After a Turn, Honor Clock Settings, Show Refresh Failures

- a settled run invalidates the cached trace pages, so reopening shows
  the new turn instead of pages fresh for another 30 seconds
- trace durations and clock times format in the app language and the
  user's 12/24-hour setting, as message timestamps do
- a failed refresh with every page loaded surfaces an alert and a retry

* 🔐 fix: Only Server-Authored Responses Grant Trace Access

- the message-create route and imports strip langfuseSampled and
  langfuseDestinationIds, and trace reference queries ignore client-authored
  rows (isUserSubmitted) and user turns, so a forged row cannot claim a trace
- the Langfuse reader receives its destination resolver and HTTP client
  from the route instead of defaulting to process-global ones
- availability asks the client to check again while central's project id
  is still resolving, instead of caching a false answer
- listRecords tries the next readable project when legacy responses left
  the first one empty
- a settled run also invalidates cached record details
- running is read from a record's status, not a missing end time

* 🏢 fix: Scope Trace Reads to the Tenant and Reach Every Project Holding a Turn

- ownership and both trace reference queries carry the requester's tenant
  (tenantless is its own scope), so a duplicate user and conversation id in
  another tenant cannot grant or supply trace references
- a read visits, page by page, every project that could hold a turn no
  earlier project provably holds, so legacy turns in another project stay
  reachable after a newer destination ranks first
- the trace read limiter takes its store from the route
- transient availability failures retry instead of hiding the control

* 🛟 fix: Fail Over Between Projects, Refresh One Page, Search Visible Labels

- a fresh read fails over to another project holding the same turns when the
  preferred one fails; the cursor carries the failed sources so later pages
  keep the rebuilt plan, and detail reads fail over the same way
- destinations for one project collapse into one source
- the trace read limiter keys each user within their tenant
- refresh and a settled run keep only the newest page before refetching, so
  older pages are not replayed through the limiter; retry repeats the read
  that failed
- search matches the localized kind and status labels a row shows

* 🧭 fix: Read the Newest Turn First, Report Failures Behind Empty Projects, Fit Trimmed Views

Rank trace sources by the newest sampled response they could hold before coverage and preference, so a conversation that moved projects shows its latest turn first.

Throw a list failure when a failed project could hold a turn that no answering project provably holds, instead of rendering an empty trace. Detail reads keep probing projects after an empty answer, skip ones whose turns an answering project provably holds, and surface a failure that could hide the record.

Refresh and settled runs trim the cache to the newest page; the viewer now fits a focused interval to the remaining records or clears it, and drops a selection whose record left with the older pages.

* 🧵 fix: Page Traces by Turn, Include Failed Runs, Keep the Active Row Mounted

List reads now page by turn segments instead of whole projects. A segment is the newest unread turn and up to 49 older consecutive turns one project could hold. The request filters Langfuse to those turns' run and title traces, so pages stay newest first when a conversation's responses alternate between projects. Legacy turns without destination ids are asked of the other projects in the segment. A project that failed for newer turns is retried when it alone holds older turns. A page whose rows all fail the observation schema is an upstream failure, not an empty trace.

A failed turn's error row now stores the sampling fields of the run that failed, plus langfuseRunId when that run was created, so the trace viewer and feedback scores follow the run's trace. langfuseRunId is stripped from client message writes and imports, dropped by forks, and hidden from client message projections.

The ledger keeps its active tree item mounted while virtualization scrolls it out of the window, so aria-activedescendant always names a rendered row.

* 🕰️ fix: Merge Multi-Project Trace Pages by Start Time, Keep Literal Content

A segment that reads more than one project now runs every read before returning. A read that fills the record budget cuts the page at the oldest start time all reads loaded, and the continuation carries that start time as an inclusive bound instead of a Langfuse cursor. Every project continues from the same point, so a newer turn held by another project is never left behind an older one, and failover works on any page. A page drawn from several projects no longer names a single source. A continuation that would repeat its own bound fails instead of looping.

Input and output that are literally the strings {} or null are kept; only an empty object or JSON null value is dropped.

* 🔎 fix: Read Each Trace Segment From a Project Shown to Hold It

Recorded destination ids say where a trace was eligible to go, not where it landed. A list read now looks up the turns more than one readable project could hold before choosing where to read them. Each project, preferred first, is asked which of those traces it holds a root observation for, and each turn is read from the preferred project that holds it, or from the preferred eligible one while a run's root is not exported yet. Turns only one project could hold need no lookup.

Segments are consecutive turns from one project, paged by that project's own Langfuse cursor. Records that share a start time page correctly, and the page budget bounds a single read. A failed project is dropped for the request. A turn only it could hold raises its failure at the turn's own place in page order, instead of leaving an empty or partial trace. Cursors carry the segment's trace-set hash and start over when the segment changed or Langfuse rejects them. Detail reads ask every project until the record turns up.

The trace summary withholds its cost total when any generation has no cost, with or without usage.

* 🧷 fix: Evidence per Trace, Continuations Without Replays, Stable Turn Order

A turn's run and title run are exported separately, so each trace id is looked up and read from a project shown to hold it. A trace no project shows (a title run the turn never had, or a run whose root is not exported yet) joins an adjacent segment whose project could hold it. Only a run that a failed project alone could hold raises that failure.

A continuation now stays on its project. Its failure is returned as is, and a cursor the segment no longer matches, or one Langfuse rejects, returns invalid_request instead of replaying the segment's first page. The viewer answers invalid_request on an older page by reloading from the newest page, with a message saying the trace changed.

Sampled responses sort by createdAt then _id, so responses saved in the same millisecond keep one order across page requests.

* 🪪 fix: Scope Trace Reads to the Exporting User, Order Split Title Runs

Trace ids derive from message ids, which a request can influence, so a colliding id must not return someone else's trace. Every Langfuse list, probe and detail read now also filters by userId: the requester's internal id, plus the configured langfuse.trace.userIdField value when set. The server stamps that id on the trace at export. The handler passes the requesting user into the trace query.

When a turn's run and title run sit in different projects, the probes' root start times decide which is read first, so a title generated after the response is not left behind Load older. Cursors name the trace a segment starts at within its turn.

The failed-turn trace lookup moves into packages/api as getFailedTurnTraceFields. The agent controller only passes the error row id, the run id and whether the run was created.

* 🔒 fix: Authorize Trace Reads by the Internal User Id Only

A configured langfuse.trace.userIdField can export a non-unique value (name, username) or one that repeats across tenants (email, provider ids) as the trace userId, so it cannot prove who a trace belongs to. Reads now require the internal user id alone. A deployment that exports another field reports no trace and logs why once, instead of trusting that value.

A failed read of a title run that a probe found in that project now raises its failure instead of dropping the title. The page contract now states that pages are ordered by turn and a turn's records may continue on the next page.

* 📚 fix: Page Sampled Responses With the Trace, Share Ingress Stripping

List reads load sampled response references a segment at a time: up to 50 ending at the cursor's turn, plus one older response that names the next page. Pages of a long conversation no longer rematerialize every sampled response. getConversationTraceRefs gains through and limit, anchored by createdAt and _id.

The message-create route and conversation imports strip trace sampling fields through withoutTraceRefs instead of their own field lists. A userIdField the exporter ignores counts as the internal id. A root lookup whose cursor repeats fails that project instead of paging forever.

In the viewer, Refresh also rereads the open record's detail, and closing the inspector after a filter removed every row returns focus to the search field.

* ⏱️ fix: One Round Trip per Trace Reference Read, Detail Reads Scoped to Their Turn

Cursors carry the anchor response's order key (creation time, then _id). The bounded sampled-response read then runs alongside the first-message lookup, with no anchor lookup ahead of it. A key that no longer names its response returns nothing, and the reader answers the cursor as changed.

Detail reads take the turn the list attributed the record to (?message=). They load only that sampled response instead of the conversation's whole history, and ask Langfuse within that turn's traces. The route rejects a detail read without it, and the client sends it from the listed record.

A list read whose Langfuse cursor repeats now fails instead of handing the next page the same records.

* 🧭 fix: Reject Out-of-range Trace Cursors, Open Any Listed Turn, Mark Copies Unsampled

A page cursor whose base-36 time parses to a number outside the Date range reached Mongoose as an Invalid Date and surfaced as a 500 the viewer retried; it now resumes nothing, which the reader reports as invalid_request so the viewer reloads its newest page.

The detail read capped the message id at the Langfuse record-id bound while the app stores response ids at any length, so a listed turn could refuse to open; message ids only name a stored row and are accepted at any length. Ids that reach a Langfuse filter keep their caps.

Copied and client-authored rows now carry langfuseSampled: false instead of no record, so a feedback score on a fork, import or posted message never recomputes sampling for a trace that was never made.

* 🧪 test: Type a Recordless Row for the Unsampled-copy Spec

An object literal with none of the trace fields fails TypeScript's weak-type check against the helper's all-optional constraint; the spec now passes a typed row that declares the optional field, as every stored message type does.

* 🚰 fix: Release Failed Langfuse Response Bodies

A non-success Langfuse response was turned into a TraceReadError without consuming or cancelling its body, and Node's fetch returns a connection to its pool only once the body is released, so a sustained upstream failure left sockets checked out until garbage collection. The body is now cancelled before the status becomes an error.
2026-09-13 00:56:52 -04:00
Danny Avila
8f13845da5
📿 fix: Inspect Stored File Locators Per Message (#15841)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
* fix: inspect stored file locators per message

* fix: declare traversal diagnostics for isolated declaration builds

* refactor: inject locator traversal reporting from application callers

* fix: propagate traversal diagnostics across protected entry points

* fix: complete middleware reporters and hydration message counts
2026-09-11 16:47:48 -04:00
Danny Avila
ab6cde0c79
⏳ fix: Separate Temporary and General Chat Retention (#15734)
* fix: separate temporary and general chat retention

* fix: inherit retention when branching messages

* fix: harden retention lifecycle boundaries

* fix: preserve subagent retention context

* fix: trust stored chat retention for uploads

* perf: overlap generated file retention lookup

* fix: derive retention from trusted chat state

* fix: preserve retention across the full turn lifecycle

* test: accept the conversation retention read adapter

* fix: carry retention deadlines through durable job saves

* fix: resolve retention at cached and new chat admission

* fix: resolve direct tool retention before execution

* fix: preserve retention across assistant writes and direct tools
2026-09-08 11:22:54 -04:00
Dustin Healy
037cbf530d
🧺 fix: Contain Conversation Import and Upload Failures (#15681)
* 🛡️ fix: Harden Conversation Imports and Upload Handling

* 🩹 fix: Address Import Hardening Review Findings

* 🩹 fix: Map Clone Size Validation Errors

* 🩹 fix: Chunk Import Cleanup Queries
2026-09-06 22:42:46 -04:00
Danny Avila
8a46ca070c
🧊 feat: Persist Context Fading Tier Across Agent Runs (#15505)
* 🧊 feat: Persist Context Fading Tier Across Agent Runs

Carry the pruner's latched context-fading tier from `@librechat/agents`
through `contextMeta` the way `calibrationRatio` already travels, so a
historical tool result keeps the same truncated bytes from one run to the
next and Anthropic's prefix-based prompt cache survives across turns
(danny-avila/agents#497).

- Add `IAgentFadingTier` to the message and conversation `contextMeta`
  types, schemas and the data-provider zod schema
- Add `isAgentFadingTier`, `resolvePersistableFadingTier` and
  `resolveRunContextMeta` in `packages/api`; a tier is persisted once it
  carries information (masking active or a budget below the window), and
  a latched tier is kept even at a neutral calibration ratio
- Seed `createRun({ fadingTier })` from the parent response's contextMeta
  regardless of encoding, since caps are character-based; the field rides
  the existing forward-compatible `runConfig` variable and older SDK
  versions ignore it, as does `Run.getFadingTier` being absent
- Accept a valid `fading` field in event actor context and reject a
  malformed one, matching the calibration checks
- Capture run context meta through one module-level helper for both
  completion paths instead of two inline copies

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Seed Fading Tier Across HITL Resume, Validate at the DB Layer

Address review findings on fading tier persistence:

- Leave the "is this tier worth persisting" decision to the SDK, whose
  `Run.getFadingTier()` now returns only informative tiers; comparing
  against the client's window misclassified untouched conversations
  whenever a reserve ratio was configured
- Carry `contextMeta` through the HITL pause projection (staged approval,
  job metadata, resume state) and seed the rebuilt client from it, so a
  paused turn resumes with the same tier and calibration instead of
  re-deriving a shallower one and rewriting the prefix
- Share run seeding between `chatCompletion` and `resumeCompletion`, and
  make capture tolerate partial runs and bare resume contexts
- Validate `contextMeta.fading` in `commitAgentEventActorState` with a
  shared `isAgentFadingTier` guard in data-schemas, so a malformed tier is
  rejected at the DB layer instead of silently cold-starting the actor
- Reuse `IAgentEventActorContextMeta` for `IMessage.contextMeta` instead of
  extending an inline duplicate

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 test: Cover Resume Seeding From Paused Context Meta

Assert the resume controller seeds the rebuilt client from the context meta
captured at the pause before rebuilding the run, and move the capture
helper's JSDoc back above its function.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧹 style: Sort Imports in Fading Tier Plumbing

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Persist Fading Tier Through Redis Jobs and Branches

Address Codex and Copilot review findings on fading tier persistence:

- Deserialize `contextMeta` in `RedisJobStore`, whose explicit read mapper
  rebuilt every other pause field but this one, so Redis-backed
  deployments resumed with no tier to seed after every HITL pause
- Copy server-private `contextMeta` onto the assistant message created by
  `POST /api/messages/branch`, so the next turn seeds its pruner the same
  way it would from the parallel source response
- Constrain the `fading` subdocuments in both Mongoose schemas (version
  enum, positive budget, required fields) to match the zod schema
- Throw a distinct "fading tier is invalid" error from both context meta
  validators instead of reporting a calibration failure

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Carry Context Meta Through Stop and Re-Pause

Addresses the second Codex pass on the fading-tier persistence:

- A re-pause whose resumed segment has nothing persistable now clears the
  job's `contextMeta` in the same transition instead of leaving the first
  pause's calibration and tier for the next resume to seed from.
- The client publishes the run's live calibration and fading tier onto the
  job after each pre-invoke context snapshot (deduplicated on value), and
  the Stop path copies `jobData.contextMeta` onto the stopped response, so
  a follow-up from a stopped turn keeps the prompt prefix stable.
- Mid-run captures (HITL pause, Stop) read the graph's live state; `Run`
  refreshes its own getters only after the stream settles, so the pause
  used to persist the seeds it started from.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Fence Context Meta Publication Ahead of Stop

A Stop handled by another replica reads the job before the owner is
signaled, so a fire-and-forget publish could lose the race, and a run
stopped before its first context snapshot never published the tier it
inherited. The client now publishes the inherited seed before the run
streams or resumes, and the context-usage handler awaits each
snapshot's publish so the write lands before the model call it
describes begins. Failures still only log; the pre-run publish is
optional-called so bare resume contexts keep working.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Read Back Context Meta After the Abort Claim

The run owner publishes its context meta ahead of each model call and
awaits the write, so a publish landing between the abort's initial job
read and its terminal claim describes the call whose partial output the
abort snapshot carries. abortJob now reads the same-epoch job back after
the claim and content re-read and takes its context meta, so the stopped
response persists the tier that produced its bytes. The refresh is
best-effort and logs on failure like the content refresh beside it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Share In-Flight Context Meta Writes and Clear Neutral Live State

Two snapshot callbacks in a parallel-agent run can capture the same tier
before the first write settles; the publisher now caches the in-flight
promise per serialized value so the second caller awaits the same
durable write instead of treating an uncommitted value as published.
A live snapshot with nothing left to persist after an earlier publish now
writes a neutral record (ratio 1, no tier), since a running job's fields
cannot be deleted through the metadata writer, so a Stop no longer
persists an earlier non-neutral state onto the response. The pre-run
seed publish is unchanged and is told apart from live snapshots by a
flag from the context-usage sink.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 chore: Drop Unused Rest Parameter in Context Meta Test

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 feat: Persist Per-Agent Fading Tiers as Compact Context Meta

Reconciles the host with @librechat/agents at 9aacad1a, where graph
history stays canonical and each Run derives a provider-only projection
from the latched tiers. LibreChat therefore persists only compact state:
the calibration ratio, the default agent's tier, and now the per-agent
tiers from Run.getFadingTiers(), stored as validated entries so agent IDs
never become MongoDB field names and restored onto a null-prototype
record for RunConfig.fadingTiers. Truncated messages, canonical tool
content, projection provenance and per-message truncation state are
never persisted; the SDK's latched flag is stripped on capture.

The per-agent map travels everywhere the single tier already did:
response contextMeta, event-actor checkpoint state, HITL pause and
resume job metadata in both stores, the Stop path, and branch creation.
A shared Mongoose definition keeps the message and conversation schemas
identical, the zod schema validates entries, and commitAgentEventActorState
rejects malformed maps. Comments no longer describe graph messages as
truncated; only the provider projection is.

Tests cover: tool results and inputs reach the persisted content parts
at full size; contextMeta carries exactly the compact tier and
calibration fields; a persisted snapshot round-trips into the seeded
RunConfig unchanged and prototype-safe; per-agent tiers survive pause,
Redis job round-trips, and event-actor context validation.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Restore Event Actor Types and Publish Seeds Before Abortable Setup

The previous commit's doc rewording in the conversation types dropped the
event-actor interfaces that precede the fading tier, which broke the
declaration bundle in CI; they are restored unchanged.

The inherited context meta is now published onto the job as soon as the
parent's state is loaded and awaited at the top of chatCompletion and
resumeCompletion, ahead of run creation and the other abortable setup
stages, so a Stop during setup still persists the parent's tiers onto the
stopped response. The partial response saved when every subscriber
disconnects now carries the job's context meta like the Stop and pause
paths do. The branch route keeps the source's context meta in the saved
message but strips it from the HTTP response, matching client reads.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Serialize Distinct Context Meta Publications

Overlapping snapshots from parallel agents could carry different tiers;
each started its own write and both job stores keep whichever write
finished last, so an older snapshot could overwrite a newer one. Each
distinct publication is now issued after the previous one settles, in
call order, so the newest snapshot is what a Stop reads back. Equal
values still share one write.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Project Root Message Reads and Keep Inherited Meta on Setup Failure

The root GET /api/messages paths returned raw documents: the
single-message query read called getMessages without a projection and the
cursor page performed an unprojected find. Both now apply
CLIENT_MESSAGE_SELECT, with getMessagesByCursor accepting a select option,
so the persisted fading tiers never reach the client through the
paginated endpoint; the search-hydration read projects the same way.

A turn that loads a parent's context meta but fails before its run exists
used to replace the inherited meta with the capture of a missing run, so
the persisted error response lost the last valid tiers. Both finalizers
now keep the inherited meta when no run was created while a created run's
neutral state may still clear it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 refactor: Move Context Meta Publication Into packages/api

The publication coordinator (ordering, deduplication, in-flight sharing,
failure state) now lives in packages/api as createContextMetaPublisher,
with selectRunContextMetaToPublish deciding what a publication carries;
the legacy client keeps only the wiring to the job metadata writer. A
transiently failing write is retried with a short backoff before the
publication is reported and forgotten, so a snapshot's model call no
longer proceeds on the first rejection while the job holds the previous
record. The SDK swallows handler errors, so retrying is the only way to
strengthen the fence without failing the turn.

A terminal save now supplies contextMeta: null when the finished run has
nothing to carry, and saveMessage unsets a previously stored value in
that case, so a disconnect snapshot's record cannot outlive the run that
completed neutrally.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Widen the saveMessage Method Type for the Null Unset

The exported method interface still typed contextMeta as the message
field, so the spec exercising the null unset failed typechecking; it now
mirrors the implementation signature.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Publish Every Agent's Snapshot, Sanitize Search Hits, Unbound Agent IDs

The context-usage handler now awaits the run's context meta publish for
every ON_CONTEXT_USAGE event, hidden sequential agents included, while
the visible-snapshot bookkeeping and the client emission keep their gate;
a Stop before the next visible snapshot finds the tier a hidden agent's
model call latched. Search results are built from the hydrated hit minus
its server-private contextMeta, since Meilisearch hydration projects every
schema field. The per-agent tier entry guard no longer caps the agent ID
length: ephemeral agent IDs encode endpoint, model and sender without a
bound, and dropping such an entry would let that agent alone re-derive
its tier on the next turn. Also formats the publication spec so the
static checks pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Unset Context Meta Atomically, Keep Publication State After a Failed Write

- `saveMessage` folds the `contextMeta` unset into the same update that
  persists the terminal response, on both the plain and the provenance-merge
  paths, so a failure between two writes can no longer leave a completed row
  carrying a disconnect snapshot's state.
- `createContextMetaPublisher` tracks whether any record has committed
  separately from the retryable latest publication, so `hasPublished` stays
  true after a later publication exhausts its retries and a following neutral
  snapshot still overwrites the earlier record on the job.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Carry Context Meta Through the Resumable Stop Route and Neutral Resumed Completions

- The resumable Stop route copies the refreshed `jobData.contextMeta` onto the
  terminal assistant message it persists, as the abort middleware already
  does, so a stopped resumable response keeps the tiers that produced its
  partial output.
- A resumed run that completes with neutral context state now saves
  `contextMeta: null`, which unsets the paused segment's record instead of
  letting an omitted field keep it on the completed row.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Trust Only Server-Authored Context Meta, Unset It on a Neutral Stop, Read It From the Job on Disconnect

- `POST /api/messages/:conversationId` and conversation imports drop any
  client-supplied `contextMeta`, and the run seeds its calibration and fading
  tiers only from a server-authored parent response, so a forged tier can never
  shape a provider projection.
- Both Stop paths save `contextMeta: jobData.contextMeta ?? null`, so a job that
  re-paused with neutral state unsets what an earlier pause stored on the row
  instead of leaving it behind.
- The disconnect partial save reads the same-epoch job record for the run's
  published context meta, since the client-facing resume snapshot never carries
  server-private state.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧪 test: Mock the Job Store in the Resume Metadata Spec

The disconnect partial save now reads the same-epoch job record through
`GenerationJobManager.getJobStore()`, so the resume metadata spec's manager
mock provides it as the other disconnect spec already does.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 🧊 fix: Keep Context Meta Off the POST Message Response

`POST /api/messages/:conversationId` returned the saved row verbatim, so a
client write against an existing server-authored message echoed the row's
stored context meta. The route now projects the row through the same client
projection the branch route uses, which drops the server-private field.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

* 📦 chore: Bump @librechat/agents to 3.7.16 for Persisted Fading Tiers

3.7.16 is the first release that carries danny-avila/agents#497, so
`Run.getFadingTier()` / `getFadingTiers()` now report the latched tier
this branch persists and `RunConfig.fadingTier` / `fadingTiers` seed the
next run from it. The dependency set is unchanged from 3.7.15; only the
package entry moves.

Adds an SDK-backed round trip to the fading spec: a first pruner latches
an informative tier, `resolveRunContextMeta` reduces it to the persisted
shape, and a second pruner seeded from that shape reproduces the first
run's projected tool-exchange bytes under instruction and calibration
drift, while an unseeded pruner under the same drift relaxes its tier and
rewrites them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNbgMeR2aZDyhsqAy4L8gF

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-02 20:29:43 -04:00
Danny Avila
de59da9636
🎟️ refactor: Require Credentials for Local Image Access by Default (#15252)
* 🔐 fix: Protect Local Image Access by Default

* 🔐 fix: Scope Image Authorization to Active Sessions

* 🧹 style: Format Image Authorization Checks

* 🛡️ fix: Harden Image Avatar Authorization

* 🧭 style: Sort Image Authorization Imports

* 🔐 fix: Close Image Authorization Review Gaps

* 🧭 fix: Normalize Stored Avatar Base Paths

* 🏢 fix: Resolve Tenant Assistant Image Policy

* 🛂 fix: Enforce Effective Image Access Policy

* 🧹 style: Flatten Assistant Config Selection

* 🧷 fix: Preserve Image Access Compatibility

* 🪪 fix: Make Image Sessions Revocable

* 🏗️ fix: Move Image Session Policy Into API
2026-08-27 09:55:27 -04:00
Danny Avila
1d9c2fc591
🪴 feat: Fork Completed Subagents Into Continuable Chats (#15133)
* feat: continue completed subagents as chats

* chore: sort continuation imports
2026-08-23 10:09:14 -04:00
Danny Avila
67b7b441b2
🛂 feat: Filter Model-Bound Content by Source (#14425)
* feat: introduce optional content protection seam

* feat: enforce source-aware content filters

* feat: complete source-aware content enforcement

* test: activate skill file-text fail-close fixtures

* fix: harden source-aware content filters

* fix: harden model-bound content filtering

* fix: preserve legacy filters and generated files

* fix: inspect shared scalar metadata

* test: align mocks with current dev dependencies

* feat: add persisted content filter safeguards

* feat: complete source-aware content filter enforcement

* fix: move resume content preflight into TypeScript

* fix: close content inspection edge cases

* fix: harden content protection boundaries

* fix: complete content protection safeguards

* test: align persisted memory filter coverage

* fix: reconcile content protection with current dev

* fix: reconcile content protection with latest dev

* fix: close content protection review gaps

* fix: enforce source-aware provider boundaries

* fix: preserve legacy PII preflight semantics

* test: stabilize stored branch preflight fixture

* fix: defer agent writes until protected model admission

* perf: harden source-aware model-bound filtering

* fix: canonicalize provider lineage before validation

* fix: satisfy model-bound callback type checks

* perf: Bound content protection filtering work

* fix: Bound submission array traversal

* fix: Stabilize bounded content snapshots

* fix: Scope model-bound traversal overflows

* fix: Preserve scoped content inspection

* fix: Accumulate aggregate traversal scopes

* fix: centralize content policy boundaries

* test: align deferred tool policy context

* test: align controller policy mocks

* style: normalize content protection imports

* fix: close content policy review gaps

* fix: narrow active skill policy config

* fix: address content protection review boundaries

* fix: retain exact provenance overflow sentinel

* fix: preserve literal and scoped provenance updates

* fix: narrow persisted edit provenance

* fix: isolate exact overflow attribution

* fix: centralize stored prompt protection

* fix: fail closed on incomplete transcript evidence

* fix: align canonical transcript routing

* refactor: centralize content policy preflights

* fix: isolate upload policy error typing

* style: sort policy preflight imports

* refactor: centralize content policy boundaries
2026-08-21 22:43:32 -04:00
Danny Avila
547bd8c4bf
🧵 feat: Persist View-Only Subagent Threads (#14957) 2026-08-18 07:41:42 -04:00
Danny Avila
1d789c41a5
🧩 fix: Normalize MCP UI Resource Rendering (#14868)
* fix: normalize MCP UI resource rendering

* fix: filter unsupported MCP UI resources

* fix: preserve MCP UI marker examples

* fix: handle MCP UI resource edge cases

* fix: harden MCP UI marker sanitization

* fix: scope MCP UI marker sanitization

* fix: parse MCP UI marker contexts

* fix: align MCP UI sanitizer parsing

* fix: match MCP UI renderer syntax

* fix: align blockquote marker spans

* fix: decode MCP UI text node sources

* fix: sanitize nested subagent markers

* fix: bound MCP UI sanitizer traversal

* fix: keep MCP UI marker mapping linear

* style: sort security patch imports

* fix: harden nested MCP UI sanitization

* fix: mirror citation cleanup for MCP UI markers

* fix: clean decoded citation markers

* fix: clean assembled citation markers

* fix: align MCP marker sanitization with rendering

* fix: match persisted MCP marker render paths

* fix: preserve highlighted citation boundaries

* fix: align MCP markers across content renderers

* fix: preserve citation renderer boundaries

* fix: match legacy thinking trim semantics
2026-08-15 12:49:59 -04:00
Marco Beretta
152dcf4721
🔗 feat: Shared Conversation Badge and Stable Share Links (#14712)
* feat: improve shared conversation links

* test: Cover Shared Link Lifecycle

* test: Cover Shared File Snapshots

* fix: address review findings on shared links

Stop double-decoding the conversation search term. Express already decodes
req.query, so the route's extra decodeURIComponent threw URIError on any term
containing a bare percent sign and mangled percent-escape-looking text. The
sidebar already sent the term raw, so this failed there too.

Advance a share's stored target to its branch tail when an update omits one.
Updating from the conversation list could not resolve the tail and reused the
stored target verbatim, silently republishing the same snapshot instead of the
turns added since.

Require revalidation on shared files. Updates now keep the shareId, so the file
URL no longer changes and a cached response could outlive a revoked share-files
choice; an ETag over the pinned snapshot fields keeps unchanged files on 304.

* fix: keep the shared badge across conversation cache replacements

isShared is derived per list request and absent from single-conversation
payloads, so rename, pin, and the SSE conversation updates dropped it when they
swapped a server response into the sidebar cache, hiding the badge until an
unrelated list refetch. Carry the cached value forward in updateConvoInAllQueries
so every replacing caller is covered, while an explicit value still wins.

* test: mock syncStaticTools in server boot specs

initializeMCPs now calls syncStaticTools when no MCP servers are configured, but the server boot specs stub ~/server/services/Config without it. Post-listen initialization threw, hit process.exit(1), and took the jest worker down until it exceeded the retry limit.

* fix: address codex findings on the shared DataTable and file ETag

Keep the published DataTable export bound to the legacy component and ship the design-system table as VirtualizedDataTable, so external consumers of @librechat/client keep the props they compile against.

Fold the snapshot's stored location into the shared-file ETag, so a re-published output that keeps its size and revision but moves its object no longer revalidates to a stale 304.

Auto-fill the table when a first page is too short to overflow its container, since pagination is otherwise only reachable through the scroll handler.

* fix: re-scope share grants before publishing and retry stalled auto-fill

Move the shared-link ACL expiration write ahead of the content update. The shareId survives an update, so a failed ACL write after the write-through left the new messages and file snapshot readable at the same URL while the owner saw a 500.

Retry a rejected auto-fill fetch up to three times: an unscrollable table has no scroll event to fall back on, and the sentinel alone would strand it on the first page.

* fix: follow regenerated branches and pin forks to the payload they saw

advanceTargetToBranchTail only walked descendants, so a target replaced by a regeneration (a sibling, not a child) left the update parked on the obsolete branch and published none of the turns that followed. A childless target now hops once to the newest sibling the conversation continued under.

A shareId survives an update, so an owner republishing between a viewer's load and their Continue click would resolve targetMessageIndex against different messages. The fork request now carries the payload's updatedAt and is rejected with 409 when it no longer matches; the viewer gets the current version pulled in and can retry.

* fix: keep table sorting and legacy backfills from breaking share flows

Restore the union formatting a local lint-staged prettier collapsed in data-provider types, which broke the CI lint run.

Header clicks now toggle direction instead of cycling through an unsorted state, which the controlled tables translated straight back into the default and made one direction unreachable.

Re-arm the auto-fill guard on a sort change: a re-sorted first page arrives with the same row count, and the guard would otherwise suppress paging on a container that still cannot scroll.

Lazy fileSnapshots backfills no longer touch updatedAt. That timestamp is the revision a viewer's fork is validated against, so a legacy share's first read would have made the Continue click that followed it fail with a 409.

* fix: break pagination ties by id and reset share state per conversation

Both list cursors marked a page boundary with values that repeat: conversations by (sort field, updatedAt) and shared links by the sort field alone. Imported chats share a title and a timestamp, so every row tied with the boundary was skipped. Both now carry the boundary row's _id and sort by it last, and the shared-links cursor is an opaque composite the route still validates before querying.

The share dialog outlives a switch between conversations, so a link with files disabled left the next conversation's dialog showing the switch off and quietly published without files. The stored choice now falls back to the enabled default, and a stale link no longer sits in the copy field.

* fix: keep titleless shared links in the paginated list

A share has no title default, and BSON orders a missing title before every string, so encoding the boundary as an empty string skipped the remaining untitled links when sorting Name ascending and re-admitted all of them descending.

The cursor now carries the boundary's null rather than flattening it, and because $lt and $gt are type-bracketed against a string, descending adds an explicit clause for the untitled tail that a string comparison can never reach.

* style: sort share method imports

* fix: fail closed on orphaned share targets and guard snapshot backfills

getMessagesUpToTarget walked levels from the roots and returned everything it had accumulated when the target was never reached. An imported or partially deleted branch whose parent is missing therefore published the whole conversation instead of the selected branch; the walk now returns nothing unless it actually reaches the target.

A lazy backfill wrote fileSnapshots unconditionally, so a viewer's first read of a legacy link could land after a republish and restore the snapshot it replaced, re-authorizing the stable URL of a file the owner had just removed. The write is now conditional on the link still having no snapshot, and the stored one wins any race.

Regenerating a message above the shared tail leaves the whole stored branch childless, so the target walk now climbs to the closest ancestor the conversation continued under instead of stopping at the stored tail's own siblings.

Changing the search or sort also returns the table's viewport to the top, since the query holds the previous rows while it refetches.

* fix: page through titleless rows on both sides of the cursor

The route validator still required a string primary, so the composite cursor the data layer issues for a titleless boundary came back as a 400 and the shared-links table stopped at that page.

Conversations had the same type-bracketing gap the shared links just closed: a name-sorted page could not reach conversations with no title, since a comparison against a string never matches a missing field. The cursor now carries the null and the filter spells out the titleless clauses for both directions.

* fix: keep the share badge read-only and refresh rows on cell changes

ensureLinkPermissions re-granted the owner ACL entry on every call, so rendering the header's shared badge turned ordinary navigation into a permission write. It now checks for the grant first and only migrates a link that still lacks one.

A fork carrying a positional target but no revision falls back to the whole share, since nothing proves which payload the index was counted against.

The memoized table row compared row data and selection only, so a cell rendering external state (the archived list's pending Restore, for one) kept its stale rendering until the row object itself moved; rows now also compare a marker that moves with the column definitions.

* fix: keep the shared badge honest when a delete fails or a link remains

A failed delete left the conversation looking unshared: the optimistic snapshot covered only the shared-link queries, not the conversation caches the badge reads. The cleared conversations are now restored with the rest.

A conversation can hold one link per target message, so clearing the badge on delete is a guess. The conversation list is invalidated once the mutation settles, letting the server decide from the links that are actually left.

* fix: refetch every cached conversation page after deleting a link

The invalidation was pinned to page zero, so a conversation cached further down the sidebar kept the badge the optimistic update had already cleared even when another targeted link survived.

* fix: treat a failed page fetch as a failed auto-fill

React Query resolves fetchNextPage with an error result instead of rejecting, so the rejection handler never ran: the guard stayed armed on the unchanged row count and an unscrollable table could never reach the next page.

* refactor: move the share request helpers into the typed backend

Cursor validation, page-size clamping and the shared-file cache validator were plain backend logic sitting in the legacy JS route. They now live in packages/api with their own tests, and the route keeps only the Express-side wiring: reading query params, mapping domain error codes to status codes, and writing the response.

Also carries the requested file choice into the cache entry the create and update mutations synthesize, since the response never echoes it and the dialog reads a resolved entry with no choice as the enabled default.

* fix: hold auto-fill while the replacement page is in flight

A search or sort swap keeps the previous rows and hasNextPage on screen while the new first page loads, so the re-armed auto-fill asked for page two against a query that was still fetching its first. An infinite query runs one fetch at a time, so that request could cancel or interfere with the one already out. Both tables now pass their fetching state and the guard waits for it.

* fix: stop advertising links a deployment no longer serves

The sidebar badge rendered from the derived flag alone, so a deployment that turned shared links off still told owners a link was live while the public routes were unregistered.

The share dialog called the link a snapshot, but the payload populates the referenced message documents on every request: an edit to an already-shared message is visible immediately, and Update only adds newly referenced ones. The copy now says that.

Scroll pagination inspects a resolved error the way auto-fill already does, since React Query reports a failed page that way instead of rejecting.

* a11y: gate the shared conversation label on the feature flag

The icon stopped rendering when a deployment turns shared links off, but the row still announced the conversation as shared to screen readers. Both now read the same condition.

* fix: accept long title cursors and stop badge work the feature disables

The cursor cap was tight enough that a Name-sorted page ending on a long title produced a nextCursor the next request rejected, stranding the rest of the list. It now sits well clear of anything the server can issue.

The conversation list skipped straight into the shared-link lookup even where ALLOW_SHARED_LINKS is off, paying a round trip on the sidebar's first page for a badge that is never rendered.

A regeneration is newer than what it replaced, so only a newer sibling counts: an older one that still has follow-ups is the branch the target was regenerated away from, and resuming there published turns the target had excluded.

* fix: hold scroll pagination while a replacement page loads

Resetting the viewport to the top after a search or sort change fires a scroll event, and the retained previous rows still report another page, so the handler asked for page two of a query that was still loading page one.

* fix: keep the legacy share migration ahead of the owner-grant shortcut

A legacy row keeps its marker until every grant it needs exists, so an owner grant on its own is not proof the migration finished. Reading the marker first means a half-migrated public link still gets its public grant, while a fully migrated one keeps the read-only settled path the badge lookup depends on.
2026-08-09 08:14:54 -04:00
Marco Beretta
61016e328a
🔄 feat: Continue Shared Conversations as Personal Copies (#13714)
Adds a "Continue this chat" button to the shared conversation view that forks
the shared conversation into a new conversation owned by the viewer and opens it
to continue (issue #13001).

- POST /api/share/:shareId/fork, gated by requireJwtAuth, the fork rate
  limiters, and the canAccessSharedLink ACL (view access = fork access).
- forkSharedConversation clones from the anonymized getSharedMessages payload,
  so only share-visible data is copied.
- Strips file ids from cloned files/attachments so a fork grants no more file
  access than viewing the read-only share, and honors the global shared-file
  kill switch via the snapshotFiles option.
- Reduces the clone to the viewer's active branch, located by its index in the
  shared payload (shared ids are re-anonymized per request and createdAt can
  collide, while the payload order is stable).
- Resolves config/retention, persists, and reads back under the requesting
  user's tenant, not the share owner's; canAccessSharedLink also falls back to
  a system-wide share lookup so cross-tenant public shares resolve (ACL still
  enforced under the share's own tenant).
- Resolves a usable endpoint/model from the viewer's models config instead of
  hard-coding OpenAI, so deployments without OpenAI can send the first message.
- Routes the fork's 401s (logged-out or cold-loaded viewers) through login,
  including when the refresh itself is rejected for a stale session.
- Hides the Temporary Chat toggle once a conversation has a real id, and
  portals the share-settings theme/language dropdowns above the dialog.

Rebased onto dev; collapses the share-fork feature and its review fixes into a
single commit.
2026-06-24 16:27:01 -04:00
Danny Avila
788cc5ac07
🛟 fix: Auto-Recover from Stale Service Worker Assets After Deploys (#13686)
Some checks are pending
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
* 🛟 fix: Auto-Recover from Stale Service Worker Assets After Deploys

- 404 missing static assets in the SPA fallback instead of serving index.html
- inline recovery script unregisters stale SWs and reloads once on chunk failure
- route vite:preloadError into the same recovery path for stale lazy chunks

* 🛟 fix: Address Review — SW-Side Recovery, Scoped Unregister, Shared Fallback

- importScripts'd sw-heal.js pings window clients on activation and reloads
  ones that can't pong: stale pages carry no recovery code of their own
- scope SW unregistration to the deployment base for subpath installs
- preventDefault vite:preloadError only when a recovery reload was initiated
- extract createSpaFallback and apply the asset 404 guard to experimental.js
2026-06-11 11:57:06 -04:00
Ravi Kumar L
346ebea2d9
⚙️ refactor: brotli asset serving behind a feature toggle (#13641) 2026-06-10 08:49:50 -04:00
Danny Avila
ba5778d0df
📥 fix: ChatGPT Conversation Import Failures (#13637) 2026-06-09 19:04:21 -04:00
Danny Avila
9dd062e42e
🧯 fix: Harden Data Retention Semantics (#13049)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
* feat: support data retention for normal chats

Add retentionMode config variable supporting "all" and "temporary" values.
When "all" is set, data retention applies to all chats, not just temporary ones.
Adds isTemporary field to conversations for proper filtering.

Adapted to new TS method files in packages/data-schemas since upstream
moved models out of api/models/.

Based on danny-avila/LibreChat#10532

Co-Authored-By: WhammyLeaf <233105313+WhammyLeaf@users.noreply.github.com>
(cherry picked from commit 30109e90b0)

* feat: extend data retention to files, tool calls, and shared links

Add expiredAt field and TTL indexes to file, toolCall, and share schemas.
Set expiredAt on tool calls, shared links, and file uploads when
retentionMode is "all" or chat is temporary.

(cherry picked from commit 48973752d3)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: lint/test

(cherry picked from commit 310c514e6a)

* fix: address code review feedback for data retention PR

Critical:
- Fix BookmarkMenu crash: restore optional chaining on conversation
- Fix migration hazard: backward-compatible sidebar filter that also
  checks expiredAt for documents without isTemporary field

Major:
- Add logging to getRetentionExpiry error path, align with tools.js
- Add tests for retentionMode: ALL in saveConvo and saveMessage
- Fix share route: apply expiredAt for temporary chats too by
  querying the conversation's isTemporary flag server-side
- Add assertions for getRetentionExpiry mocks in process tests

Minor:
- Fix ChatRoute isTemporaryChat to be strictly boolean via Boolean()
- Fix stale test description (expired -> temporary)
- Comment out retentionMode default in example yaml
- Simplify verbose if/else to isTemporary === true
- Add compound index on { user: 1, isTemporary: 1 }
- Remove narrating comment from process.spec.js

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
(cherry picked from commit 6bad535f90)

* chore: fix typescript

(cherry picked from commit 826527a46b)

* fix: lint

(cherry picked from commit 77817e80ea)

* fix: use mockSanitizeArtifactPath in retention test

The 'getRetentionExpiry is called with the request object' test
referenced an undefined `mockSanitizeFilename` identifier, breaking
both lint (no-undef) and the test suite. Use the existing
`mockSanitizeArtifactPath` mock that the surrounding tests already
use, since `processCodeOutput` calls `sanitizeArtifactPath` (not
`sanitizeFilename`) before invoking `getRetentionExpiry`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit 52ea2da66d)

* fix: forward isTemporary from client for retention on file uploads and tool calls

Server-side `getRetentionExpiry` (file uploads) and the tool-call
controller both read `req.body.isTemporary`, but the file upload
multipart form and the tool-call payload did not include that field.
In `retentionMode: temporary` (default), files uploaded and tool
calls created from temporary chats were therefore retained
indefinitely.

Forward the Recoil `isTemporary` flag in both client paths so the
existing server checks can fire correctly. `ToolParams` gains an
optional `isTemporary` field.

Addresses Codex P1 review feedback on PR #29.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit 7e937df05a)

* test: stub store.isTemporary in useFileHandling test mocks

Previous commit added `useRecoilValue(store.isTemporary)` to the
hook. The test file mocks `~/store` with only `ephemeralAgentByConvoId`
and does not stub `useRecoilValue`, so all 7 cases threw
"Invalid argument to useRecoilValue: expected an atom or selector but
got undefined". Add a stub default export with `isTemporary` and a
`useRecoilValue` mock returning `false`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit eb1609537d)

* fix: harden data retention semantics

* fix: provide sweep request context for expired files

* fix: preserve temporary flags in all-retention updates

* fix: honor assistant versions in retention sweeps

* fix: retain non-temporary flags in all mode

* fix: hide expired retained records

* fix: propagate retained conversation expiry

* fix: refresh meili retention cutoff

* fix: prevent overlapping file sweeps

* fix: show legacy retained conversations

* fix: index legacy retained records

* fix: harden retention cleanup edge cases

* fix: count failed file storage sweeps

* fix: preserve legacy temporary retention

* fix: assign retention sweep worker deterministically

* fix: hide expired shared links on reads

* fix: prevent retention refresh after parent expiry

* fix: break code output retention import cycle

* fix: harden retention review findings

* fix: ignore expired share duplicates

* fix: reject expired retained share creation

* fix: harden retention review edge cases

* fix: address retention audit findings

* fix: enforce expired conversation shares in all retention

* fix: scope temporary upload flag to chat files

* fix: address retention review findings

* fix: address codex retention review findings

* fix: tighten missing storage detection

* test: remove unused file process spec bindings

---------

Co-authored-by: WhammyLeaf <233105313+WhammyLeaf@users.noreply.github.com>
Co-authored-by: Aron Gates <aron@muonspace.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-19 21:58:42 -04:00
Danny Avila
f2de3a219c
🌐 fix: Preserve Unicode Filenames (#12977)
* fix: Preserve unicode filenames

* fix: Cap unicode filenames by bytes

* fix: Preserve clean artifact directories

* fix: Disambiguate normalized artifact names
2026-05-06 14:57:38 -04:00
Danny Avila
65990a33e9
📥 fix: Resolve Imported-Conversation Default Model From Runtime modelsConfig (#12885)
* 📥 fix: Use Endpoint-Aware Default Model on Imported Conversations

Claude conversations imported from claude.ai's data export display
"gpt-4o-mini" in the chat UI until the page is refreshed, and any
attempt to send a message before refreshing fails with "The model
'gpt-4o-mini' is not available for Anthropic."

Root cause: ImportBatchBuilder.finishConversation() unconditionally
defaulted the saved conversation's `model` field to
openAISettings.model.default, regardless of `this.endpoint`. Claude
exports don't carry a model name, so every imported Claude conversation
landed with endpoint=anthropic but model=gpt-4o-mini.

Fix: pick the default based on `this.endpoint` via a small lookup
(openAI -> gpt-4o-mini, anthropic -> claude-3-5-sonnet-latest), keeping
the existing OpenAI default as the fallback for unknown endpoints.

Fixes #12844

* 🪄 refactor: Resolve Import Default Model From `modelsConfig`

Replace the hardcoded per-endpoint default lookup added in the previous
commit with a runtime resolver that consults the same models config the
chat UI uses (`getModelsConfig` in ModelController -> `loadDefaultModels`
+ `loadConfigModels`). This way an imported conversation defaults to a
model the LibreChat instance has actually configured / discovered for
the endpoint, instead of a hardcoded constant that may not exist on this
deployment.

Resolution order:
1. First non-empty model in `modelsConfig[endpoint]`.
2. Per-endpoint hardcoded fallback (anthropic/openAI settings) if the
   runtime config is empty for the endpoint or `getModelsConfig` throws.
3. `openAISettings.model.default` if even the per-endpoint fallback is
   missing (unknown endpoint).

`importBatchBuilder.finishConversation` now accepts an optional
`defaultModel` argument; each importer resolves it once at the top via
`resolveImportDefaultModel({ endpoint, requestUserId, userRole })` and
threads it through. ChatGPT message-level model selection also falls
back to the resolved default before the hardcoded gpt-4o-mini.
2026-04-30 00:43:04 -04:00
Marco Beretta
f6ee2ea0ee 📜 feat: Skills UI + Initial E2E CRUD / Sharing (#12580)
* 🎨 feat: Skills UI — Create/Edit/Share/List with Conditional File Tree

First-pass UI on top of the CRUD API scaffolding (#12613). Ships the full
user-facing flow for inline, single-SKILL.md skills and leaves a clean
drop-in for phase-2 multi-file support.

- Create a skill from /skills/new with name (kebab-case, validated),
  description, and SKILL.md body — wired to the real `useCreateSkillMutation`
  and `TCreateSkill` payload.
- List skills in a sidebar (SkillsSidePanel) via `useListSkillsQuery` with
  live search filtering.
- Edit any skill the caller has EDIT permission on — `useUpdateSkillMutation`
  passes `expectedVersion` for optimistic concurrency and surfaces 409
  conflicts as a warning toast + cache refetch.
- Non-blocking `TSkillWarning[]` (e.g. "description too short") are shown
  inline above the form after a successful create/patch.
- Read-only mode when the current user lacks EDIT — the form still renders
  but inputs are marked `readOnly` and the save/reset buttons are hidden.
- Share via ACL using the existing `GenericGrantAccessDialog` — the
  `ShareSkill` button is gated on the SHARE permission.
- Delete with confirmation, driven by `useDeleteSkillMutation({ id })`.
- Conditional file tree: only rendered when `useListSkillFilesQuery`
  returns > 0 files. The tree groups flat `relativePath` strings into a
  nested view (no `react-arborist` dependency) and supports per-file
  deletion via `useDeleteSkillFileMutation`. Upload is intentionally
  deferred — the backend stubs it at 501 in phase 1.

- New routes: `/skills`, `/skills/new`, `/skills/:skillId`.
- Sidebar accordion (`SkillsAccordion` wrapping `SkillsSidePanel`) added
  to `useSideNavLinks` gated on `PermissionTypes.SKILLS` USE.

The initial UI branch (#12580) shipped a lot of exploration code on top of
a now-superseded placeholder backend. Kept as complementary: the `Skills/`
component tree, translation keys, role descriptions, `PublicSharingToggle`
SKILL mapping, `resources.ts` SKILL config, `useCanSharePublic` SKILL
mapping, and `data-provider/roles.ts` `useUpdateSkillPermissionsMutation`.

Deferred out of this first pass:
- Skill favorites (`useSkillFavorites`, `getSkillFavorites` endpoint) —
  the backend route doesn't exist yet; saving for a follow-up.
- AgentConfig `SkillSelectDialog` integration — the UI branch had this
  gated behind `false &&`; rolled back with the config.
- `InvocationMode` / `CategorySelector` / `parseSkillMd` / tree-node
  mutations — not in the Anthropic skill spec and not in the CRUD API.
- `react-arborist` dependency — replaced with a hand-rolled recursive
  tree built from flat `TSkillFile[]`.

- 38 data-schemas skill model tests: pass
- 25 api skill route tests: pass
- 16 user-controller cleanup tests: pass

* 🔐 feat: Default-On Skills in Interface Config and Role Seeder

The skills accordion was registered in the side nav gated on
`PermissionTypes.SKILLS` USE, but no one was actually seeding that
permission on startup, so a fresh install had the USER role with
zero skill permissions and the accordion never rendered.

Fixes three gaps:

1. `interfaceSchema` in data-provider's `config.ts` had no `skills`
   field at all. Added it alongside the existing agents/prompts shape
   (boolean | { use, create, share, public }) and a default of
   `{ use: true, create: true, share: false, public: false }`.

2. `loadDefaultInterface` in data-schemas passed every interface key
   through to the loaded config EXCEPT `skills`. Added the one-line
   passthrough so `appConfig.interfaceConfig.skills` is actually
   populated on boot.

3. `updateInterfacePermissions` in packages/api/src/app/permissions.ts
   seeds role permissions from the interface config on every restart.
   Added:
   - `SKILLS` case to `hasExplicitConfig`
   - `skillsDefaultUse/Create/Share/Public` extraction (mirrors
     prompts/agents)
   - `PermissionTypes.SKILLS` block in `allPermissions` that falls
     through config → roleDefaults → schema default, same pattern as
     AGENTS and PROMPTS
   - `SKILLS` entry in the share-backfill array so that pre-existing
     SKILL role docs missing SHARE/SHARE_PUBLIC get them filled on the
     next restart

Test expectations updated: seven `expectedPermissionsFor(User|Admin)`
blocks in `permissions.spec.ts` now include SKILLS, matching the
role-default values (USER: use+create true, share/public false;
ADMIN: all true).

Result: on a fresh install, a regular USER gets skill USE/CREATE
and the "Skills" accordion shows up in the chat side panel without
any yaml config. Admins can lock it down per role or per tenant via
`interface.skills` in librechat.yaml.

Tests:
- 34 packages/api permissions.spec.ts: pass
- 151 packages/api app tests: pass
- 38 data-schemas skill.spec.ts: pass
- 928 data-provider tests: pass
- 25 api skills.test.js: pass

* ♻️ fix: Resolve Skills UI Review Findings

Addresses the 13 findings from the PR review against the prior commit.

1. **canEdit consistency** — extracted `useSkillPermissions(skill)` as the
   single source of truth for owner/admin/ACL gating. `SkillsView`,
   `SkillForm`, `ShareSkill` all consume it; `SkillFileTree`'s per-file
   delete button now honors admin + EDIT-bit permissions instead of just
   ownership. Unit tests cover owner, admin, editor-ACL, viewer-ACL,
   owner-ACL, loading, and undefined-skill cases.

2. **Disabled submit buttons** — create/edit form submit buttons now set
   native `disabled` (not just `aria-disabled`) during `isLoading`.
   `onSubmit` also guards with an early return when the mutation is still
   in-flight so a duplicate enter-key submit can't create two skills.

3. **Wrong maxLength error message** — description/name `maxLength` rules
   no longer re-use `com_ui_skill_*_required`. Added dedicated
   `com_ui_skill_name_too_long` and `com_ui_skill_description_too_long`
   keys with the literal limit interpolated (`{{0}}`).

4. **Search debouncing** — `SkillsSidePanel` now threads the filter input
   through the existing `useDebounce` hook (250ms) so typing "skills" no
   longer fires six separate list queries.

5. **Frontend test coverage** — added:
   - `tree.test.ts` (9 tests) covering `buildTree` / `nodeKey` edge cases:
     empty input, single root file, multiple roots, nested folders,
     deeply-nested trees, lexicographic sort, empty paths, stable keys
   - `useSkillPermissions.test.ts` (7 tests) covering every precedence
     branch (owner / admin / EDIT / VIEW / owner-ACL / loading / undef)

   Form integration tests proved flaky against react-hook-form's async
   `isValid` with our jest-dom mock setup; deferred to a follow-up PR
   with a proper `@librechat/client` test harness.

6. **Shared `SKILL_NAME_PATTERN`** — promoted the regex plus the four
   length constants (`SKILL_NAME_MAX_LENGTH`,
   `SKILL_DESCRIPTION_MAX_LENGTH`, `SKILL_DESCRIPTION_SHORT_THRESHOLD`,
   `SKILL_DISPLAY_TITLE_MAX_LENGTH`, `SKILL_BODY_MAX_LENGTH`) out of
   `packages/data-schemas/src/methods/skill.ts` and into
   `packages/data-provider/src/types/skills.ts`. The data-schemas
   module now aliases the shared exports so the backend validator and
   the frontend form share one source of truth. Also fixed a latent bug:
   the client regex was stricter than the backend
   (`^[a-z0-9]+(?:-[a-z0-9]+)*$` vs. the real `^[a-z0-9][a-z0-9-]*$`),
   which would have rejected valid names like `foo--bar` client-side.

7. **Removed hardcoded "Claude"** — replaced `com_ui_skill_description_help`
   ("Claude uses this to...") with a new `com_ui_skill_create_subtitle`
   for the form header and `com_ui_skill_description_field_hint`
   ("This is what the model reads to decide...") for the inline hint.
   LibreChat is LLM-agnostic; the old copy misled GPT/Gemini users.

8. **Lifted tree mutation hook** — `useDeleteSkillFileMutation` is now
   instantiated once in `SkillFileTree` (not per `TreeRow`). A
   `TreeContext` provides `onDeleteFile` + `isDeleting` + `canEdit` to
   rows. A 60-node tree used to instantiate 60 mutation hooks; it now
   instantiates one.

9. **List O(n) re-render** — `SkillListItem` no longer reads
   `useParams()` directly. `SkillList` reads the active id once and
   passes `isActive` as a prop, so navigation only re-renders the two
   items whose `isActive` flipped (memo'd), not all N items.

10. **Deduped help text** — the field-level hint and form-level subtitle
    now use different translation keys with distinct copy instead of
    showing the same sentence twice on the same page.

11. **Removed ineffective `useCallback`** — `DeleteSkill.handleDelete`,
    `CreateSkillForm.onSubmit` / `.handleCancel`, `SkillForm.onSubmit`,
    and `SkillFileTree.handleDeleteFile` all wrapped closures around
    React Query `mutation` refs, whose identities change every render.
    Their dep arrays invalidated every render, making the memo a no-op
    with extra overhead. `SkillFileTree` now destructures the stable
    `mutate` function and inlines the arrow inside the memoized
    `contextValue` — one stable reference per deps change.

12. **Import order** — fixed shortest→longest package ordering and
    longest→shortest local ordering across all touched skill files per
    AGENTS.md. `react` always first where imported.

13. **Memoization principle** — documented the rule with inline comments:
    `memo` on components that appear in repeated contexts (`TreeRow`,
    `SkillListItem`) or as children of frequently-re-rendering parents
    (`ShareSkill` / `DeleteSkill` under `SkillForm`'s per-keystroke
    form-state updates). Removed `memo` from `SkillFileTree` since its
    parent `SkillDetailPanel` only re-renders on query-data changes.

- 38 data-schemas skill.spec.ts
- 34 packages/api permissions.spec.ts
- 25 api skills.test.js
- 16 client unit tests (9 buildTree + 7 useSkillPermissions)
- All type-checks + eslint clean on touched files

* 🧹 fix: Skills Duplication, Input Styling, Remove LLM-specific Copy

Three UI fixes from an in-chat review pass:

1. **Sidebar duplication** — `SkillsView` was rendering its own
   `SkillsSidePanel` aside alongside the chat side panel's
   `SkillsAccordion`, so on `/skills` the user saw the skill list
   twice. Fixed by mirroring the `InlinePromptsView` pattern: the
   route content is now just the detail / create panel and the
   chat side panel is the sole list. Added `/skills → /skills/new`
   redirect and a `/skills/new` literal route so `useParams().skillId`
   is `undefined` for "new" (matches prompts).

2. **Name Input styling** — the big floating-label pattern used by
   prompts/agents for the primary name field was replaced with a
   conventional `<Label>` + `<Input>` above it, diverging from the
   rest of the app. Restored the prompts-style `text-2xl` input with
   the peer-focus animated label on both `CreateSkillForm` and
   `SkillForm`. Kept the conventional pattern for description and
   body since they're textareas.

3. **Remove LLM-specific copy from skill translations** — dropped
   `com_ui_skill_description_help` ("Claude uses this to...") and
   the transitional "This is what the model reads..." phrasing.
   Field hint is now a neutral "Be specific about when this skill
   should apply." and the create-page subtitle is a neutral "Author
   a new skill your agents can invoke." LibreChat is LLM-agnostic;
   baking product names into user-facing copy is wrong outside the
   `com_endpoint_anthropic_*` keys where the setting actually only
   applies to Claude models.

Side-effect: the `SkillDetailView` wrapper in `SkillsView` now only
renders the file-tree aside when the skill has > 0 files — same
conditional-tree behavior as before, just scoped to this route
instead of also trying to also render a list sidebar.

- 16 client skill tests still pass
- Type-check + eslint clean on touched files

* 🎁 feat: Restore Skills UI from PR #12580

Brings back everything the original UI PR (#12580, commit da039917c)
shipped that my earlier rebase dropped. Verbatim restores where possible;
adapts the new hooks/types where the backend contract has shifted.

**Scoped-out / gated-off (now restored as inert UI scaffolding):**
- `hooks/useSkillFavorites.ts` + `utils/favoritesError.ts` + the
  `useGetSkillFavoritesQuery` / `useUpdateSkillFavoritesMutation` additions
  in `data-provider/Favorites.ts`. The backend route doesn't exist yet —
  the data-service functions resolve with empty arrays so the Star UI is a
  visual-only no-op until phase 2.
- `dialogs/SkillSelectDialog.tsx` + the "Add Skills" section in
  `SidePanel/Agents/AgentConfig.tsx` (still gated behind the original
  `false &&`) + `skills?: string[]` on `AgentForm` / `Agent` /
  `AgentCreateParams` / `AgentUpdateParams` + the `skills: []` entry in
  `defaultAgentFormValues`.
- `TUserFavorite.skillId` reserved on the shared favorites type.

**Concept-is-gone / deleted-types (restored as UI-only types + stubs):**
- `InvocationMode` enum and `TSkillNode`, `TSkillTreeResponse`,
  `TCreateSkillNodeRequest`, `TUpdateSkillNodeRequest` types in
  `packages/data-provider/src/types.ts`. UI-facing only; the backend flat
  `TSkillFile[]` contract is unchanged.
- `TSkill.invocationMode?: InvocationMode` as an optional field. Forms
  read/write it in local state and deliberately drop it from the PATCH
  payload until the backend column lands.
- `tree/SkillFileTree.tsx` (`react-arborist`-based), `SkillTreeNode.tsx`,
  `TreeToolbar.tsx`, `SkillFileEditor.tsx`, `SkillFilePreview.tsx` — full
  filesystem-style browser UI restored verbatim.
- `data-provider/Skills/tree-queries.ts` + `tree-mutations.ts` hooks
  (`useGetSkillTreeQuery`, `useCreateSkillNodeMutation`, etc.). The
  `data-service` stubs them: `getSkillTree` returns `{ nodes: [] }`,
  `createSkillNode` / `updateSkillNode` / `updateSkillNodeContent` return
  synthetic node shapes, `deleteSkillNode` resolves void. Hooks compile
  and run; tree is empty until phase 2 wires a real backend.
- `MutationKeys.createSkillNode` / `updateSkillNode` / `deleteSkillNode` /
  `updateSkillNodeContent` + `CreateSkillNodeBody` /
  `UpdateSkillNodeVariables` / `DeleteSkillNodeBody` /
  `UpdateSkillNodeContentVariables` types.
- `QueryKeys.skillTree` / `skillNodeContent` / `skillFavorites` /
  `favorites` and the `skillTree()` endpoint helper.

**Scope-simplified (restored with minimal adaptation):**
- `display/SkillDetailHeader.tsx` + `display/SkillDetail.tsx`. Header now
  falls back to `InvocationMode.auto` when `skill.invocationMode` is
  undefined.
- `forms/SkillContentEditor.tsx` — click-to-edit markdown preview toggle
  for the SKILL.md body field. Wired into both `CreateSkillForm` and
  `SkillForm` replacing the plain `<TextareaAutosize>`.
  (Needed `@ts-ignore` on `remarkPlugins` / `rehypePlugins` for the same
  `PluggableList` vs `Pluggable[]` shape drift `MarkdownLite.tsx` already
  works around.)
- `forms/InvocationModePicker.tsx` + `forms/CategorySelector.tsx` — the
  auto/manual/both dropdown and the skill category selector. Wired into
  both forms inside a `FormProvider` so the Controller-based widgets can
  read `useFormContext`. `category` flows to the PATCH / POST payload as
  before; `invocationMode` is UI-only per the type note above.
- `buttons/CreateSkillMenu.tsx` + `utils/parseSkillMd.ts` — dropdown with
  AI / Manual / Upload SKILL.md entries + the YAML frontmatter parser for
  the upload path. `CreateSkillForm.defaultValues` now accepts the parsed
  shape, so the upload → redirect → pre-populated form flow works again.
- `buttons/AdminSettings.tsx` — admin permissions dialog. Uses the
  existing `useUpdateSkillPermissionsMutation` which was already wired.
- `sidebar/FilterSkills.tsx` — restored filter + AdminSettings +
  CreateSkillMenu wrapper. `SkillsSidePanel.tsx` is back to the original
  `FilterSkills`-based layout.
- `lists/SkillList.tsx` + `lists/SkillListItem.tsx` — restored verbatim.
- `layouts/SkillsView.tsx` — restored the full tree + file editor + file
  preview layout. The chat side panel keeps its own accordion list; this
  view is the inline detail experience.
- `hooks/Generic/useUnsavedChangesPrompt.ts` — route-leave guard hook.

- `useGetSkillByIdQuery` is aliased to `useGetSkillQuery` so restored
  components (`SkillsView`, `SkillForm`) that import the old name resolve
  to the new hook.
- `SkillSelectDialog` + `AgentConfig` coerce `skillsData?.skills` instead
  of `.data` (list response shape drift from the CRUD PR).
- `CreateSkillForm` / `SkillForm` wrap their JSX in `FormProvider` so the
  restored `CategorySelector` and `SkillContentEditor` components —
  which read `useFormContext` — work inside the existing forms without
  another refactor.
- `CreateSkillForm.defaultValues` prop accepts `Partial<Values> &
  { invocationMode?: unknown }` so the upload flow's
  `{ name, description, invocationMode }` shape passes through cleanly.
- `SkillsView` route map gains `/skills/:skillId/edit` and
  `/skills/:skillId/file/:nodeId` so the tree-navigation URLs the original
  view produces actually resolve.
- `client/package.json` gains `react-arborist@^3.4.3`.
- ~60 translation keys the restored files reference — invocation labels,
  edit/create page titles, file editor chrome, tree toolbar tooltips,
  favorites, admin allow-settings, unknown-file-type, sr_public_skill,
  delete/rename _var variants — all added to `en/translation.json`.

- Prompts-style floating-label name input — kept from my earlier commit
  so it matches the rest of the app (user reviewed and approved that
  styling). Hidden skill-body textarea is replaced by `SkillContentEditor`
  in both forms.

- 38 data-schemas skill.spec.ts
- 34 packages/api permissions.spec.ts
- 25 api skills.test.js
- 7 client useSkillPermissions.test.ts
- Type-check: pre-existing error count (188) dropped to 120 because my
  restorations fixed some previously-broken field types.

* chore: Update package-lock.json to include react-arborist and memoize-one

* feat: Add support for react-arborist in Vite configuration

This update introduces a new condition in the Vite configuration to handle the 'react-arborist' package, ensuring it is properly recognized during the build process. This change enhances compatibility with the recently added 'react-arborist' dependency in the project.

* 🩹 fix: Hide InvocationMode, Fix SkillContentEditor Click-to-Edit

1. Hide InvocationModePicker from both CreateSkillForm and SkillForm.
   Component stays on disk for when the backend lands the column.

2. Fix "Click to edit" doing nothing on SkillContentEditor. The
   `onBlur={() => setIsEditing(false)}` on the TextareaAutosize was
   racing with `autoFocus` — React renders the textarea, autoFocus
   fires, then a layout/reconciliation blur fires immediately,
   bouncing back to preview mode before the user can interact.
   Removed onBlur; users toggle via the header button or Escape key.

* 🎨 feat: Reader-First Skills UI — Match Claude.ai Layout

Reworks the Skills UI from form-first to reader-first, matching
Claude.ai's skill detail pattern.

**Default view is now read-only.** Clicking a skill in the sidebar
navigates to `/skills/:id` which renders `SkillDetail` — a clean
content view with:
- Skill name as the primary heading
- Metadata row: "Added by" + "Last updated" (formatted date)
- Description block
- Rendered SKILL.md body in a bordered card with a source/rendered
  toggle (eye + code icons, matching Claude.ai's segmented control)

No form fields, no save/cancel buttons. The user reads the skill
first and takes action deliberately.

**Create is now a dialog.** The `/skills/new` route is gone.
`CreateSkillMenu` (the + dropdown in the sidebar) now opens
`CreateSkillDialog` — a minimal modal with name, description, and
instructions fields. Upload-from-file still works: parse → populate
dialog → create. Matches Claude.ai's "Write skill instructions"
modal.

**Edit is behind an action.** The detail view shows an "Edit" button
(permission-gated) that navigates to `/skills/:id/edit`, rendering
the existing `SkillForm`. The edit route is preserved for direct
linking.

**Navigation goes to detail, not edit.** `SkillListItem` now
navigates to `/skills/:id` (detail) instead of `/skills/:id/edit`.

- `display/SkillMarkdownRenderer.tsx` — shared ReactMarkdown
  component extracted from `SkillContentEditor`. Same remark/rehype
  plugins, no form dependency.
- `display/SkillDetail.tsx` — the reader-first view (replaces the
  old thin wrapper).
- `dialogs/CreateSkillDialog.tsx` — OGDialog modal for skill
  creation.

- `layouts/SkillsView.tsx` — gutted and rebuilt. Three states:
  no-skill (empty state), skillId (SkillDetail), skillId+edit
  (SkillForm). Removed full-page CreateSkillForm, removed TreeView.
- `buttons/CreateSkillMenu.tsx` — opens dialog instead of navigating
  to `/skills/new`. Upload flow: parse → set dialog defaults → open.
- `lists/SkillListItem.tsx` — navigate to detail, not edit.
- `routes/index.tsx` — removed `/skills/new` and file/nodeId routes;
  `/skills` renders SkillsView directly (empty state).
- `display/index.ts`, `dialogs/index.ts` — added new exports.
- `locales/en/translation.json` — added ~10 new keys for metadata,
  toggle labels, dialog title, empty state.

* 🩹 fix: SkillContentEditor click-to-edit z-index — button was z-0 behind rendered content

* 🩹 fix: Align Edit button size with Share/Delete (size-9)

* 🎨 feat: Claude.ai-Style Skill List Panel

Rewrites the skills sidebar to match Claude.ai's panel layout:

- Header: "Skills" title + search icon (toggles input) + add icon
  (opens CreateSkillDialog directly, no dropdown menu)
- Collapsible "Skills" section with chevron toggle
- Skill items: 24px icon badge (rounded square with ScrollText icon)
  + name only. No description text in the list — that lives in the
  detail view. Active item gets highlighted bg + bold font.
- Removed AdminSettings button from sidebar header — admin config
  is accessible via the admin dashboard, not cluttering every user's
  skill list.
- Removed FilterSkills wrapper (was Filter + AdminSettings +
  CreateSkillMenu). The search + create are now inline in the panel
  header.

Files changed:
- sidebar/SkillsSidePanel.tsx — full rewrite
- sidebar/SkillsAccordion.tsx — simplified wrapper
- lists/SkillList.tsx — collapsible section, no description
- lists/SkillListItem.tsx — icon badge + name, memo'd

* 🎨 fix: Align Skills UI Styling with Prompts Patterns

Style alignment pass based on direct comparison with claude.ai and
the existing prompts preview dialog.

SkillsSidePanel search now replaces the title in the header row when
toggled (search icon + input + X close), matching Claude.ai's pattern.
Previously it pushed a separate input below the header, wasting
vertical space. Close button clears the search term.

Replaced `text-text-tertiary` with `text-text-secondary` across
SkillDetail, SkillList, SkillForm, CreateSkillForm, CreateSkillDialog,
SkillContentEditor. Tertiary was too dark / low contrast.

SkillList section chevron label now reads "Personal skills" (matching
Claude.ai) via the existing `com_ui_my_skills` key, instead of the
generic "Skills" which duplicated the header.

Aligned with `PromptDetailHeader` styling:
- 48px round icon (ScrollText in bg-surface-secondary circle)
- Name + public badge in the icon row
- Metadata below the icon: User icon + author, Calendar icon + date
  (text-xs text-text-secondary with gap-3, matching prompts exactly)
- Description uses the same label-above-text pattern as prompts
- Content card uses `bg-transparent` border (not bg-surface-primary-alt)
- Toggle buttons use size-5 icons and text-text-secondary for inactive

Changed from `max-w-lg p-0` to `max-w-5xl` with the same max-height
and padding pattern as the prompts PreviewPrompt dialog:
`max-h-[80vh] p-1 sm:p-2 gap-3 sm:gap-4`. Close button now renders
via default OGDialogContent behavior (removed showCloseButton=false).

* 🩹 fix: SkillDetail fills parent height, tighter spacing (px-6 pb-6 gap-2)

* 🩹 fix: Align Skills panel header padding (px-4) with list content below

* 🩹 fix: Reduce Skills header top padding (pt-2) to align with sidebar icon strip

* 🩹 fix: Tighten Skills header (py-2) and detail top (py-2) to align with sidebar icons and match edit view

* 🩹 fix: Offset SidePanel Nav pt-2 with -mt-2 on SkillsAccordion so Skills header aligns with icon strip

* 🛠️ fix: Increase Node memory limit for production build in package.json

* 🩹 fix: Remove top padding from SkillDetail header row (py-2 → pb-2)

* 🏗️ refactor: Move pt-2 from SidePanel/Nav wrapper to each panel

Removed the global `pt-2` from `SidePanel/Nav.tsx` and pushed it
into each panel's own top-level wrapper. This lets each panel own
its vertical alignment independently — Skills can sit flush at the
top to align with the sidebar icon strip, while other panels keep
their original spacing.

Panels updated with `pt-2`:
- PromptsAccordion (via className on PromptSidePanel)
- BookmarkPanel
- FilesPanel
- MemoryPanel
- MCPBuilderPanel
- AgentPanel (form wrapper)
- AssistantPanel (form wrapper)
- ParametersPanel (already had pt-2)

SkillsAccordion: removed the -mt-2 hack, now naturally flush.

* 🧹 fix: Align CreateSkillDialog field styling + remove 19 unused i18n keys

Dialog fields: all three inputs now use consistent `rounded-xl
border-border-medium px-3 py-2 text-sm` styling. Replaced the
`<Input>` component with a plain `<input>` to avoid the component's
built-in `rounded-lg border-border-light` overriding the dialog's
border style. Labels use `font-medium` for consistency.

Removed 19 unused translation keys from translation.json:
com_ui_skill_body, com_ui_skill_body_placeholder,
com_ui_skill_create_subtitle, com_ui_skill_file_delete_confirm,
com_ui_skill_file_delete_error, com_ui_skill_file_deleted,
com_ui_skill_files_empty, com_ui_skill_files_multi_hint,
com_ui_skill_list, com_ui_skill_load_error,
com_ui_skill_resize_file_tree, com_ui_skill_select_file,
com_ui_skill_select_file_desc, com_ui_skills_load_error,
com_ui_add_first_skill, com_ui_create_skill_page,
com_ui_edit_skill_page, com_ui_save_skill, com_ui_no_skills_title

* 🎁 feat: Upload Skill Dialog + Simplified Create Menu

New `UploadSkillDialog` matching Claude.ai's upload modal:
- Dashed drop zone with drag-and-drop support
- Accepts .md, .zip, .skill files
- Phase 1: processes .md files (parses YAML frontmatter → creates
  skill with body as the full file content)
- Shows file requirements below the drop zone
- On success: navigates to the new skill's detail view

`CreateSkillMenu` now has two flat options (no sub-menu):
- "Write skill instructions" → opens `CreateSkillDialog`
- "Upload a skill" → opens `UploadSkillDialog`

Removed the disabled "Create with AI" option and the old file input
hidden-element approach. The sidebar `+` button now renders
`CreateSkillMenu` directly instead of a standalone create dialog.

- Removed 5 unused i18n keys (com_ui_skill_added_by,
  com_ui_skill_last_updated, com_ui_skills_add_first,
  com_ui_skills_filter_placeholder, com_ui_skills_new)
- Tightened metadata gap in SkillDetail (mt-1 → mt-0.5)
- Added 7 new upload-related i18n keys

* 🔒 feat: Zip/Skill File Upload Support with Safety Limits

Rewrites UploadSkillDialog to properly handle all three accepted
file types:

- `.md` — reads as text, parses YAML frontmatter, creates skill
- `.zip` / `.skill` — reads as ArrayBuffer, extracts with JSZip,
  finds SKILL.md (at root or one level deep), parses its content,
  creates skill. Shows spinner during processing.

Security guards against zip bombs:
- MAX_ZIP_SIZE: 50MB compressed file limit
- MAX_ENTRIES: 500 file limit inside the archive
- Path traversal rejection: skips entries with `..` or leading `/`
- SKILL.md search limited to depth ≤ 2 segments

Added `jszip@^3.10.1` to client dependencies (already in the
monorepo's node_modules from backend usage).

The name is inferred from the zip filename if SKILL.md frontmatter
doesn't have one (e.g. `skills-autofix.zip` → `skills-autofix`).

* 🚀 feat: Backend Skill Import + Live File Upload Endpoints

New endpoint that accepts a single multipart file (.md, .zip, .skill)
and creates a skill with all its files in one request:

- **.md**: parse YAML frontmatter → create skill with body
- **.zip / .skill**: extract with JSZip, find SKILL.md (root or one
  level deep), create skill from its content, then persist every
  additional file via `upsertSkillFile` + local file storage strategy.
  Returns the created skill + an `_importSummary` with per-file
  results.

Security:
- 50MB compressed file size limit (multer)
- 500 max entries in archive
- 10MB per individual file
- Path traversal rejection (no `..`, no absolute, validated charset)
- File type filter: only .md/.zip/.skill accepted
- Rate limited via existing `fileUploadIpLimiter` + `fileUploadUserLimiter`

Handler lives in `packages/api/src/skills/import.ts` with injectable
deps (`createSkill`, `upsertSkillFile`, `saveBuffer`) for testability.

Replaced the 501 stub with a real handler:
- Accepts multipart FormData with `file` + `relativePath`
- Saves file via local storage strategy
- Calls `upsertSkillFile` to persist the SkillFile record
- Returns the upserted document
- Rate limited, ACL-gated (EDIT permission required)
- 10MB per file limit

`UploadSkillDialog` now sends the file to `/api/skills/import` via
`dataService.importSkill(formData)` — no more client-side JSZip.
Removed `jszip` from client dependencies (only backend needs it).

Added `importSkill()` in data-service + `importSkill()` endpoint
builder in api-endpoints.

Updated the file upload test from expecting 501 stub to expecting
400 "no file provided" (live validation). All 25 skill route tests
pass.

* 🔒 fix: Complete Import Handler — Validation, Ownership, Error Surfacing

Fixes several gaps in the skill import flow:

1. **Skill validation now runs and surfaces properly.** The import
   handler calls the real `createSkill(CreateSkillInput)` which runs
   `validateSkillName`, `validateSkillDescription`, `validateSkillBody`.
   Validation errors (SKILL_VALIDATION_FAILED) are caught and returned
   as 400 with the issue messages. Duplicate-key errors return 409.
   Previously all errors were swallowed into a generic 500.

2. **`authorName` is now populated.** The `CreateSkillInput` requires
   `authorName` which was missing — resolved from `req.user.name ??
   req.user.username ?? 'Unknown'`, matching the existing create handler.

3. **SKILL_OWNER permission is granted after import.** Calls
   `grantPermission` with `AccessRoleIds.SKILL_OWNER` so the uploader
   can edit/delete/share the imported skill. This was entirely missing —
   imported skills would have been ownerless.

4. **`tenantId` propagated.** Both the skill and each SkillFile record
   receive `req.user.tenantId` for multi-tenant deployments.

5. **SkillFile records are created in the DB.** Each non-SKILL.md file
   in the zip is saved to file storage via `saveBuffer` and recorded
   via `upsertSkillFile`, which validates the relativePath, infers the
   category from the path prefix, and atomically bumps the skill's
   `fileCount` and `version`.

Import deps now include `grantPermission` from PermissionService,
injected in `api/server/routes/skills.js`.

* 🐛 fix: Import grant uses accessRoleId (not roleId) — fixes skill not appearing in list

* 🎨 fix: Cache invalidation, file tree, frontmatter rendering

Three fixes for the skill detail view:

1. **Cache invalidation after import.** UploadSkillDialog now calls
   `queryClient.invalidateQueries([QueryKeys.skills])` after a
   successful import so the sidebar list picks up the new skill
   without requiring a page refresh.

2. **File tree in detail view.** When a skill has `fileCount > 0`,
   the detail view now queries `useListSkillFilesQuery` and renders
   a file list below the body card — SKILL.md first, then folders
   and root files. Icons: Folder for directories, FileText for files.

3. **Frontmatter stripped and rendered as metadata.** YAML frontmatter
   (`---\nversion: 0.1.0\ntriggers: ...\n---`) is now parsed out of
   the body before markdown rendering. The `name` and `description`
   fields are skipped (already shown in the header). Remaining fields
   (version, triggers, dependencies, etc.) are displayed in a
   Claude.ai–style grid: label on the left, value on the right,
   above the rendered markdown content. Source view still shows the
   full raw body including frontmatter.

* 🩹 fix: Always fetch skill files — fileCount may be stale in cached skill object

* 🌳 feat: Inline File Tree in Sidebar Skill List

Moves the file tree from the bottom of SkillDetail into the sidebar
list, matching Claude.ai's pattern:

- Multi-file skills show a chevron toggle on the right side of the
  skill list item
- Clicking the chevron expands an inline file tree below the skill
  name: SKILL.md first, then folders (with folder icon + right
  chevron) and root files
- File list is fetched lazily (only when expanded) via
  useListSkillFilesQuery
- Clicking a file navigates to the skill detail view
- Files section removed from SkillDetail — the sidebar is now the
  sole file tree location, keeping the detail panel clean

SkillDetail cleaned up: removed groupFiles helper, file-related
state, useListSkillFilesQuery import, FileText/Folder icon imports.

* 🌲 feat: Virtualized inline file tree with react-vtree

Replace hand-rolled recursive FolderRow/FileRow buttons with a proper
virtualized FixedSizeTree from react-vtree for the sidebar skill list.
Dynamic height tracks open folders; capped at 350px with smooth
expand/collapse transitions.

* chore: Remove no longer used SkillFileTree and SkillTreeNode components

* chore: Update Vite config to replace 'react-arborist' with 'react-vtree' for module resolution

* feat: Skill file content viewing with lazy DB caching

- Add `skills` field to `fileStrategiesSchema` so operators can
  configure a dedicated storage backend for skill files. Falls back
  by type (image/document) when unset.
- Fix hardcoded `FileSources.local` in skill save/import — now uses
  the resolved strategy via `getFileStrategy(req.config, { context })`.
- Replace 501 download stub with real handler that streams from any
  storage backend and returns JSON `{ content, mimeType, isBinary }`.
- Binary detection (null-byte + non-printable ratio on first 8 KB)
  flags files on first read so they're never re-fetched.
- Text content ≤ 512 KB is cached in the SkillFile MongoDB document;
  subsequent reads skip storage entirely.
- Clicking a skill row now expands inline files (not just chevron).
- Clicking a file navigates to `?file=<path>` and renders content
  in a new SkillFileViewer (markdown, code, images, binary placeholder).

* chore: Remove react-window and its type definitions from package.json and package-lock.json

- Deleted `react-window` and `@types/react-window` dependencies from both `package.json` and `package-lock.json` to streamline the project and reduce unnecessary bloat.

* fix: Build errors — remove endpoints import, fix Uint8Array cast

- Replace `import { endpoints }` (not public) with inline URL in
  SkillFileViewer
- Remove `as Uint8Array` cast in stream chunk handling
- Extend getSkillFileByPath return type with content/isBinary to
  decouple from data-schemas build artifact resolution

* chore: Remove 8 unused i18next keys

com_ui_create_skill_ai, com_ui_create_skill_manual,
com_ui_delete_folder_confirm_var, com_ui_delete_skill,
com_ui_delete_skill_confirm_var, com_ui_delete_var,
com_ui_rename_var, com_ui_skill_files

* fix: Add configMiddleware to skills router, handle SKILL.md in viewer

- Add configMiddleware to skills router so req.config is populated
  when getLocalFileStream (or any strategy) reads file paths.
- Handle SKILL.md in download handler — serves skill.body directly
  from the Skill document instead of looking for a SkillFile record.
- Clicking SKILL.md in sidebar tree now opens the file viewer
  (matching Claude.ai behavior: file view vs default detail view).

* ci: Run unit tests on PRs to any branch

Remove the branches filter from both test workflows so contributor
PRs targeting feature branches (not just main/dev) get CI coverage.
Path filters are kept so tests only run when relevant files change.

* fix: Update skills route tests for download handler changes

- Mock configMiddleware (sets req.config for file storage access)
- Mock getStrategyFunctions and getFileStrategy (storage strategy deps)
- Replace 501 stub test with SKILL.md content test + 404 test

* fix: Auto-expand files, frontmatter parsing, select-none, prefetch

- Auto-expand file tree when navigating directly to a skill URL
- Prefetch files for the active skill (eliminates first-expand lag)
- Fix frontmatter parser to handle multi-line YAML list values
  (triggers field was missing because it uses list syntax)
- SkillFileViewer now parses frontmatter for .md files — shows
  structured grid + rendered body (matching SkillDetail's display)
  with source/rendered toggle
- Add select-none to all sidebar skill and file tree buttons

* refactor: Derive expanded state from isActive instead of useEffect

Replace useEffect sync with deterministic derivation:
expanded = hasFiles && (isActive || !collapsed)

Active skill is always open. collapsed is a manual toggle that
only takes effect on non-active items.

* fix: Remove empty space above body card — overlay view toggle

Move the rendered/source toggle from a dedicated row (40px of empty
space) to an absolute-positioned overlay in the card's top-right
corner, matching Claude.ai's layout.

* fix: Remove header bars from content editors — overlay action buttons

Collapse the full-width header bars ("Skill Content", "Text") in
SkillContentEditor, PromptTextCard, and PromptEditor. Action buttons
(edit/save toggle, copy, variables) are now absolute-positioned in
the card's top-right corner, reclaiming ~46px of vertical space.

* fix: Spinner visibility in file viewer — use text-text-secondary

* fix: Address review findings — security, correctness, code quality

Codex P1: Use $unset instead of undefined to clear cached content
and isBinary fields on file re-upload (Mongoose strips undefined).

Codex P2: Match skill-file validation errors by error.code instead
of error.message substring.

F1: Zip bomb defense — track cumulative decompressed bytes (500 MB
cap), check declared uncompressed size before buffering each entry.

F2: Remove misleading "atomically" from import handler JSDoc.

F3: Static import for isBinaryBuffer instead of dynamic import().

F4: Replace console.error with logger in upload handler.

F6: Add multer error handler middleware to skills router.

F7: Move React import to top of SkillDetail.tsx.

F9: Fix variable shadowing (trimmed → item) in parseFrontmatter.

F11: Replace JSON.parse(JSON.stringify()) with toJSON() for
Mongoose document serialization.

F12: Remove dead dynamic import('fs') fallback (memoryStorage
always provides file.buffer).

F13: Hoist MIME_MAP to module scope to avoid per-call allocation.

F16: Share single multer.memoryStorage() instance.

* fix: Follow-up review — close zip bomb gap, fix error handler

F1: Add post-decompression cumulative byte check with break (the
pre-decompression check relies on undocumented JSZip internals
that may be absent; this closes the gap unconditionally).

F2+F3: Multer error handler now forwards non-multer errors via
next(err) instead of swallowing them. Also catches file filter
rejections (plain Error, not MulterError) by message prefix.

F4: Move isBinaryBuffer import to local imports section per
CLAUDE.md import order rules.

F5: Simplify dead toJSON branch — createSkill returns a POJO.

* nit: Link filter error message to handler prefix check

* feat: Accordion expansion + active file highlight in sidebar

- Only one skill's file tree can be expanded at a time (accordion).
  Expansion state lifted from SkillListItem to SkillList.
- Selected file gets bg-surface-active highlight in the tree.
  Skill row uses subtle style (no background) when a file is active,
  matching Claude.ai's pattern where the file — not the skill —
  carries the selection state.

* style: Adjust margin for file tree in SkillListItem component

- Reduced left margin from 10 to 5 for improved layout consistency in the file tree display.

* fix: TS error on FileTreeNode, nested ternary, chevron collapse

- Make style prop optional to match react-vtree's NodeComponentProps
- Flatten nested ternary for skill row active styles
- Skill row click expands (but doesn't collapse) files + navigates
- Chevron click explicitly toggles collapse (matching Claude.ai
  where clicking the chevron is how you collapse files)

* fix: Upload basePath, reject SKILL.md uploads, add skills permission route

- Pass basePath: 'uploads' in per-file upload handler (was defaulting
  to 'images' path, inconsistent with the import flow).
- Reject uploads targeting SKILL.md (reserved path — download handler
  special-cases it to return skill.body, making an uploaded file
  unreachable via the API).
- Add skills entry to roles router permissionConfigs so PUT
  /api/roles/:roleName/skills actually reaches a handler instead
  of returning 404.

* feat: Expand content area, move controls to header, reduce padding

Default detail view:
- Remove rounded-xl bordered card wrapper — content flows directly
  into the article, capitalizing on full screen width
- Move eye/code toggle inline with the divider row
- Reduce px-6/pb-6 to px-4/pb-4

File viewer:
- Move eye/code toggle from card overlay to the header bar
- Add copy-to-clipboard button for text files in the header bar
- Remove rounded-xl bordered card wrapper for markdown content
- Remove bordered pre wrapper for non-markdown text
- Reduce px-6/py-4 to px-4/py-3

Both views maximize content space over decorative chrome.

* fix: Stable header height, restore some padding

- Fix layout shift in file viewer header: use fixed h-10 so the
  bar height stays constant whether the eye/code toggle renders
  (markdown) or not (plain text).
- Bump content padding from px-4/py-3 back to px-5/py-4 in both
  views — the previous reduction was too aggressive.

* fix: Grant rollback, path validation, error format, dead code cleanup

F2: grantOwnership now rolls back (compensating delete) on failure,
matching the create handler. Both markdown and zip import paths
check the result and return 500 on grant failure.

F4: Upload handler validates relativePath with regex + traversal
check before calling downstream upsertSkillFile.

F5: Document JSZip _data.uncompressedSize as best-effort; the
post-decompression cumulative check is the real safety net.

F10: Standardize all upload handler error responses to { error }
(was { message }, inconsistent with handlers.ts).

F13: Single-pass fileResults accumulation in import handler.

F1-5: Remove dead uploadFileStubHandler (no route references it).

Codex P2: Fix delete nav from /skills/new to /skills.

F12: Use cn() in UploadSkillDialog instead of template literals.

* perf: Stream-first binary detection + O(1) public skill check

F1: Download handler now reads only the first 8 KB for binary
detection. If binary, the stream is destroyed immediately without
buffering the remaining file. Text files continue reading for
caching. Eliminates buffering up to 10 MB per request for binary
files under concurrent load.

F7: Single-skill GET and PATCH now use hasPublicPermission (O(1)
ACL lookup) instead of getPublicSkillIdSet (queries ALL public
skill IDs). The list handler still uses the Set approach since it
serializes multiple skills. serializeSkill/serializeSkillSummary
now accept boolean | Set for flexibility.

* fix: Update test to match { error } response format

* fix: Critical stream truncation bug, grantedBy, error format

NF-1 (CRITICAL): Rewrite binary detection to single for-await loop.
Breaking out of for-await-of destroys the stream via iterator.return(),
so the previous two-loop approach silently truncated text files > 8KB.
Now: one loop collects chunks, checks binary after 8KB accumulated,
and either destroys+returns (binary) or continues reading (text).

NF-2: Add grantedBy to import handler's grantPermission call and
interface (was missing, inconsistent with create handler).

NF-3: Standardize all import handler error responses from { message }
to { error }, matching handlers.ts convention. Update client's
UploadSkillDialog to read response.data.error accordingly.

* fix: Prefer specific validation message over generic error field

* fix: YAML quote stripping, saveBuffer null guard, dot segment rejection

- Strip surrounding YAML quotes from frontmatter values so
  name: "my-skill" parses as my-skill (not "my-skill" with quotes
  that fails the name validator).
- Guard resolveSkillStorage against backends with saveBuffer: null
  (e.g. OpenAI/vector strategies) — throws a descriptive error
  caught by the handler's try/catch instead of a TypeError.
- Tighten upload path validation to reject . segments (e.g.
  docs/./a.md) matching the model-layer validator, preventing
  storage writes for paths the DB will reject.

* fix: Orphan cleanup, stream errors, malformed zip, cache latency

F1: Upload handler now deletes the stored blob if the subsequent
DB upsert fails, preventing orphaned files on disk/cloud.

F2: Multer error handler returns { error } (was { message }).

F3: Wrap JSZip.loadAsync in try/catch — malformed zip returns 400
instead of falling through to 500.

F4: Raw download stream gets an error handler — logs the error and
destroys the response if headers were already sent.

F8: Strip leading hyphens from inferred skill name so filenames
like _my-skill.zip don't produce -my-skill (invalid name pattern).

F9: Fire-and-forget all updateSkillFileContent cache writes so the
response is sent immediately. Cache failures are logged but don't
block or fail the read.

* fix: Import orphan cleanup + Content-Disposition sanitization

Finding A: Add deleteFile dep to ImportSkillDeps. The per-file loop
in handleZip now cleans up stored blobs when upsertSkillFile fails,
closing the second half of the F1 orphan fix (upload handler was
already fixed).

Finding B: Sanitize filename in Content-Disposition header for raw
downloads — strip quotes, backslashes, and newlines to prevent
header injection from user-uploaded filenames.

* security: Prevent stored XSS via raw file downloads

Non-image files served via ?raw=true now use Content-Disposition:
attachment (force download) instead of inline. An uploaded .html or
.svg file served inline from the LibreChat origin could execute
scripts with access to the user's session — this closes that vector.

Images stay inline (needed for <img> rendering in SkillFileViewer).
X-Content-Type-Options: nosniff added to prevent MIME sniffing.

* security: Block SVG XSS — allowlist safe raster MIME types for inline

SVG (image/svg+xml) passed the startsWith('image/') check and was
served inline, but SVG is a scriptable format — embedded <script>
tags execute in the LibreChat origin. Replace the prefix match with
a Set of safe raster-only MIME types (png, jpeg, gif, webp, avif,
bmp). SVGs and any future scriptable image/* subtypes now get
Content-Disposition: attachment (forced download).

* fix: Cap JSON text response at 1MB, consistent md name inference

F3: Text files > 1MB now return { isBinary: false } with no content
field, forcing the client to use ?raw=true for download. Prevents
buffering 10MB files into heap for JSON serialization. Frontend
shows a download fallback when content is absent.

F4: handleMarkdown now infers skill name from filename (same as
handleZip) when frontmatter has no name, instead of rejecting
with 400. Consistent behavior across import paths.

F1 (reviewer concern): upsertSkillFile is NOT affected — it uses
{ new: false } for insert-vs-replace detection but does a follow-up
findOne (lines 855-859) to return the post-upsert document.

* fix: deleteFile arg shape, raw URL base path, hoist SAFE_INLINE_MIMES

Codex P2: deleteFile expects { filepath } object, not a raw string.
Both upload handler cleanup and import handler cleanup now pass
{ filepath } to match the strategy contract (deleteLocalFile,
deleteFileFromS3 all expect a file object).

Codex P2: Raw download URL in SkillFileViewer now uses apiBaseUrl
prefix so subpath deployments (/chat, etc.) resolve correctly.

NIT: Hoist SAFE_INLINE_MIMES Set to factory scope — was re-allocated
per raw download request inside the if block.

* fix: Remove inert cache write for large text files, localize aria-label

N2: The { isBinary: false } cache write for text files > 1MB had no
effect — subsequent requests still fell through to stream read since
neither isBinary nor content provided a fast-path short-circuit.
Removed the pointless DB updateOne per request.

N4: Replace hardcoded "Back to skill" aria-label with localize().

* refactor: Extract shared parseFrontmatter, widen deleteFile type

N3: Extract parseFrontmatter into Skills/utils/frontmatter.ts —
single implementation shared by SkillDetail and SkillFileViewer.
Accepts optional skipKeys set so callers control which frontmatter
fields are excluded (SkillDetail skips name/description since
they're shown in the header; other .md files show all fields).

N5: Widen ImportSkillDeps.deleteFile file param from { filepath }
to { filepath; [key: string]: unknown } to signal extensibility
if strategies start accessing additional file properties.

* fix: Advance i past list items for skipped keys, DRY parseSkillMd

Finding A: parseFrontmatter now consumes multi-line YAML list items
before checking skipKeys — prevents list lines from leaking into
subsequent key parsing as spurious fields.

Finding B: parseSkillMd now delegates to the shared parseFrontmatter
instead of re-implementing the same frontmatter scanning loop.
Reduces client-side parseFrontmatter implementations from 3 to 1.

* fix: Call apiBaseUrl(), delete storage blob on file removal

- apiBaseUrl is a function, not a string — call it in the template
  literal so raw download URLs resolve correctly.
- deleteFileHandler now looks up the file record before deleting,
  then fire-and-forget deletes the storage blob via the strategy's
  deleteFile. Previously only the DB record was removed, leaving
  orphaned blobs in local/S3/Firebase/Azure storage.

* fix: Clean up storage blobs when deleting an entire skill

deleteHandler now lists all files for the skill before calling
deleteSkill, then fire-and-forget deletes each blob via the
storage strategy. Previously only per-file deletion cleaned up
blobs — deleting a whole skill left all associated files orphaned
in local/S3/Firebase/Azure storage.

* refactor: useImportSkillMutation hook, fix TSkill[] unsafe cast

- Create useImportSkillMutation in mutations.ts + ImportSkillOptions
  type. UploadSkillDialog now uses the mutation hook instead of
  calling dataService.importSkill directly with manual useState
  loading management. Eliminates unmounted-component state update
  risk and aligns with the React Query mutation pattern used by
  every other mutation in the codebase.

- SkillSelectDialog: replace as unknown as TSkill[] with proper
  TSkillSummary typing. SkillCard props updated to TSkillSummary.
  The dialog only uses summary-level fields (name, description,
  category, author) — the cast was hiding a type mismatch.

* fix: Use saved source for import cleanup, delete old blob on replace

Codex P2: Import cleanup now uses file.source (the backend the file
was actually saved to) instead of re-resolving from config. In mixed
strategy setups, the previous approach could target the wrong backend.

Codex P2: When re-uploading a file to an existing relativePath, the
old blob is now deleted after successful upsert. Previously only the
DB record was replaced, leaving the old storage object orphaned.

* fix: Register PUT /:roleName/skills route in roles router

* fix: Re-read skill after zip file processing for fresh metadata

The import response was built from the skill object created before
the file loop, but each upsertSkillFile bumps version and fileCount.
Clients caching the stale response would get 409 conflicts on first
edit and see incorrect file counts.

Now re-reads the skill via getSkillById after the loop so the
response reflects the current version, fileCount, and updatedAt.

* fix: Size-check SKILL.md before decompression, don't gate on fileCount

P1: SKILL.md was decompressed before any size accounting. A crafted
archive could expand SKILL.md past 10MB before validation ran. Now
checks declared size pre-decompression and actual size post, both
against MAX_SINGLE_FILE_BYTES.

P2: File list query was gated on cached fileCount which can be stale
after mutations. Now fetches files for the active skill regardless
of fileCount. hasFiles derived from fetched data with fileCount as
fallback, so newly uploaded files appear without hard refresh.

* fix: Move files declaration before hasFiles to avoid TDZ error

* security: Stream-decompress zip entries with enforced byte cap

Replace zipEntry.async('nodebuffer') (buffers entire entry before
checking limits) with zipEntry.nodeStream('nodebuffer') piped
through a byte counter that destroys the stream when the per-file
or cumulative limit is exceeded.

Previously, when JSZip's _data.uncompressedSize was absent (the
common case), a high-ratio entry could allocate hundreds of MB
before the post-decompression check caught it. Now decompression
is aborted mid-stream at the exact byte threshold — no entry can
exceed its limit regardless of compression ratio.

* refactor: Reorganize access check for prompts in useSideNavLinks hook

Moved the prompts access check to a new position in the code to improve readability and maintainability. This change ensures that the prompts link is added to the navigation only if the user has the appropriate access, without altering the existing functionality.

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-04-25 04:02:00 -04:00
Danny Avila
fda72ac621
🏗️ refactor: Remove Redundant Caching, Migrate Config Services to TypeScript (#12466)
* ♻️ refactor: Remove redundant scopedCacheKey caching, support user-provided key model fetching

Remove redundant cache layers that used `scopedCacheKey()` (tenant-only scoping)
on top of `getAppConfig()` which already caches per-principal (role+user+tenant).
This caused config overrides for different principals within the same tenant to
be invisible due to stale cached data.

Changes:
- Add `requireJwtAuth` to `/api/endpoints` route for proper user context
- Remove ENDPOINT_CONFIG, STARTUP_CONFIG, PLUGINS, TOOLS, and MODELS_CONFIG
  cache layers — all derive from `getAppConfig()` with cheap computation
- Enhance MODEL_QUERIES cache: hash(baseURL+apiKey) keys, 2-minute TTL,
  caching centralized in `fetchModels()` base function
- Support fetching models with user-provided API keys in `loadConfigModels`
  via `getUserKeyValues` lookup (no caching for user keys)
- Update all affected tests

Closes #1028

* ♻️ refactor: Migrate config services to TypeScript in packages/api

Move core config logic from CJS /api wrappers to typed TypeScript in
packages/api using dependency injection factories:

- `createEndpointsConfigService` — endpoint config merging + checkCapability
- `createLoadConfigModels` — custom endpoint model loading with user key support
- `createMCPToolCacheService` — MCP tool cache operations (update, merge, cache)

/api files become thin wrappers that wire dependencies (getAppConfig,
loadDefaultEndpointsConfig, getUserKeyValues, getCachedTools, etc.)
into the typed factories.

Also moves existing `endpoints/config.ts` → `endpoints/config/providers.ts`
to accommodate the new `config/` directory structure.

* 🔄 fix: Invalidate models query when user API key is set or revoked

Without this, users had to refresh the page after entering their API key
to see the updated model list fetched with their credentials.

- Invalidate QueryKeys.models in useUpdateUserKeysMutation onSuccess
- Invalidate QueryKeys.models in useRevokeUserKeyMutation onSuccess
- Invalidate QueryKeys.models in useRevokeAllUserKeysMutation onSuccess

* 🗺️ fix: Remap YAML-level override keys to AppConfig equivalents in mergeConfigOverrides

Config overrides stored in the DB use YAML-level keys (TCustomConfig),
but they're merged into the already-processed AppConfig where some fields
have been renamed by AppService. This caused mcpServers overrides to land
on a nonexistent key instead of mcpConfig, so config-override MCP servers
never appeared in the UI.

- Add OVERRIDE_KEY_MAP to remap mcpServers→mcpConfig, interface→interfaceConfig
- Apply remapping before deep merge in mergeConfigOverrides
- Add test for YAML-level key remapping behavior
- Update existing tests to use AppConfig field names in assertions

* 🧪 test: Update service.spec to use AppConfig field names after override key remapping

* 🛡️ fix: Address code review findings — reliability, types, tests, and performance

- Pass tenant context (getTenantId) in importers.js getEndpointsConfig call
- Add 5 tests for user-provided API key model fetching (key found, no key,
  DB error, missing userId, apiKey-only with fixed baseURL)
- Distinguish NO_USER_KEY (debug) from infrastructure errors (warn) in catch
- Switch fetchPromisesMap from Promise.all to Promise.allSettled so one
  failing provider doesn't kill the entire model config
- Parallelize getUserKeyValues DB lookups via batched Promise.allSettled
  instead of sequential awaits in the loop
- Hoist standardCache instance in fetchModels to avoid double instantiation
- Replace Record<string, unknown> types with Partial<TConfig>-based types;
  remove as unknown as T double-cast in endpoints config
- Narrow Bedrock availableRegions to typed destructure
- Narrow version field from string|number|undefined to string|undefined
- Fix import ordering in mcp/tools.ts and config/models.ts per AGENTS.md
- Add JSDoc to getModelsConfig alias clarifying caching semantics

* fix: Guard against null getCachedTools in mergeAppTools

* 🔍 fix: Address follow-up review — deduplicate extractEnvVariable, fix error discrimination, add log-level tests

- Deduplicate extractEnvVariable calls: resolve apiKey/baseURL once, reuse
  for both the entry and isUserProvided checks (Finding A)
- Move ResolvedEndpoint interface from function closure to module scope (Finding B)
- Replace fragile msg.includes('NO_USER_KEY') with ErrorTypes.NO_USER_KEY
  enum check against actual error message format (Finding C). Also handle
  ErrorTypes.INVALID_USER_KEY as an expected "no key" case.
- Add test asserting logger.warn is called for infra errors (not debug)
- Add test asserting logger.debug is called for NO_USER_KEY errors (not warn)

* fix: Preserve numeric assistants version via String() coercion

* 🐛 fix: Address secondary review — Ollama cache bypass, cache tests, type safety

- Fix Ollama success path bypassing cache write in fetchModels (CRITICAL):
  store result before returning so Ollama models benefit from 2-minute TTL
- Add 4 fetchModels cache behavior tests: cache write with TTL, cache hit
  short-circuits HTTP, skipCache bypasses read+write, empty results not cached
- Type-safe OVERRIDE_KEY_MAP: Partial<Record<keyof TCustomConfig, keyof AppConfig>>
  so compiler catches future field rename mismatches
- Fix import ordering in config/models.ts (package types longest→shortest)
- Rename ToolCacheDeps → MCPToolCacheDeps for naming consistency
- Expand getModelsConfig JSDoc to explain caching granularity

* fix: Narrow OVERRIDE_KEY_MAP index to satisfy strict tsconfig

* 🧩 fix: Add allowedProviders to TConfig, remove Record<string, unknown> from PartialEndpointEntry

The agents endpoint config includes allowedProviders (used by the frontend
AgentPanel to filter available providers), but it was missing from TConfig.
This forced PartialEndpointEntry to use & Record<string, unknown> as an
escape hatch, violating AGENTS.md type policy.

- Add allowedProviders?: (string | EModelEndpoint)[] to TConfig
- Remove Record<string, unknown> from PartialEndpointEntry — now just Partial<TConfig>

* 🛡️ fix: Isolate Ollama cache write from fetch try-catch, add Ollama cache tests

- Separate Ollama fetch and cache write into distinct scopes so a cache
  failure (e.g., Redis down) doesn't misattribute the error as an Ollama
  API failure and fall through to the OpenAI-compatible path (Issue A)
- Add 2 Ollama-specific cache tests: models written with TTL on fetch,
  cached models returned without hitting server (Issue B)
- Replace hardcoded 120000 with Time.TWO_MINUTES constant in cache TTL
  test assertion (Issue C)
- Fix OVERRIDE_KEY_MAP JSDoc to accurately describe runtime vs compile-time
  type enforcement (Issue D)
- Add global beforeEach for cache mock reset to prevent cross-test leakage

* 🧪 fix: Address third review — DI consistency, cache key width, MCP tests

- Inject loadCustomEndpointsConfig via EndpointsConfigDeps with default
  fallback, matching loadDefaultEndpointsConfig DI pattern (Finding 3)
- Widen modelsCacheKey from 64-bit (.slice(0,16)) to 128-bit (.slice(0,32))
  for collision-sensitive cross-credential cache key (Finding 4)
- Add fetchModels.mockReset() in loadConfigModels.spec beforeEach to
  prevent mock implementation leaks across tests (Finding 5)
- Add 11 unit tests for createMCPToolCacheService covering all three
  functions: null/empty input, successful ops, error propagation,
  cold-cache merge (Finding 2)
- Simplify getModelsConfig JSDoc to @see reference (Finding 10)

* ♻️ refactor: Address remaining follow-ups from reviews

OVERRIDE_KEY_MAP completeness:
- Add missing turnstile→turnstileConfig mapping
- Add exhaustiveness test verifying all three renamed keys are remapped
  and original YAML keys don't leak through

Import role context:
- Pass userRole through importConversations job → importLibreChatConvo
  so role-based endpoint overrides are honored during conversation import
- Update convos.js route to include req.user.role in the job payload

createEndpointsConfigService unit tests:
- Add 8 tests covering: default+custom merge, Azure/AzureAssistants/
  Anthropic Vertex/Bedrock config enrichment, assistants version
  coercion, agents allowedProviders, req.config bypass

Plugins/tools efficiency:
- Use Set for includedTools/filteredTools lookups (O(1) vs O(n) per plugin)
- Combine auth check + filter into single pass (eliminates intermediate array)
- Pre-compute toolDefKeys Set for O(1) tool definition lookups

* fix: Scope model query cache by user when userIdQuery is enabled

* fix: Skip model cache for userIdQuery endpoints, fix endpoints test types

- When userIdQuery is true, skip caching entirely (like user_provided keys)
  to avoid cross-user model list leakage without duplicating cache data
- Fix AgentCapabilities type error in endpoints.spec.ts — use enum values
  and appConfig() helper for partial mock typing

* 🐛 fix: Restore filteredTools+includedTools composition, add checkCapability tests

- Fix filteredTools regression: whitelist and blacklist are now applied
  independently (two flat guards), matching original behavior where
  includedTools=['a','b'] + filteredTools=['b'] produces ['a'] (Finding A)
- Fix Set spread in toolkit loop: pre-compute toolDefKeysList array once
  alongside the Set, reuse for .some() without per-plugin allocation (Finding B)
- Add 2 filteredTools tests: blacklist-only path and combined
  whitelist+blacklist composition (Finding C)
- Add 3 checkCapability tests: capability present, capability absent,
  fallback to defaultAgentCapabilities for non-agents endpoints (Finding D)

* 🔑 fix: Include config-override MCP servers in filterAuthorizedTools

Config-override MCP servers (defined via admin config overrides for
roles/groups) were rejected by filterAuthorizedTools because it called
getAllServerConfigs(userId) without the configServers parameter. Only
YAML and DB-backed user servers were included in the access check.

- Add configServers parameter to filterAuthorizedTools
- Resolve config servers via resolveConfigServers(req) at all 4 callsites
  (create, update, duplicate, revert) using parallel Promise.all
- Pass configServers through to getAllServerConfigs(userId, configServers)
  so the registry merges config-source servers into the access check
- Update filterAuthorizedTools.spec.js mock for resolveConfigServers

* fix: Skip model cache for userIdQuery endpoints, fix endpoints test types

For user-provided key endpoints (userProvide: true), skip the full model
list re-fetch during message validation — the user already selected from
a list we served them, and re-fetching with skipCache:true on every
message send is both slow and fragile (5s provider timeout = rejected model).

Instead, validate the model string format only:
- Must be a string, max 256 chars
- Must match [a-zA-Z0-9][a-zA-Z0-9_.:\-/@+ ]* (covers all known provider
  model ID formats while rejecting injection attempts)

System-configured endpoints still get full model list validation as before.

* 🧪 test: Add regression tests for filterAuthorizedTools configServers and validateModel

filterAuthorizedTools:
- Add test verifying configServers is passed to getAllServerConfigs and
  config-override server tools are allowed through
- Guard resolveConfigServers in createAgentHandler to only run when
  MCP tools are present (skip for tool-free agent creates)

validateModel (12 new tests):
- Format validation: missing model, non-string, length overflow, leading
  special char, script injection, standard model ID acceptance
- userProvide early-return: next() called immediately, getModelsConfig
  not invoked (regression guard for the exact bug this fixes)
- System endpoint list validation: reject unknown model, accept known
  model, handle null/missing models config

Also fix unnecessary backslash escape in MODEL_PATTERN regex.

* 🧹 fix: Remove space from MODEL_PATTERN, trim input, clean up nits

- Remove space character from MODEL_PATTERN regex — no real model ID
  uses spaces; prevents spurious violation logs from whitespace artifacts
- Add model.trim() before validation to handle accidental whitespace
- Remove redundant filterUniquePlugins call on already-deduplicated output
- Add comment documenting intentional whitelist+blacklist composition
- Add getUserKeyValues.mockReset() in loadConfigModels.spec beforeEach
- Remove narrating JSDoc from getModelsConfig one-liner
- Add 2 tests: trim whitespace handling, reject spaces in model ID

* fix: Match startup tool loader semantics — includedTools takes precedence over filteredTools

The startup tool loader (loadAndFormatTools) explicitly ignores
filteredTools when includedTools is set, with a warning log. The
PluginController was applying both independently, creating inconsistent
behavior where the same config produced different results at startup
vs plugin listing time.

Restored mutually exclusive semantics: when includedTools is non-empty,
filteredTools is not evaluated.

* 🧹 chore: Simplify validateModel flow, note auth requirement on endpoints route

- Separate missing-model from invalid-model checks cleanly: type+presence
  guard first, then trim+format guard (reviewer NIT)
- Add route comment noting auth is required for role/tenant scoping

* fix: Write trimmed model back to req.body.model for downstream consumers
2026-03-30 16:49:48 -04:00
Danny Avila
877c2efc85
🏗️ feat: bulkWrite isolation, pre-auth context, strict-mode fixes (#12445)
* fix: wrap seedDatabase() in runAsSystem() for strict tenant mode

seedDatabase() was called without tenant context at startup, causing
every Mongoose operation inside it to throw when
TENANT_ISOLATION_STRICT=true. Wrapping in runAsSystem() gives it the
SYSTEM_TENANT_ID sentinel so the isolation plugin skips filtering,
matching the pattern already used for performStartupChecks and
updateInterfacePermissions.

* fix: chain tenantContextMiddleware in optionalJwtAuth

optionalJwtAuth populated req.user but never established ALS tenant
context, unlike requireJwtAuth which chains tenantContextMiddleware
after successful auth. Authenticated users hitting routes with
optionalJwtAuth (e.g. /api/banner) had no tenant isolation.

* feat: tenant-safe bulkWrite wrapper and call-site migration

Mongoose's bulkWrite() does not trigger schema-level middleware hooks,
so the applyTenantIsolation plugin cannot intercept it. This adds a
tenantSafeBulkWrite() utility that injects the current ALS tenant
context into every operation's filter/document before delegating to
native bulkWrite.

Migrates all 8 runtime bulkWrite call sites:
- agentCategory (seedCategories, ensureDefaultCategories)
- conversation (bulkSaveConvos)
- message (bulkSaveMessages)
- file (batchUpdateFiles)
- conversationTag (updateTagsForConversation, bulkIncrementTagCounts)
- aclEntry (bulkWriteAclEntries)

systemGrant.seedSystemGrants is intentionally not migrated — it uses
explicit tenantId: { $exists: false } filters and is exempt from the
isolation plugin.

* feat: pre-auth tenant middleware and tenant-scoped config cache

Adds preAuthTenantMiddleware that reads X-Tenant-Id from the request
header and wraps downstream in tenantStorage ALS context. Wired onto
/oauth, /api/auth, /api/config, and /api/share — unauthenticated
routes that need tenant scoping before JWT auth runs.

The /api/config cache key is now tenant-scoped
(STARTUP_CONFIG:${tenantId}) so multi-tenant deployments serve the
correct login page config per tenant.

The middleware is intentionally minimal — no subdomain parsing, no
OIDC claim extraction. The private fork's reverse proxy or auth
gateway sets the header.

* feat: accept optional tenantId in updateInterfacePermissions

When tenantId is provided, the function re-enters inside
tenantStorage.run({ tenantId }) so all downstream Mongoose queries
target that tenant's roles instead of the system context. This lets
the private fork's tenant provisioning flow call
updateInterfacePermissions per-tenant after creating tenant-scoped
ADMIN/USER roles.

* fix: tenant-filter $lookup in getPromptGroup aggregation

The $lookup stage in getPromptGroup() queried the prompts collection
without tenant filtering. While the outer PromptGroup aggregate is
protected by the tenantIsolation plugin's pre('aggregate') hook,
$lookup runs as an internal MongoDB operation that bypasses Mongoose
hooks entirely.

Converts from simple field-based $lookup to pipeline-based $lookup
with an explicit tenantId match when tenant context is active.

* fix: replace field-level unique indexes with tenant-scoped compounds

Field-level unique:true creates a globally-unique single-field index in
MongoDB, which would cause insert failures across tenants sharing the
same ID values.

- agent.id: removed field-level unique, added { id, tenantId } compound
- convo.conversationId: removed field-level unique (compound at line 50
  already exists: { conversationId, user, tenantId })
- message.messageId: removed field-level unique (compound at line 165
  already exists: { messageId, user, tenantId })
- preset.presetId: removed field-level unique, added { presetId, tenantId }
  compound

* fix: scope MODELS_CONFIG, ENDPOINT_CONFIG, PLUGINS, TOOLS caches by tenant

These caches store per-tenant configuration (available models, endpoint
settings, plugin availability, tool definitions) but were using global
cache keys. In multi-tenant mode, one tenant's cached config would be
served to all tenants.

Appends :${tenantId} to cache keys when tenant context is active.
Falls back to the unscoped key when no tenant context exists (backward
compatible for single-tenant OSS deployments).

Covers all read, write, and delete sites:
- ModelController.js: get/set MODELS_CONFIG
- PluginController.js: get/set PLUGINS, get/set TOOLS
- getEndpointsConfig.js: get/set/delete ENDPOINT_CONFIG
- app.js: delete ENDPOINT_CONFIG (clearEndpointConfigCache)
- mcp.js: delete TOOLS (updateMCPTools, mergeAppTools)
- importers.js: get ENDPOINT_CONFIG

* fix: add getTenantId to PluginController spec mock

The data-schemas mock was missing getTenantId, causing all
PluginController tests to throw when the controller calls
getTenantId() for tenant-scoped cache keys.

* fix: address review findings — migration, strict-mode, DRY, types

Addresses all CRITICAL, MAJOR, and MINOR review findings:

F1 (CRITICAL): Add agents, conversations, messages, presets to
SUPERSEDED_INDEXES in tenantIndexes.ts so dropSupersededTenantIndexes()
drops the old single-field unique indexes that block multi-tenant inserts.

F2 (CRITICAL): Unknown bulkWrite op types now throw in strict mode
instead of silently passing through without tenant injection.

F3 (MAJOR): Replace wildcard export with named export for
tenantSafeBulkWrite, hiding _resetBulkWriteStrictCache from the
public package API.

F5 (MAJOR): Restore AnyBulkWriteOperation<IAclEntry>[] typing on
bulkWriteAclEntries — the unparameterized wrapper accepts parameterized
ops as a subtype.

F7 (MAJOR): Fix config.js tenant precedence — JWT-derived
req.user.tenantId now takes priority over the X-Tenant-Id header for
authenticated requests.

F8 (MINOR): Extract scopedCacheKey() helper into tenantContext.ts and
replace all 11 inline occurrences across 7 files.

F9 (MINOR): Use simple localField/foreignField $lookup for the
non-tenant getPromptGroup path (more efficient index seeks).

F12 (NIT): Remove redundant BulkOp type alias.
F13 (NIT): Remove debug log that leaked raw tenantId.

* fix: add new superseded indexes to tenantIndexes test fixture

The test creates old indexes to verify the migration drops them.
Missing fixture entries for agents.id_1, conversations.conversationId_1,
messages.messageId_1, and presets.presetId_1 caused the count assertion
to fail (expected 22, got 18).

* fix: restore logger.warn for unknown bulk op types in non-strict mode

* fix: block SYSTEM_TENANT_ID sentinel from external header input

CRITICAL: preAuthTenantMiddleware accepted any string as X-Tenant-Id,
including '__SYSTEM__'. The tenantIsolation plugin treats SYSTEM_TENANT_ID
as an explicit bypass — skipping ALL query filters. A client sending
X-Tenant-Id: __SYSTEM__ to pre-auth routes (/api/share, /api/config,
/api/auth, /oauth) would execute Mongoose operations without tenant
isolation.

Fixes:
- preAuthTenantMiddleware rejects SYSTEM_TENANT_ID in header
- scopedCacheKey returns the base key (not key:__SYSTEM__) in system
  context, preventing stale cache entries during runAsSystem()
- updateInterfacePermissions guards tenantId against SYSTEM_TENANT_ID
- $lookup pipeline separates $expr join from constant tenantId match
  for better index utilization
- Regression test for sentinel rejection in preAuthTenant.spec.ts
- Remove redundant getTenantId() call in config.js

* test: add missing deleteMany/replaceOne coverage, fix vacuous ALS assertions

bulkWrite spec:
- deleteMany: verifies tenant-scoped deletion leaves other tenants untouched
- replaceOne: verifies tenantId injected into both filter and replacement
- replaceOne overwrite: verifies a conflicting tenantId in the replacement
  document is overwritten by the ALS tenant (defense-in-depth)
- empty ops array: verifies graceful handling

preAuthTenant spec:
- All negative-case tests now use the capturedNext pattern to verify
  getTenantId() inside the middleware's execution context, not the
  test runner's outer frame (which was always undefined regardless)

* feat: tenant-isolate MESSAGES cache, FLOWS cache, and GenerationJobManager

MESSAGES cache (streamAudio.js):
- Cache key now uses scopedCacheKey(messageId) to prefix with tenantId,
  preventing cross-tenant message content reads during TTS streaming.

FLOWS cache (FlowStateManager):
- getFlowKey() now generates ${type}:${tenantId}:${flowId} when tenant
  context is active, isolating OAuth flow state per tenant.

GenerationJobManager:
- tenantId added to SerializableJobData and GenerationJobMetadata
- createJob() captures the current ALS tenant context (excluding
  SYSTEM_TENANT_ID) and stores it in job metadata
- SSE subscription endpoint validates job.metadata.tenantId matches
  req.user.tenantId, blocking cross-tenant stream access
- Both InMemoryJobStore and RedisJobStore updated to accept tenantId

* fix: add getTenantId and SYSTEM_TENANT_ID to MCP OAuth test mocks

FlowStateManager.getFlowKey() now calls getTenantId() for tenant-scoped
flow keys. The 4 MCP OAuth test files mock @librechat/data-schemas
without these exports, causing TypeError at runtime.

* fix: correct import ordering per AGENTS.md conventions

Package imports sorted shortest to longest line length, local imports
sorted longest to shortest — fixes ordering violations introduced by
our new imports across 8 files.

* fix: deserialize tenantId in RedisJobStore — cross-tenant SSE guard was no-op in Redis mode

serializeJob() writes tenantId to the Redis hash via Object.entries,
but deserializeJob() manually enumerates fields and omitted tenantId.
Every getJob() from Redis returned tenantId: undefined, causing the
SSE route's cross-tenant guard to short-circuit (undefined && ... → false).

* test: SSE tenant guard, FlowStateManager key consistency, ALS scope docs

SSE stream tenant tests (streamTenant.spec.js):
- Cross-tenant user accessing another tenant's stream → 403
- Same-tenant user accessing own stream → allowed
- OSS mode (no tenantId on job) → tenant check skipped

FlowStateManager tenant tests (manager.tenant.spec.ts):
- completeFlow finds flow created under same tenant context
- completeFlow does NOT find flow under different tenant context
- Unscoped flows are separate from tenant-scoped flows

Documentation:
- JSDoc on getFlowKey documenting ALS context consistency requirement
- Comment on streamAudio.js scopedCacheKey capture site

* fix: SSE stream tests hang on success path, remove internal fork references

The success-path tests entered the SSE streaming code which never
closes, causing timeout. Mock subscribe() to end the response
immediately. Restructured assertions to verify non-403/non-404.

Removed "private fork" and "OSS" references from code and test
descriptions — replaced with "deployment layer", "multi-tenant
deployments", and "single-tenant mode".

* fix: address review findings — test rigor, tenant ID validation, docs

F1: SSE stream tests now mock subscribe() with correct signature
(streamId, writeEvent, onDone, onError) and assert 200 status,
verifying the tenant guard actually allows through same-tenant users.

F2: completeFlow logs the attempted key and ALS tenantId when flow
is not found, so reverse proxy misconfiguration (missing X-Tenant-Id
on OAuth callback) produces an actionable warning.

F3/F10: preAuthTenantMiddleware validates tenant ID format — rejects
colons, special characters, and values exceeding 128 chars. Trims
whitespace. Prevents cache key collisions via crafted headers.

F4: Documented cache invalidation scope limitation in
clearEndpointConfigCache — only the calling tenant's key is cleared;
other tenants expire via TTL.

F7: getFlowKey JSDoc now lists all 8 methods requiring consistent
ALS context.

F8: Added dedicated scopedCacheKey unit tests — base key without
context, base key in system context, scoped key with tenant, no
ALS leakage across scope boundaries.

* fix: revert flow key tenant scoping, fix SSE test timing

FlowStateManager: Reverts tenant-scoped flow keys. OAuth callbacks
arrive without tenant ALS context (provider redirects don't carry
X-Tenant-Id), so completeFlow/failFlow would never find flows
created under tenant context. Flow IDs are random UUIDs with no
collision risk, and flow data is ephemeral (TTL-bounded).

SSE tests: Use process.nextTick for onDone callback so Express
response headers are flushed before res.write/res.end are called.

* fix: restore getTenantId import for completeFlow diagnostic log

* fix: correct completeFlow warning message, add missing flow test

The warning referenced X-Tenant-Id header consistency which was only
relevant when flow keys were tenant-scoped (since reverted). Updated
to list actual causes: TTL expiry, missing flow, or routing to a
different instance without shared Keyv storage.

Removed the getTenantId() call and import — no longer needed since
flow keys are unscoped.

Added test for the !flowState branch in completeFlow — verifies
return false and logger.warn on nonexistent flow ID.

* fix: add explicit return type to recursive updateInterfacePermissions

The recursive call (tenantId branch calls itself without tenantId)
causes TypeScript to infer circular return type 'any'. Adding
explicit Promise<void> satisfies the rollup typescript plugin.

* fix: update MCPOAuthRaceCondition test to match new completeFlow warning

* fix: clearEndpointConfigCache deletes both scoped and unscoped keys

Unauthenticated /api/endpoints requests populate the unscoped
ENDPOINT_CONFIG key. Admin config mutations clear only the
tenant-scoped key, leaving the unscoped entry stale indefinitely.
Now deletes both when in tenant context.

* fix: tenant guard on abort/status endpoints, warn logs, test coverage

F1: Add tenant guard to /chat/status/:conversationId and /chat/abort
matching the existing guard on /chat/stream/:streamId. The status
endpoint exposes aggregatedContent (AI response text) which requires
tenant-level access control.

F2: preAuthTenantMiddleware now logs warn for rejected __SYSTEM__
sentinel and malformed tenant IDs, providing observability for
bypass probing attempts.

F3: Abort fallback path (getActiveJobIdsForUser) now has tenant
check after resolving the job.

F4: Test for strict mode + SYSTEM_TENANT_ID — verifies runAsSystem
bypasses tenantSafeBulkWrite without throwing in strict mode.

F5: Test for job with tenantId + user without tenantId → 403.

F10: Regex uses idiomatic hyphen-at-start form.

F11: Test descriptions changed from "rejects" to "ignores" since
middleware calls next() (not 4xx).

Also fixes MCPOAuthRaceCondition test assertion to match updated
completeFlow warning message.

* fix: test coverage for logger.warn, status/abort guards, consistency

A: preAuthTenant spec now mocks logger and asserts warn calls for
__SYSTEM__ sentinel, malformed characters, and oversized headers.

B: streamTenant spec expanded with status and abort endpoint tests —
cross-tenant status returns 403, same-tenant returns 200 with body,
cross-tenant abort returns 403.

C: Abort endpoint uses req.user.tenantId (not req.user?.tenantId)
matching stream/status pattern — requireJwtAuth guarantees req.user.

D: Malformed header warning now includes ip in log metadata,
matching the sentinel warning for consistent SOC correlation.

* fix: assert ip field in malformed header warn tests

* fix: parallelize cache deletes, document tenant guard, fix import order

- clearEndpointConfigCache uses Promise.all for independent cache
  deletes instead of sequential awaits
- SSE stream tenant guard has inline comment explaining backward-compat
  behavior for untenanted legacy jobs
- conversation.ts local imports reordered longest-to-shortest per
  AGENTS.md

* fix: tenant-qualify userJobs keys, document tenant guard backward-compat

Job store userJobs keys now include tenantId when available:
- Redis: stream:user:{tenantId:userId}:jobs (falls back to
  stream:user:{userId}:jobs when no tenant)
- InMemory: composite key tenantId:userId in userJobMap

getActiveJobIdsByUser/getActiveJobIdsForUser accept optional tenantId
parameter, threaded through from req.user.tenantId at all call sites
(/chat/active and /chat/abort fallback).

Added inline comments on all three SSE tenant guards explaining the
backward-compat design: untenanted legacy jobs remain accessible
when the userId check passes.

* fix: parallelize cache deletes, document tenant guard, fix import order

Fix InMemoryJobStore.getActiveJobIdsByUser empty-set cleanup to use
the tenant-qualified userKey instead of bare userId — prevents
orphaned empty Sets accumulating in userJobMap for multi-tenant users.

Document cross-tenant staleness in clearEndpointConfigCache JSDoc —
other tenants' scoped keys expire via TTL, not active invalidation.

* fix: cleanup userJobMap leak, startup warning, DRY tenant guard, docs

F1: InMemoryJobStore.cleanup() now removes entries from userJobMap
before calling deleteJob, preventing orphaned empty Sets from
accumulating with tenant-qualified composite keys.

F2: Startup warning when TENANT_ISOLATION_STRICT is active — reminds
operators to configure reverse proxy to control X-Tenant-Id header.

F3: mergeAppTools JSDoc documents that tenant-scoped TOOLS keys are
not actively invalidated (matching clearEndpointConfigCache pattern).

F5: Abort handler getActiveJobIdsForUser call uses req.user.tenantId
(not req.user?.tenantId) — consistent with stream/status handlers.

F6: updateInterfacePermissions JSDoc clarifies SYSTEM_TENANT_ID
behavior — falls through to caller's ALS context.

F7: Extracted hasTenantMismatch() helper, replacing three identical
inline tenant guard blocks across stream/status/abort endpoints.

F9: scopedCacheKey JSDoc documents both passthrough cases (no context
and SYSTEM_TENANT_ID context).

* fix: clean userJobMap in evictOldest — same leak as cleanup()
2026-03-28 16:43:50 -04:00
Danny Avila
8ba2bde5c1
📦 refactor: Consolidate DB models, encapsulating Mongoose usage in data-schemas (#11830)
* chore: move database model methods to /packages/data-schemas

* chore: add TypeScript ESLint rule to warn on unused variables

* refactor: model imports to streamline access

- Consolidated model imports across various files to improve code organization and reduce redundancy.
- Updated imports for models such as Assistant, Message, Conversation, and others to a unified import path.
- Adjusted middleware and service files to reflect the new import structure, ensuring functionality remains intact.
- Enhanced test files to align with the new import paths, maintaining test coverage and integrity.

* chore: migrate database models to packages/data-schemas and refactor all direct Mongoose Model usage outside of data-schemas

* test: update agent model mocks in unit tests

- Added `getAgent` mock to `client.test.js` to enhance test coverage for agent-related functionality.
- Removed redundant `getAgent` and `getAgents` mocks from `openai.spec.js` and `responses.unit.spec.js` to streamline test setup and reduce duplication.
- Ensured consistency in agent mock implementations across test files.

* fix: update types in data-schemas

* refactor: enhance type definitions in transaction and spending methods

- Updated type definitions in `checkBalance.ts` to use specific request and response types.
- Refined `spendTokens.ts` to utilize a new `SpendTxData` interface for better clarity and type safety.
- Improved transaction handling in `transaction.ts` by introducing `TransactionResult` and `TxData` interfaces, ensuring consistent data structures across methods.
- Adjusted unit tests in `transaction.spec.ts` to accommodate new type definitions and enhance robustness.

* refactor: streamline model imports and enhance code organization

- Consolidated model imports across various controllers and services to a unified import path, improving code clarity and reducing redundancy.
- Updated multiple files to reflect the new import structure, ensuring all functionalities remain intact.
- Enhanced overall code organization by removing duplicate import statements and optimizing the usage of model methods.

* feat: implement loadAddedAgent and refactor agent loading logic

- Introduced `loadAddedAgent` function to handle loading agents from added conversations, supporting multi-convo parallel execution.
- Created a new `load.ts` file to encapsulate agent loading functionalities, including `loadEphemeralAgent` and `loadAgent`.
- Updated the `index.ts` file to export the new `load` module instead of the deprecated `loadAgent`.
- Enhanced type definitions and improved error handling in the agent loading process.
- Adjusted unit tests to reflect changes in the agent loading structure and ensure comprehensive coverage.

* refactor: enhance balance handling with new update interface

- Introduced `IBalanceUpdate` interface to streamline balance update operations across the codebase.
- Updated `upsertBalanceFields` method signatures in `balance.ts`, `transaction.ts`, and related tests to utilize the new interface for improved type safety.
- Adjusted type imports in `balance.spec.ts` to include `IBalanceUpdate`, ensuring consistency in balance management functionalities.
- Enhanced overall code clarity and maintainability by refining type definitions related to balance operations.

* feat: add unit tests for loadAgent functionality and enhance agent loading logic

- Introduced comprehensive unit tests for the `loadAgent` function, covering various scenarios including null and empty agent IDs, loading of ephemeral agents, and permission checks.
- Enhanced the `initializeClient` function by moving `getConvoFiles` to the correct position in the database method exports, ensuring proper functionality.
- Improved test coverage for agent loading, including handling of non-existent agents and user permissions.

* chore: reorder memory method exports for consistency

- Moved `deleteAllUserMemories` to the correct position in the exported memory methods, ensuring a consistent and logical order of method exports in `memory.ts`.
2026-03-21 14:28:53 -04:00
mfish911
4e5ae28fa9
📡 feat: Support Unauthenticated SMTP Relays (#12322)
* allow smtp server that does not have authentication

* fix: align checkEmailConfig with optional SMTP credentials and add tests

Remove EMAIL_USERNAME/EMAIL_PASSWORD requirements from the hasSMTPConfig
predicate in checkEmailConfig() so the rest of the codebase (login,
startup checks, invite-user) correctly recognizes unauthenticated SMTP
as a valid email configuration.

Add a warning when only one of the two credential env vars is set,
in both sendEmail.js and checkEmailConfig(), to catch partial
misconfigurations early.

Add test coverage for both the transporter auth assembly in sendEmail.js
and the checkEmailConfig predicate in packages/api.

Document in .env.example that credentials are optional for
unauthenticated SMTP relays.

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-03-20 13:07:39 -04:00
Brad Russell
ecd6d76bc8
🚦 fix: ERR_ERL_INVALID_IP_ADDRESS and IPv6 Key Collisions in IP Rate Limiters (#12319)
* fix: Add removePorts keyGenerator to all IP-based rate limiters

Six IP-based rate limiters are missing the `keyGenerator: removePorts`
option that is already used by the auth-related limiters (login,
register, resetPassword, verifyEmail). Without it, reverse proxies that
include ports in X-Forwarded-For headers cause
ERR_ERL_INVALID_IP_ADDRESS errors from express-rate-limit.

Fixes #12318

* fix: make removePorts IPv6-safe to prevent rate-limit key collisions

The original regex `/:\d+[^:]*$/` treated the last colon-delimited
segment of bare IPv6 addresses as a port, mangling valid IPs
(e.g. `::1` → `::`, `2001:db8::1` → `2001:db8::`). Distinct IPv6
clients could collapse into the same rate-limit bucket.

Use `net.isIP()` as a fast path for already-valid IPs, then match
bracketed IPv6+port and IPv4+port explicitly. Bare IPv6 addresses
are now returned unchanged.

Also fixes pre-existing property ordering inconsistency in
ttsLimiters.js userLimiterOptions (keyGenerator before store).

* refactor: move removePorts to packages/api as TypeScript, fix import order

- Move removePorts implementation to packages/api/src/utils/removePorts.ts
  with proper Express Request typing
- Reduce api/server/utils/removePorts.js to a thin re-export from
  @librechat/api for backward compatibility
- Consolidate removePorts import with limiterCache from @librechat/api
  in all 6 limiter files, fixing import order (package imports shortest
  to longest, local imports longest to shortest)
- Remove narrating inline comments per code style guidelines

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-03-19 21:48:03 -04:00
Danny Avila
f380390408
🛡️ fix: Prevent loop in ChatGPT import on Cyclic Parent Graphs (#12313)
Cap adjustTimestampsForOrdering to N passes and add cycle detection
to findValidParent, preventing DoS via crafted ChatGPT export files
with cyclic parentMessageId relationships.

Add breakParentCycles to sever cyclic back-edges before saving,
ensuring structurally valid message trees are persisted to the DB.
2026-03-19 17:15:12 -04:00
Danny Avila
35a35dc2e9
📏 refactor: Add File Size Limits to Conversation Imports (#12221)
* fix: add file size limits to conversation import multer instance

* fix: address review findings for conversation import file size limits

* fix: use local jest.mock for data-schemas instead of global moduleNameMapper

The global @librechat/data-schemas mock in jest.config.js only provided
logger, breaking all tests that depend on createModels from the same
package. Replace with a virtual jest.mock scoped to the import spec file.

* fix: move import to top of file, pre-compute upload middleware, assert logger.warn in tests

* refactor: move resolveImportMaxFileSize to packages/api

New backend logic belongs in packages/api as TypeScript. Delete the
api/server/utils/import/limits.js wrapper and import directly from
@librechat/api in convos.js and importConversations.js. Resolver unit
tests move to packages/api; the api/ spec retains only multer behavior
tests.

* chore: rename importLimits to import

* fix: stale type reference and mock isolation in import tests

Update typeof import path from '../importLimits' to '../import' after
the rename. Clear mockLogger.warn in beforeEach to prevent cross-test
accumulation.

* fix: add resolveImportMaxFileSize to @librechat/api mock in convos.spec.js

* fix: resolve jest.mock hoisting issue in import tests

jest.mock factories are hoisted above const declarations, so the
mockLogger reference was undefined at factory evaluation time. Use a
direct import of the mocked logger module instead.

* fix: remove virtual flag from data-schemas mock for CI compatibility

virtual: true prevents the mock from intercepting the real module in
CI where @librechat/data-schemas is built, causing import.ts to use
the real logger while the test asserts against the mock.
2026-03-14 03:06:29 -04:00
Danny Avila
ca79a03135
🚦 fix: Add Rate Limiting to Conversation Duplicate Endpoint (#12218)
* fix: add rate limiting to conversation duplicate endpoint

* chore: linter

* fix: address review findings for conversation duplicate rate limiting

* refactor: streamline test mocks for conversation routes

- Consolidated mock implementations into a dedicated `convos-route-mocks.js` file to enhance maintainability and readability of test files.
- Updated tests in `convos-duplicate-ratelimit.spec.js` and `convos.spec.js` to utilize the new mock structure, improving clarity and reducing redundancy.
- Enhanced the `duplicateConversation` function to accept an optional title parameter for better flexibility in conversation duplication.

* chore: rename files
2026-03-13 23:40:44 -04:00
Danny Avila
e1e204d6cf
🧮 refactor: Bulk Transactions & Balance Updates for Token Spending (#11996)
* refactor: transaction handling by integrating pricing and bulk write operations

- Updated `recordCollectedUsage` to accept pricing functions and bulk write operations, improving transaction management.
- Refactored `AgentClient` and related controllers to utilize the new transaction handling capabilities, ensuring better performance and accuracy in token spending.
- Added tests to validate the new functionality, ensuring correct behavior for both standard and bulk transaction paths.
- Introduced a new `transactions.ts` file to encapsulate transaction-related logic and types, enhancing code organization and maintainability.

* chore: reorganize imports in agents client controller

- Moved `getMultiplier` and `getCacheMultiplier` imports to maintain consistency and clarity in the import structure.
- Removed duplicate import of `updateBalance` and `bulkInsertTransactions`, streamlining the code for better readability.

* refactor: add TransactionData type and CANCEL_RATE constant to data-schemas

Establishes a single source of truth for the transaction document shape
and the incomplete-context billing rate constant, both consumed by
packages/api and api/.

* refactor: use proper types in data-schemas transaction methods

- Replace `as unknown as { tokenCredits }` with `lean<IBalance>()`
- Use `TransactionData[]` instead of `Record<string, unknown>[]`
  for bulkInsertTransactions parameter
- Add JSDoc noting insertMany bypasses document middleware
- Remove orphan section comment in methods/index.ts

* refactor: use shared types in transactions.ts, fix bulk write logic

- Import CANCEL_RATE from data-schemas instead of local duplicate
- Import TransactionData from data-schemas for PreparedEntry/BulkWriteDeps
- Use tilde alias for EndpointTokenConfig import
- Pass valueKey through to getMultiplier
- Only sum tokenValue for balance-enabled docs in bulkWriteTransactions
- Consolidate two loops into single-pass map

* refactor: remove duplicate updateBalance from Transaction.js

Import updateBalance from ~/models (sourced from data-schemas) instead
of maintaining a second copy. Also import CANCEL_RATE from data-schemas
and remove the Balance model import (no longer needed directly).

* fix: test real spendCollectedUsage instead of IIFE replica

Export spendCollectedUsage from abortMiddleware.js and rewrite the test
file to import and test the actual function. Previously the tests ran
against a hand-written replica that could silently diverge from the real
implementation.

* test: add transactions.spec.ts and restore regression comments

Add 22 direct unit tests for transactions.ts financial logic covering
prepareTokenSpend, prepareStructuredTokenSpend, bulkWriteTransactions,
CANCEL_RATE paths, NaN guards, disabled transactions, zero tokens,
cache multipliers, and balance-enabled filtering.

Restore critical regression documentation comments in
recordCollectedUsage.spec.js explaining which production bugs the
tests guard against.

* fix: widen setValues type to include lastRefill

The UpdateBalanceParams.setValues type was Partial<Pick<IBalance,
'tokenCredits'>> which excluded lastRefill — used by
createAutoRefillTransaction. Widen to also pick 'lastRefill'.

* test: use real MongoDB for bulkWriteTransactions tests

Replace mock-based bulkWriteTransactions tests with real DB tests using
MongoMemoryServer. Pure function tests (prepareTokenSpend,
prepareStructuredTokenSpend) remain mock-based since they don't touch
DB. Add end-to-end integration tests that verify the full prepare →
bulk write → DB state pipeline with real Transaction and Balance models.

* chore: update @librechat/agents dependency to version 3.1.54 in package-lock.json and related package.json files

* test: add bulk path parity tests proving identical DB outcomes

Three test suites proving the bulk path (prepareTokenSpend/
prepareStructuredTokenSpend + bulkWriteTransactions) produces
numerically identical results to the legacy path for all scenarios:

- usage.bulk-parity.spec.ts: mirrors all legacy recordCollectedUsage
  tests; asserts same return values and verifies metadata fields on
  the insertMany docs match what spendTokens args would carry

- transactions.bulk-parity.spec.ts: real-DB tests using actual
  getMultiplier/getCacheMultiplier pricing functions; asserts exact
  tokenValue, rate, rawAmount and balance deductions for standard
  tokens, structured/cache tokens, CANCEL_RATE, premium pricing,
  multi-entry batches, and edge cases (NaN, zero, disabled)

- Transaction.spec.js: adds describe('Bulk path parity') that mirrors
  7 key legacy tests via recordCollectedUsage + bulk deps against
  real MongoDB, asserting same balance deductions and doc counts

* refactor: update llmConfig structure to use modelKwargs for reasoning effort

Refactor the llmConfig in getOpenAILLMConfig to store reasoning effort within modelKwargs instead of directly on llmConfig. This change ensures consistency in the configuration structure and improves clarity in the handling of reasoning properties in the tests.

* test: update performance checks in processAssistantMessage tests

Revise the performance assertions in the processAssistantMessage tests to ensure that each message processing time remains under 100ms, addressing potential ReDoS vulnerabilities. This change enhances the reliability of the tests by focusing on maximum processing time rather than relative ratios.

* test: fill parity test gaps — model fallback, abort context, structured edge cases

- usage.bulk-parity: add undefined model fallback test
- transactions.bulk-parity: add abort context test (txns inserted,
  balance unchanged when balance not passed), fix readTokens type cast
- Transaction.spec: add 3 missing mirrors — balance disabled with
  transactions enabled, structured transactions disabled, structured
  balance disabled

* fix: deduct balance before inserting transactions to prevent orphaned docs

Swap the order in bulkWriteTransactions: updateBalance runs before
insertMany. If updateBalance fails (after exhausting retries), no
transaction documents are written — avoiding the inconsistent state
where transactions exist in MongoDB with no corresponding balance
deduction.

* chore: import order

* test: update config.spec.ts for OpenRouter reasoning in modelKwargs

Same fix as llm.spec.ts — OpenRouter reasoning is now passed via
modelKwargs instead of llmConfig.reasoning directly.
2026-03-01 12:26:36 -05:00
Jakub Fidler
c7469ce884
✨ feat: Add Claude conversation importer with thinking support (#11124)
* ✨ feat: Add Claude conversation importer with thinking support

Add support for importing Claude conversation exports:
- Detect Claude format by checking for chat_messages property
- Extract text and thinking content from content array
- Format thinking blocks using LibreChat's { type: 'think' } format
- Preserve timestamps from original conversation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* ✨ feat: Improve Claude importer with tests and timestamp handling

- Remove hardcoded model (Claude exports don't include model info)
- Add timestamp ordering to ensure parents appear before children
- Add fallback to conv.created_at for null message timestamps
- Add comprehensive tests for Claude importer:
  - Basic import, thinking content, timestamp handling
  - Empty messages, text fallback, default conversation name

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 21:37:52 -05:00
Jakub Fidler
180d0f18fe
🐛 fix: ChatGPT import creating fragmented conversation tree (#11123)
- Use ChatGPT timestamps (create_time) for proper message ordering
- Fallback to conv.create_time for null timestamps
- Adjust timestamps so children are always after parents
- Guard against circular references in findThinkingContent
- Skip thoughts and reasoning_recap messages (merged into responses)
- Add comprehensive timestamp ordering tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 21:31:18 -05:00
Danny Avila
656e1abaea
🪦 refactor: Remove Legacy Code (#10533)
* 🗑️ chore: Remove unused Legacy Provider clients and related helpers

* Deleted OpenAIClient and GoogleClient files along with their associated tests.
* Removed references to these clients in the clients index file.
* Cleaned up typedefs by removing the OpenAISpecClient export.
* Updated chat controllers to use the OpenAI SDK directly instead of the removed client classes.

* chore/remove-openapi-specs

* 🗑️ chore: Remove unused mergeSort and misc utility functions

* Deleted mergeSort.js and misc.js files as they are no longer needed.
* Removed references to cleanUpPrimaryKeyValue in messages.js and adjusted related logic.
* Updated mongoMeili.ts to eliminate local implementations of removed functions.

* chore: remove legacy endpoints

* chore: remove all plugins endpoint related code

* chore: remove unused prompt handling code and clean up imports

* Deleted handleInputs.js and instructions.js files as they are no longer needed.
* Removed references to these files in the prompts index.js.
* Updated docker-compose.yml to simplify reverse proxy configuration.

* chore: remove unused LightningIcon import from Icons.tsx

* chore: clean up translation.json by removing deprecated and unused keys

* chore: update Jest configuration and remove unused mock file

    * Simplified the setupFiles array in jest.config.js by removing the fetchEventSource mock.
    * Deleted the fetchEventSource.js mock file as it is no longer needed.

* fix: simplify endpoint type check in Landing and ConversationStarters components

    * Updated the endpoint type check to use strict equality for better clarity and performance.
    * Ensured consistency in the handling of the azureOpenAI endpoint across both components.

* chore: remove unused dependencies from package.json and package-lock.json

* chore: remove legacy EditController, associated routes and imports

* chore: update banResponse logic to refine request handling for banned users

* chore: remove unused validateEndpoint middleware and its references

* chore: remove unused 'res' parameter from initializeClient in multiple endpoint files

* chore: remove unused 'isSmallScreen' prop from BookmarkNav and NewChat components; clean up imports in ArchivedChatsTable and useSetIndexOptions hooks; enhance localization in PromptVersions

* chore: remove unused import of Constants and TMessage from MobileNav; retain only necessary QueryKeys import

* chore: remove unused TResPlugin type and related references; clean up imports in types and schemas
2025-12-11 16:36:12 -05:00
Danny Avila
8bdc808074
⚡ refactor: Optimize & Standardize Tokenizer Usage (#10777)
* refactor: Token Limit Processing with Enhanced Efficiency

- Added a new test suite for `processTextWithTokenLimit`, ensuring comprehensive coverage of various scenarios including under, at, and exceeding token limits.
- Refactored the `processTextWithTokenLimit` function to utilize a ratio-based estimation method, significantly reducing the number of token counting function calls compared to the previous binary search approach.
- Improved handling of edge cases and variable token density, ensuring accurate truncation and performance across diverse text inputs.
- Included direct comparisons with the old implementation to validate correctness and efficiency improvements.

* refactor: Remove Tokenizer Route and Related References

- Deleted the tokenizer route from the server and removed its references from the routes index and server files, streamlining the API structure.
- This change simplifies the routing configuration by eliminating unused endpoints.

* refactor: Migrate countTokens Utility to API Module

- Removed the local countTokens utility and integrated it into the @librechat/api module for centralized access.
- Updated various files to reference the new countTokens import from the API module, ensuring consistent usage across the application.
- Cleaned up unused references and imports related to the previous countTokens implementation.

* refactor: Centralize escapeRegExp Utility in API Module

- Moved the escapeRegExp function from local utility files to the @librechat/api module for consistent usage across the application.
- Updated imports in various files to reference the new centralized escapeRegExp function, ensuring cleaner code and reducing redundancy.
- Removed duplicate implementations of escapeRegExp from multiple files, streamlining the codebase.

* refactor: Enhance Token Counting Flexibility in Text Processing

- Updated the `processTextWithTokenLimit` function to accept both synchronous and asynchronous token counting functions, improving its versatility.
- Introduced a new `TokenCountFn` type to define the token counting function signature.
- Added comprehensive tests to validate the behavior of `processTextWithTokenLimit` with both sync and async token counting functions, ensuring consistent results.
- Implemented a wrapper to track call counts for the `countTokens` function, optimizing performance and reducing unnecessary calls.
- Enhanced existing tests to compare the performance of the new implementation against the old one, demonstrating significant improvements in efficiency.

* chore: documentation for Truncation Safety Buffer in Token Processing

- Added a safety buffer multiplier to the character position estimates during text truncation to prevent overshooting token limits.
- Updated the `processTextWithTokenLimit` function to utilize the new `TRUNCATION_SAFETY_BUFFER` constant, enhancing the accuracy of token limit processing.
- Improved documentation to clarify the rationale behind the buffer and its impact on performance and efficiency in token counting.
2025-12-02 12:22:04 -05:00
MarcAmick
e9b678dd6a
⚖️ fix: Add Configurable File Size Cap for Conversation Imports (#10012)
* Check file size of conversation being imported against a configured max size to prevent bringing down the application by uploading a large file

chore: remove non-english localization as needs to be added via locize

* feat: Implement file size validation for conversation imports to prevent oversized uploads

---------

Co-authored-by: Marc Amick <MarcAmick@jhu.edu>
Co-authored-by: Danny Avila <danny@librechat.ai>
2025-10-07 14:47:21 -04:00
Danny Avila
dbe4dd96b4
🧹 chore: Cleanup Logger and Utility Imports (#9935)
* 🧹 chore: Update logger imports to use @librechat/data-schemas across multiple files and remove unused sleep function from queue.js (#9930)

* chore: Replace local isEnabled utility with @librechat/api import across multiple files, update test files

* chore: Replace local logger import with @librechat/data-schemas logger in countTokens.js and fork.js

* chore: Update logs volume path in docker-compose.yml to correct directory

* chore: import order of isEnabled in static.js
2025-10-01 23:30:47 -04:00
Danny Avila
2489670f54
📂 refactor: File Read Operations (#9747)
* fix: axios response logging for text parsing, remove console logging, remove jsdoc

* refactor: error logging in logAxiosError function to handle various error types with type guards

* refactor: enhance text parsing with improved error handling and async file reading

* refactor: replace synchronous file reading with asynchronous methods for improved performance and memory management

* ci: update tests
2025-09-20 10:17:24 -04:00
Danny Avila
fcaf55143d
🏷️ fix: Increment Tag Counters When Forking/Duplicating Conversations (#9737)
* fix: increment tag counters when forking/duplicating conversations

- Add bulkIncrementTagCounts to update existing tag counts in bulk
- Integrate tag count updates into importBatchBuilder.saveBatch() using Promise.all
- Update frontend mutations to directly update cache instead of invalidating queries
- Optimize bulkIncrementTagCounts to skip unnecessary database queries

Fixes issue where forked/duplicated conversations with bookmarks would not increment
tag counters, leading to negative counts when bookmarks were later removed.

* chore: reorder import statements for clarity in fork.spec.js
2025-09-19 22:02:09 -04:00
Danny Avila
519645c0b0
🔻 fix: Role and System Message Handling for ChatGPT Imports (#9524)
* fix: ChatGPT import logic breaks message graph when it encounters a system message

- Implemented `findNonSystemParent` to maintain parent-child relationships by skipping system messages.
- Added a test case to ensure system messages do not disrupt the conversation flow during import.

* fix: ChatGPT import, correct sender for user messages with GPT-4 model

* fix: Enhance model name extraction for assistant messages in import process

- Updated sender assignment logic to dynamically extract model names from model slugs, improving accuracy for various GPT models.
- Added comprehensive tests to validate the extraction and formatting of model names from different model slugs, ensuring robustness in the import functionality.
2025-09-09 13:51:26 -04:00
Danny Avila
c46e0d3ecc
🔒 fix: href Attribute in Email Microsoft Template 2025-09-08 14:39:00 -04:00