mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-11-16 13:00:55 +00:00
59 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
b1591d291b
|
fix: Guard Mongo Cleanup In Api Services Tests (#17990) | ||
|
|
b7fa9c1e71
|
☄️ feat: Add GPT Image 2.6 Flare Support (#18021)
* ☄️ feat: Add GPT Image 2.6 Flare Support
* style: Format Image Tool Regression Tests
* fix: Preserve Image Aliases and Baseline Quality Support
---------
Co-authored-by: Lia <lia@librechat.ai>
|
||
|
|
e104e820c3
|
☀️ feat: Add GPT Image 2.6 Sunburst Support (#18019)
* ☀️ feat: Add GPT Image 2.6 Sunburst Support
* fix: Preserve Image Aliases and Baseline Quality Support
---------
Co-authored-by: Lia <lia@librechat.ai>
|
||
|
|
c6c79e4ea7
|
🔒 fix: Return Approved Error Text From the Remaining Structured Tools (#17934) | ||
|
|
49ecc0695b
|
🧹 chore: Clear the Whole-Repo Lint Errors on Dev (#17702) | ||
|
|
8718338728
|
⏱️ fix: Bound Outbound Calls Made Inside a Request (#17263)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/chat (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
* fix: Bound Outbound Calls Made Inside a Request RAG queries for file search and message context, OCR provider calls, action HTTP calls, the moderation check, the stored-image fetch for base64 providers, and Flux result polling had no timeout. A stalled upstream hung the chat turn, the upload or the tool call indefinitely, and Flux kept polling forever on statuses that never become Ready. Each now runs under a budget from a new librechat.yaml timeouts block with finite defaults (getOutboundTimeoutMs in @librechat/api). Action calls also receive the run's abort signal, so stopping a run closes the socket. * test: Cover a Stalled RAG Query in the Mock Harness * fix: Share One Flux Budget Across Submit and Polls, Bound the Vertex Token Exchange * refactor: Poll Flux Results Through pollUntilSettled * refactor: Leave Flux Polling for a Separate Migration * fix: Cap Configured Timeouts at the Node Timer Limit * fix: Bound the Action Token Refresh and the Whole Image Download * fix: Cap Every Outbound Budget by Wall-Clock Time Axios's timeout only fires on an idle socket, so a RAG, OCR, moderation, action or token response that keeps trickling bytes never ended. outboundRequestBudget pairs each timeout with an AbortSignal deadline, and action calls combine it with the run's own signal. * fix: Name the Budget That Ran Out and Give Actions Five Minutes The actions default rises to five minutes, since action endpoints are arbitrary third-party APIs and the old minute could cut off legitimately slow ones after an upgrade. A budget overrun now fails with an error naming the timeouts key and its value, whether it was the idle timeout or the wall-clock deadline, and logAxiosError logs it as a warning so operators know what to raise. A deadline cancellation is no longer read as a user stop, so an action that runs out of time reports a timeout instead of looking like a stopped run. * fix: Hold Outbound Deadlines Weakly So Finished Requests Leave No Timers * fix: Check the Budget Before Reading an Error Message * fix: Report a Wrapped Deadline as a Failure, Not a Stop * fix: Bound the OAuth Code Exchange and Keep the OCR Budget Hint When Protected * chore: Sort Imports in axios.ts After Rebase |
||
|
|
8363766ccb
|
🌠 chore: Default OpenAI Image Tools to GPT Image Flare (#17014)
Co-authored-by: Lia <lia@librechat.ai> |
||
|
|
73d704656e
|
📔 feat: Preserve Owner Text During PII Redaction (#16311)
* 📔 feat: Preserve Owner Text During PII Redaction * 🧵 fix: Keep Ordinary Chat Views Outside Private Text Provider * 🧷 fix: Preserve Owner Text Across Copy and Edit Paths * 🔏 fix: Protect PII on Denial and Abort Paths * fix: preserve protected text through denial, stop, and owner reload * test: cover preliminary private metadata in disconnect fixture * test: carry preliminary metadata in resume controller fixture * test: preserve Stop prerequisite identity in tenant fixture * fix: keep protected owner text out of search and exports * fix: preserve trusted PII placeholders through provider inspection * fix: coordinate protected turns across Stop and native sharing * fix: Preserve Private Text Admission and Query Lifecycles * fix: Track Private Query Effect Dependencies * fix: Fence Protected Admission and Screenshot Targets * fix: Share Protected Admission and Preserve Native Token Trust * test: Wait for Private Message Concurrency Indexes * fix: Preserve Protected Admission Across Recovery and Titles * fix: Coordinate Protected Root Admission and Early Stop --------- Co-authored-by: Lia <lia@librechat.ai> |
||
|
|
ea7ac99209
|
🛶 fix: Keep Trigger Bans Off Shared Loopback IPs (#16535)
Co-authored-by: Lia <lia@librechat.ai> |
||
|
|
a329e01ac7
|
🧨 fix: Reject Invalid AES Credentials Before Startup (#16402)
* 🧨 fix: Reject Invalid AES Credentials Before Startup * 🎯 test: Format Credential Validation Cases * 🧪 test: Use Valid AES Credentials in API Fixtures --------- Co-authored-by: Lia <lia@librechat.ai> |
||
|
|
44703da3c6
|
🫖 fix: Skip Blank RAG Queries Before Embedding (#16231)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Co-authored-by: Lia <lia@librechat.ai> |
||
|
|
f80f0be90e
|
🔑 fix: Stringify Ban Cache Keys When Redis Is Off (#16039)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
* 🔑 fix: Stringify Ban Cache Keys When Redis Is Off Keyv/Mongo throws when checkBan passes a raw ObjectId as the cache key with Redis disabled, which 500s every social OAuth login. Fixes #16025 Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com> * 🔑 fix: Stringify Violation Log Keys and Cover Bans With Real Keyv logViolation keyed its violation and ban logs by the raw `req.user.id ?? req.user._id`, the same shape checkBan used, so a lean user document would hit the same keyv 5.6 `key.startsWith` TypeError on the namespaced violation stores and record its ban under an ObjectId. The new spec drives checkBan and logViolation through real namespaced Keyv instances (Mongo-backed ban store, in-memory violation logs) with an ObjectId-only OAuth user: an unbanned user passes, and a ban logged from one address is enforced for the same user from another. * 🎯 refactor: Stringify Only the ObjectId Fallback for Ban Keys The id is converted once, where a non-string can enter: the lean document's `_id` fallback. `req.user.id` is already a string on every path that sets it (the JWT strategies, Mongoose's `id` virtual, the 2FA limiter's temp-token id), and the email lookup already stringifies, so `getBanCacheKey` and that line go back to their dev form. --------- Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com> Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
|
fdaf4db938
|
🔢 fix: Normalize PDF Citation Page Numbers (#15731) | ||
|
|
a827f35a69
|
🛤️ fix: Align File Search Citation Paths (#15650)
* fix: keep file search anchors aligned with citation passages * refactor: centralize file citation selection * test: wire citation helpers into API mocks --------- Co-authored-by: aeyeopsdev <275853971+aeyeopsdev@users.noreply.github.com> Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
|
146664e0d1
|
🔐 fix: Redirect OAuth Browser Navigations Instead of Rendering JSON (#15440)
* 🔐 fix: Redirect OAuth Browser Navigations Instead of Rendering JSON The /oauth/* routes are reached by top-level browser navigation, so a middleware that rejects one with a JSON body makes the browser render that body as the page. Hitting the login rate limit during a social sign-in dropped the user on a blank page showing: {"message": "Too many login attempts, please try again after 5 minutes."} Mark those requests with markOAuthNavigation and have the two middlewares that can reject them answer with a redirect to the login page carrying an error code the client localizes. The login limiter stays a single instance so the attempt budget is still shared with POST /api/auth/login, and that route keeps its JSON 429 for the SPA. * 🧪 fix: Address PR Review Bot Findings chatgpt-codex-connector: - Split the inline `type TranslationKeys` specifier out of the value import, per the standalone `import type` convention in CLAUDE.md. This also clears the Static checks import-sort failure. - Cover the new OAuth error branches in Login.spec.tsx: each mapped `?error=` code raises its localized toast and the query is cleaned, and unmapped codes including prototype members raise nothing. |
||
|
|
d9e6250d05
|
🛑 fix: Separate Agent Event Backpressure From User Bans (#15200)
* 🛑 fix: Separate Agent Event Backpressure From User Bans
* fix: Address Agent Event Review Findings
* fix: Mirror Case-Insensitive Agent Control Routing
|
||
|
|
b11978017d
|
🧱 fix: Enforce Agent Runtime File Trust Boundaries (#14577)
* fix: secure agent runtime file metadata * chore: sort agent resource test imports * fix: Align Agent Tool Resource Types * fix: Rehydrate Agent Image Resources * fix: preserve remote agent file authorization |
||
|
|
84329ab0ff
|
fix: use logAxiosError at the RAG file_search/context call sites (#14014)
|
||
|
|
21d98b85bd
|
🏷️ fix: Scope File Search entity_id to Agent Knowledge-Base Files Only (#13693)
User-attached files are embedded by the RAG API under the user id (no entity), while only agent knowledge-base files are embedded under the agent's entity_id. Sending entity_id in every /query request made the RAG API's entity filter return no results for user attachments — with a shared agent, files attached to the message were effectively invisible to the file_search tool, while knowledge-base files kept working (which masked the bug). primeFiles now tags each file with fromAgent (whether it belongs to the agent's file_search.file_ids) and createQueryBody only includes entity_id when fromAgent === true — the safe default for callers that omit the flag is to query without entity scoping. Tests cover KB files, user attachments, the omitted-flag default, and restore RAG_API_URL. |
||
|
|
c04bddd304
|
🪵 refactor: Bound Log Traversal And Remove Legacy api/config Logger (#13813)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
* 🛡️ fix: Bound object-traverse against DAG fan-out and shared refs Detect cycles via the ancestor chain (so shared, non-circular references in sibling branches / DAGs are traversed correctly) and add defensive maxNodes (100k) / maxDepth (100) caps. The removed global visited set was implicitly bounding work at O(distinct nodes); ancestor-chain-only detection is O(root-to-node paths), exponential on DAGs (a depth-24 diamond went from 26 to 50M visits / 1.6s of synchronous work). The caps bound it to ~9ms while leaving normal traversal untouched. Adds a spec covering shared refs, cycles, DAGs, and both bounds. The lone consumer, debugTraverse, inherits the defaults with no change. * 🪵 refactor: Remove legacy api/config logger duplicate The api/config winston logger was a stale parallel implementation of the canonical @librechat/data-schemas logger, with unbounded redaction (regex-only redactFormat, npm traverse-based debugTraverse). Its winston instance and the logger export from api/config/index.js had zero consumers — every ~/config importer uses the MCP/flow-manager exports. The only live tie was ToolService's use of redactMessage. Re-export redactMessage from @librechat/data-schemas (behaviorally identical, a superset of the regex set), point ToolService at it, delete api/config/winston.js and api/config/parsers.js, drop the dead logger export, and remove the orphaned ~/config/parsers mock from the global test setup. * 🧹 chore: Drop orphaned traverse dep and stale legacy logger tests Deleting api/config/{winston,parsers}.js left the npm 'traverse' package unused in api/package.json (flagged by the detect-unused-packages CI check) and orphaned two tests that imported the deleted modules. Remove the traverse dependency (sync package-lock), and delete api/config/__tests__/{parsers,logToFile}.spec.js — the canonical logger's behavior is covered by packages/data-schemas/src/config/parsers.spec.ts. * 🩹 fix: Make object-traverse caps bound work and survive update() Address Codex review: (1) break the child loops as soon as the node budget is spent and iterate objects via for...in instead of materializing Object.entries/Object.keys, so maxNodes actually bounds work for wide arrays/objects; (2) detect ancestor cycles against an immutable original-node stack rather than context.node, which a callback's update() can reassign (the debug formatter rewrites array nodes in place). Adds tests for the wide-array bound and the update()-cycle case. * 🎚️ fix: Tighten object-traverse defaults to a ~1ms log budget Lower maxNodes 100000 -> 2500 and maxDepth 100 -> 5. Measured cost is ~140ns/node with the debug formatter callback, so 2500 nodes keeps a single log under ~1ms even on slower prod hardware; real log objects are ~25-30 nodes at depth 3-4, leaving ample headroom. maxNodes is the fan-out/cost lever; maxDepth bounds recursion and output readability (depth-5 covers typical logs, deeper renders compactly). |
||
|
|
98704f28c1
|
🌐 fix: Centralize Outbound Proxy Handling (#13726)
* fix: centralize outbound proxy handling * chore: sort proxy imports * test: update proxy helper mocks * fix: honor proxy bypasses consistently * fix: support http axios proxy targets |
||
|
|
5b11a5a076
|
🪵 chore: Restore Winston Format Factory Shape In Test Mocks (#13139)
Four jest mocks for `winston` in the test suite return the wrong shape:
api/test/__mocks__/logger.js (returns inner fn directly)
packages/api/src/agents/__tests__/memory.test.ts (`format` is a plain object)
packages/api/src/agents/__tests__/run-summarization.test.ts (same)
packages/api/src/agents/__tests__/initialize.test.ts (same)
Real `winston.format(fn)` returns a Format constructor whose instances
expose a `.transform(info, opts)` method that winston's pipeline calls
with the log info object. The current mocks collapse this:
- `(fn) => fn` returns the inner transform fn directly. When module-load
code in `@librechat/data-schemas/dist/config/parsers.cjs:52` does
`const redactFormat = winston.format((info) => ...)`, `redactFormat`
becomes the inner fn. The next line in `winston.cjs` calls
`parsers.redactFormat()` which invokes the inner fn with no `info`,
throwing `TypeError: Cannot read properties of undefined (reading 'level')`.
- `format: { combine, colorize, simple }` makes `winston.format` not
callable at all — `winston.format((info) => ...)` throws
`TypeError: winston.format is not a function`.
These currently pass in CI on GitHub Actions Ubuntu / Node 20.19, but
fail reproducibly on Node 24.x and on some Linux distros (verified on
WSL Ubuntu with Node 24.9.0). The CI passes appears to be environmental
luck around jest's mock-hoisting interaction with the workspace symlink
chain — the mocks are genuinely wrong against the data-schemas contract.
The fix: return a thunk that yields `{ transform: fn }` — matches real
winston's shape just enough that module-load completes; the inner fn is
only ever invoked by winston's pipeline (never at load time). Also adds
the full `winston.format.*` method surface (printf, timestamp, errors,
splat, json) plus `addColors` and the `DailyRotateFile`/`File` transports
that data-schemas's dist code references at module-load.
Verification (Node 24.9.0):
npm run build:data-provider && npm run build:data-schemas && npm run build:api
cd packages/api && npx jest src/agents/__tests__/{memory,run-summarization,initialize}.test.ts
→ 3 suites, 106 tests, all pass
No production code or behavior changes — test-only patch.
Co-authored-by: Jorge Costa <8352477+JorgeCosta87@users.noreply.github.com>
|
||
|
|
24e29aa8cb
|
🌱 fix: Inject Code-Tool Files Into Graph Sessions on First Call (+ read_file Sandbox Fallback) (#12831)
* 🌱 fix: Seed Code Tool Files Into Graph Sessions on First Call
Files attached to an agent's `tool_resources.execute_code` (user uploads
or generated artifacts from a prior turn) were silently dropped on the
first `execute_code` invocation of a turn. The agents-side `ToolNode`
populates `_injected_files` only when its `sessions` map already has an
`EXECUTE_CODE` entry — but that entry is only written by a previous
successful execution, so call #1 had nothing to inject. CodeExecutor
then fell back to a `/files/{session_id}` fetch, but `session_id` was
also empty on call #1, leaving the sandbox without the primed files.
Mirror the existing skill-priming pattern (`primeInvokedSkills` →
`initialSessions`) for code-resource files: eagerly call `primeFiles`
before `createRun` and merge the result into `initialSessions` via a
new `seedCodeFilesIntoSessions` helper. Skill files and code-resource
files now share the same `EXECUTE_CODE` entry; the prior representative
`session_id` is preserved on merge.
* 🔬 chore: Add Diagnostic Logging for Code-Files Seeding
Temporary debug logs to diagnose why first-call file injection is not
firing in real agent runs. Logs `wantsCodeExec`, available tool-resource
keys, primed file count, and the seeded EXECUTE_CODE entry. Will revert
once the failure mode is identified.
* 🪛 refactor: Capture primedCodeFiles per-agent at init, merge across run
Replace the client.js eager `primeFiles` call with a per-agent capture at
initialization time so every agent in a multi-agent run (primary +
handoff + addedConvo) contributes its `tool_resources.execute_code`
files to the shared `Graph.sessions` seed.
- handleTools.js (eager loadTools): the `execute_code` factory closes
over a `primedCodeFiles` slot and surfaces it in the return.
- ToolService.js loadToolDefinitionsWrapper (event-driven): captures
`files` from the existing `primeCodeFiles` call (was dropping them
while only keeping `toolContext`) and surfaces them.
- packages/api initialize.ts: the loadTools callback contract now
includes `primedCodeFiles`, threaded onto `InitializedAgent`.
- client.js: iterate `[primary, ...agentConfigs.values()]` and merge
each agent's `primedCodeFiles` into `initialSessions`. Drop the
primary-only `primeCodeFiles` call and diagnostic logs from the prior
attempt — wrong layer (single-agent), wrong gate (`agent.tools`
contained Tool instances after init, so the `.includes("execute_code")`
string check always failed).
* 🔬 chore: Add per-agent diagnostic logs for code-files seeding
Logs `tool_resources` keys + file counts inside loadToolDefinitionsWrapper
and per-agent `primedCodeFiles` + final initialSessions inside
AgentClient. Will revert once the failure mode is confirmed.
* 🔬 chore: Add file-lookup diagnostics inside initializeAgent
Logs the inputs and intermediate counts of the conversation-file lookup
chain (convo file ids, thread message ids, code-generated and
user-code file counts) so we can pinpoint why `tool_resources.execute_code`
is arriving empty at `loadToolDefinitionsWrapper` despite the agent
having `execute_code` in its tools list.
* 🔬 chore: Probe execute_code files without messageId filter
Adds a relaxed `getFiles({conversationId, context: execute_code})` probe
that runs only when `getCodeGeneratedFiles` returns empty. Lists what's
actually in the DB for this conversation so we can confirm whether the
file is missing entirely or whether the messageId filter is rejecting it.
* 🔬 chore: Fix probe getFiles arg order (sort vs projection)
Probe was passing a projection object as the sort arg, which mongoose
rejected with `Invalid sort value`. Move it to the third arg
(selectFields) so the probe actually runs.
* 🪢 fix: Preserve Original messageId on Code-Output File Update
Each `processCodeOutput` call was overwriting the persisted file's
`messageId` with the *current* run's id. When a turn re-creates an
existing file (filename + conversationId match → `claimCodeFile`
returns the existing record, `isUpdate=true`), the file's link to
the assistant message that originally produced it gets clobbered.
`initializeAgent` later runs `getCodeGeneratedFiles({messageId: $in: <thread>})`
to seed `tool_resources.execute_code` from prior-turn artifacts. With a
stale `messageId` (e.g. from a failed read attempt that re-shelled the
same filename), the file no longer matches the parent-walk thread, so
`tool_resources` arrives empty at agent init, the new
`primedCodeFiles` channel has nothing to seed, and the LLM can't see
its own prior-turn artifacts on the next turn — defeating the
just-added Graph-sessions seeding fix.
Preserve the existing `claimed.messageId` on update; first-creation
behavior is unchanged. The runtime return value still includes the
current run's `messageId` (via `Object.assign(file, { messageId })`)
so the artifact is correctly attributed to the live tool_call.
* 🧹 chore: Remove diagnostic logs from code-files seeding path
Drops the temporary debug logs added to trace the empty-tool_resources
failure mode. Production code paths (loadToolDefinitionsWrapper,
client.js seed loop, initializeAgent file lookup) are left as the
permanent shape: capture primedCodeFiles, merge across agents, seed
initialSessions before run start.
* 🪛 feat: read_file Sandbox Fallback for /mnt/data + Non-Skill Paths
When the model called `read_file` with a code-execution path (e.g.
`/mnt/data/sentinel.txt`), the handler returned a misleading
`Use format: {skillName}/{path}` error. Adds a sandbox-aware fallback:
- Short-circuit `/mnt/data/...` (can never be a skill reference) →
route to a sandbox `cat` via the new host-provided `readSandboxFile`
callback, which POSTs to the codeapi `/exec` endpoint.
- Skip the skill resolver entirely when `accessibleSkillIds` is empty
— the resolved-output of `resolveAgentScopedSkillIds` already
collapses the admin capability + ephemeral badge + persisted
`skills_enabled` chain, so an empty value is the authoritative
"skills aren't in scope for this agent" signal.
- For `{firstSegment}/...` paths, consult the catalog-derived
`activeSkillNames` Set (no DB read) to detect non-skill names and
fall through to the sandbox before the model has to retry with
`bash_tool`.
`activeSkillNames` is captured from `injectSkillCatalog`, threaded onto
`InitializedAgent`, into `agentToolContexts`, then through
`enrichWithSkillConfigurable` into `mergedConfigurable` for the handler.
The host implementation of `readSandboxFile` lives in
`api/server/services/Files/Code/process.js` and shells `cat <path>`
through the seeded sandbox session — `tc.codeSessionContext`
(emitted by ToolNode for `read_file` calls in `@librechat/agents`
v3.1.72+) provides the `session_id` + `_injected_files` so the read
lands in the same sandbox that holds prior-turn artifacts. When the
seeded context isn't available (older agents version, no codeapi
configured), the handler returns a model-visible error pointing at
`bash_tool` instead of silently failing.
Tests: 8 new `handleReadFileCall` cases cover the new short-circuits,
the skills-not-enabled gate, the activeSkillNames lookup, the
sandbox-fallback success path, and the bash_tool retry hint on
fallback failure. Existing `read_file` tests now opt into "skills are
in scope" via a `skillsInScope()` fixture (production wouldn't reach
the skill lookup with empty `accessibleSkillIds`).
* 🔧 chore: Update @librechat/agents dependency to version 3.1.72
Bumps the version of the @librechat/agents package across package-lock.json and relevant package.json files to ensure compatibility with the latest features and fixes.
* 🪛 refactor: Centralize Tool-Session Seed in buildInitialToolSessions Helper
Addresses review feedback on the per-agent merge in client.js:
- **Run-wide semantics, named explicitly.** The merge into a single
`Graph.sessions[EXECUTE_CODE]` was a deliberate match to the
agents-library design (`Graph.sessions` is shared across every
`ToolNode` in the run), but the inline `for (const a of agents)`
loop in `AgentClient.chatCompletion` made it look per-agent. Move
the logic to a TS helper `buildInitialToolSessions` that documents
the run-wide-by-design contract in one place. The CJS controller
now contains a single call site, no business logic.
- **Subagent walk (P2).** The previous loop only iterated
`[primary, ...agentConfigs.values()]`. Pure subagents are pruned
out of `agentConfigs` after init and retained on each parent's
`subagentAgentConfigs`, so their primed code files were silently
dropped from the seed. The helper now walks recursively, with a
visited-Set keyed on object identity that terminates safely on a
malformed agent graph (cycle).
- **`jest.setup.cjs` polyfill for undici `File`.** Reviewer hit
`ReferenceError: File is not defined` running the targeted spec on
WSL — a known Node 18 issue where `globalThis.File` from
`node:buffer` isn't auto-exposed. Polyfill it inside a Jest setup
file so the suite boots regardless of Node patch version.
Helper test coverage (8 new): skill-only / agent-only / both,
recursive subagent walk, cycle-safe walk, primary+subagent
deduplication, undefined/null entries in the agents iterable, and
representative session_id preservation across the merge.
16 tests pass total in `codeFilesSession.spec.ts` (8 prior + 8 new).
No behavior change vs. the previous commit for the existing
primary+agentConfigs case — subagent inclusion is the only new
behavior, and it matches what the existing seeding logic would have
done if subagents had been in `agentConfigs`.
* 🪛 fix: FIFO Walk Order in buildInitialToolSessions (P3 review)
The traversal used `Array.pop()` (LIFO), which visited the LAST
top-level agent first. The docstring says "primary first"; the code
contradicted it. When no skill seed exists the first-visited agent's
first file supplies the representative `session_id` written to
`Graph.sessions[EXECUTE_CODE]` — so a LIFO walk silently flipped which
agent that came from. `ToolNode` ultimately uses per-file `session_id`s
for runtime injection (so behavior was indistinguishable for current
callers), but the discrepancy was a footgun for any future consumer
that read the representative.
Switch to FIFO via `Array.shift()` to match both the docstring and the
existing `loadSubagentsFor` walk pattern in
`Endpoints/agents/initialize.js`. Add a regression test that asserts
the primary's `session_id` is the representative (and that all three
agents' files still contribute, with per-file `session_id`s preserved).
* 🔬 test: Lock In Code-Files Bug Fixes Per Comprehensive Review
Addresses MAJOR + MINOR + NIT findings from the multi-pass review:
**Finding #4 (MINOR) — empty relativePath misses sandbox fallback.**
A model calling `read_file("output/")` where "output" isn't a skill
name dead-ended with `Missing file path after skill name` instead of
being routed to the sandbox like every other malformed-path branch.
Add the same `codeEnvAvailable → handleSandboxFileFallback` pattern,
plus two regression tests.
**Finding #7 (NIT) — duplicate `skillsInScope()` helper.**
Hoist the identical helper out of two nested describe blocks to
module scope. Single source of truth.
**Finding #1 (MAJOR) — `persistedMessageId` had zero test coverage.**
The fix preserves a file's original `messageId` on update so
`getCodeGeneratedFiles` can still match it on subsequent turns. A
regression in the `isUpdate ? (claimed.messageId ?? messageId) : messageId`
ternary would silently re-introduce the original cross-turn priming
bug. Five new tests cover:
- UPDATE preserves `claimed.messageId` in the persisted record
- UPDATE falls back to current run id when `claimed.messageId` is
absent (legacy records predating the field)
- CREATE uses current run id (no claimed record exists)
- The runtime return value uses the LIVE id (artifact attribution)
even when the persisted record kept the original
- The image branch follows the same contract (would silently regress
if the ternary diverged across the two file-build branches)
The tests use a `snapshotCreateFileArgs()` helper because
`processCodeOutput` mutates the file object after `createFile`
returns (`Object.assign(file, { messageId, toolCallId })`) and a
naive `createFile.mock.calls[0][0]` would reflect the post-mutation
state instead of what was actually persisted.
**Finding #2 (MAJOR) — `readSandboxFile` had no direct tests.**
The model-controlled `file_path` flows through a POSIX single-quote
escape into a shell `cat` command, making this a security boundary.
A quoting regression would let a malicious filename break out of the
quoted argument and inject arbitrary shell. 20 new tests across:
- Shell quoting (7): plain filenames, embedded `'`, `$()`, backticks,
newlines, shell metachars, multiple consecutive single-quotes
- Payload shape (6): /exec URL, bash language, conditional
session_id / files inclusion, dedicated keepAlive:false agents
- Response handling (6): `{content}` on success, null on missing
base URL or absent stdout, throws on stderr-only, partial-success
returns stdout, transport errors are logged then rethrown
- Timeout (1): matches processCodeOutput's 15s SLA
Audited findings #5 (acknowledged tech debt — readSandboxFile in JS
workspace), #6 (pre-existing positional-args debt on
enrichWithSkillConfigurable), and #8 (cosmetic JSDoc style) — no
action taken per the reviewer's own assessment.
Audited finding #3 (walk order vs docstring) — already addressed in
commit
|
||
|
|
181d705579
|
🧹 fix: Clean Up Orphaned Agent File Stubs After Deletion (#12781)
* 🧹 fix: Prune Orphaned File References on File Deletion Deleting a file via the Manage Files tab left its file_id in every agent's tool_resources.*.file_ids. Stubs accumulate until the frontend dedupe keys them as duplicates and blocks all new uploads (issue #12776). - Add removeAgentResourceFilesFromAllAgents in packages/data-schemas: a single updateMany/$pullAll across every EToolResources category. - Invoke it from processDeleteRequest after db.deleteFiles so every referencing agent is cleaned up, not just the one passed in req.body. - Wrap the cleanup in try/catch so a stale agent update cannot mask a successful file deletion. * 🧼 fix: Prune Orphaned File References on Agent Update Already-affected agents would stay broken even after the delete-time fix: the stubs sit on the agent document until something strips them. Heal them on the next save (issue #12776). - Add collectToolResourceFileIds + stripFileIdsFromToolResources helpers in @librechat/api — centralizing the tool_resources traversal used by the controller and the follow-up migration script. - In updateAgentHandler, check the effective tool_resources against the files collection. When orphans are found, either strip them from the incoming tool_resources (if the update sets them) or run the bulk cleanup (if the update leaves tool_resources untouched). * 🧰 chore: Add Migration to Clean Up Orphaned Agent File References Complements the delete-time and save-time fixes by healing agents that already accumulated orphan stubs before the upgrade (issue #12776). The script is idempotent — re-running it on a clean database is a no-op. - Add config/migrate-orphaned-agent-files.js following the existing migrate-*.js convention: --dry-run by default omitted (writes by default) and --batch-size= tuning knob. Streams agents via cursor. - Register migrate:orphaned-agent-files and :dry-run npm scripts. - Reuse collectToolResourceFileIds from @librechat/api so migration and runtime share the same traversal logic. * 🩹 fix: Address Codex/Copilot Review on Orphaned Agent File Cleanup Refines the #12776 fix series based on automated review feedback. - Scope save-time pruning to the current agent only. When a PATCH carries tool_resources, strip orphans from the incoming payload and pay the DB round-trip only then. Removes the collection-wide updateMany previously triggered when tool_resources was absent (Codex P2 / Copilot). - Wrap the orphan check in try/catch so a transient db.getFiles failure can't turn a good save into a 500 (comprehensive review #1). - Replace Object.values(EToolResources) casts with an explicit list of agent-side categories in both orphans.ts and agent.ts. code_interpreter belongs to the Assistants API and isn't a key of AgentToolResources — including it was a type lie and generated dead MongoDB clauses (comprehensive review #3, #8). - Export TOOL_RESOURCE_KEYS from @librechat/api and consume it in the migration script, dropping one duplicated definition (#4). - Cap migration results.details at 50 sample entries so the memory footprint stays bounded on deployments with thousands of corrupted agents (Codex P3). - Add migrate:orphaned-agent-files:batch npm script to match the convention set by migrate-agent-permissions / migrate-prompt-permissions (#7). - Add controller-level tests covering the three orphan-pruning paths: strip from incoming tool_resources, leave alone when tool_resources is absent, swallow db.getFiles errors and still save (#6). - Back pre-existing "should validate tool_resources in updates" test's file_ids with real File docs — the new pruning would otherwise strip them, and that test is about OCR conversion / schema filtering, not file existence. Register the File model in beforeAll so the fixture works. * 🩹 fix: Tighten TOOL_RESOURCE_KEYS Type and Align Migration Sample Output Two follow-ups from the second review pass. - Type data-schemas' TOOL_RESOURCE_KEYS as ReadonlyArray<keyof AgentToolResources> instead of readonly string[]. Data-schemas depends on data-provider, so the import is clean. Catches typos and aligns with the matching export in @librechat/api — doesn't guarantee exhaustiveness, but that's a TypeScript limitation, not a workspace one. - Align the migration's console output with DETAIL_SAMPLE_LIMIT: print every collected detail (up to 50) and, when more agents were affected than the sample size allowed, show a truncation notice. The old hard cap of 25 meant affected agents in the 26-50 range were collected but never shown. * ✅ test: Add Integration Coverage for Orphan Cleanup Paths (#12776) Exercise the delete-time and migration paths end-to-end against a real in-memory Mongo. Catches integration bugs the isolated unit tests on each layer couldn't. - api/server/services/Files/process.integration.spec.js — the primary repro: seed an Agent + File, call processDeleteRequest, assert the file_id disappears from every referencing agent's tool_resources while unrelated agents stay untouched. Also covers the no-op case and confirms a failure in the new cleanup step cannot roll back the file deletion itself. - api/test/migrate-orphaned-agent-files.spec.js — drives the migration module: --dry-run reports without writing, apply mode prunes across every tool_resource category, re-running is idempotent, and DETAIL_SAMPLE_LIMIT caps the in-memory sample on wide corruption. Mocks only the connect helper (the spec owns the mongoose instance) so the real migration code path — cursor, $pullAll, reduce — runs. * 🔒 fix: Run Orphan Cleanup Migration in System Tenant Context Codex P2 catch: under TENANT_ISOLATION_STRICT=true, the migration throws on the very first Agent.countDocuments() because the tenant isolation plugin fail-closes on queries without tenant context — which makes migrate:orphaned-agent-files unusable on the exact deployments most likely to have accumulated corruption. - Wrap the scan/prune body in runAsSystem so queries bypass the tenant filter (SYSTEM_TENANT_ID sentinel). The migration legitimately needs cross-tenant visibility — this is the same pattern seedDatabase and the S3 refresh job already use. - Add a regression test that spies on Agent.countDocuments() and asserts the active tenantStorage context is SYSTEM_TENANT_ID during the call. Pins the wrap against future regressions without the brittleness of toggling the strict-mode env var (which caches on first read). Note: the delete-time and save-time paths already run inside an authenticated HTTP request where tenantStorage.run is set by auth middleware, so the cleanup naturally scopes to the current tenant — which is the correct behavior there since file ownership is tenant-scoped. * 🧹 chore: Drop Unused path Import From Process Integration Spec Leftover from an earlier iteration that resolved the migration path via path.resolve before I switched to a relative require. The import does nothing now — removing it. |
||
|
|
48c3d31db3
|
🔊 fix: Preserve Log Metadata on Console for Warn/Error Levels (#12737)
* 🔊 fix: Preserve Log Metadata on Console for Warn/Error Levels The default console formatter discarded every structured metadata key on the winston info object — only `CONSOLE_JSON=true` preserved it. That meant failures emitted by the agents SDK (e.g. "Summarization LLM call failed") reached stdout without the provider, model, or underlying error attached, leaving users unable to diagnose the root cause. - Add `formatConsoleMeta` helper to serialize non-reserved metadata as a compact JSON trailer, with per-value string truncation and safe handling of circular references. - Append the metadata trailer to warn/error console lines; info/debug behavior is unchanged. - Relax `debugTraverse`'s debug-only gate so warn/error messages routed through the debug formatter also surface their metadata. - Add a `formatConsoleMeta` stub to the shared logger mock so existing tests keep working. * 🔐 fix: Also Redact Sensitive Patterns on Warn Console Lines The warn-level console output now includes a metadata trailer that may contain provider-returned error strings with embedded tokens or keys (e.g. `Bearer ...`, `sk-...`). Apply `redactMessage` to warn lines in addition to error, matching the new surface area. * 🔐 fix: Redact Sensitive Tokens Embedded in JSON Metadata Two gaps in the existing console redaction that became user-visible once warn/error lines started emitting structured metadata: 1. The OpenAI-key regex (`/^(sk-)[^\s]+/`) was anchored to start-of-line, so keys embedded inside JSON payloads (e.g. `{"apiKey":"sk-..."}`) were never redacted. Every console line begins with a timestamp, so the anchor effectively made this pattern dead code. 2. `formatConsoleMeta` stringified metadata values verbatim; a sensitive string value was only redacted by the whole-line regex pass, which missed the anchored `sk-` case above. Fix: - Drop the `^` anchor; add `/g` so every occurrence is redacted, not just the first. - Also exclude `"` and `'` from the token body so JSON-embedded values terminate at the closing quote rather than chewing into the next field. - Simplify `redactMessage` to apply patterns directly (dropping the `getMatchingSensitivePatterns` filter) — the filter used `.test()` which has stateful behavior on `/g` regexes and is no longer needed. - `formatConsoleMeta` now runs `redactMessage` over every string value before JSON serialization, so the metadata trailer is safe even on the warn path. - Add regression tests covering both fixes. Reviewed-by: Codex (P1 finding on PR #12737, commit |
||
|
|
a0fed6173c
|
🗂️ refactor: Migrate S3 Storage to TypeScript in packages/api (#11947)
* Migrate S3 storage module with unit and integration tests - Migrate S3 CRUD and image operations to packages/api/src/storage/s3/ - Add S3ImageService class with dependency injection - Add unit tests using aws-sdk-client-mock - Add integration tests with real s3 bucket (condition presence of AWS_TEST_BUCKET_NAME) * AI Review Findings Fixes * chore: tests and refactor S3 storage types - Added mock implementations for the 'sharp' library in various test files to improve image processing testing. - Updated type references in S3 storage files from MongoFile to TFile for consistency and type safety. - Refactored S3 CRUD operations to ensure proper handling of file types and improve code clarity. - Enhanced integration tests to validate S3 file operations and error handling more effectively. * chore: rename test file * Remove duplicate import of refreshS3Url * chore: imports order * fix: remove duplicate imports for S3 URL handling in UserController * fix: remove duplicate import of refreshS3FileUrls in files.js * test: Add mock implementations for 'sharp' and '@librechat/api' in UserController tests - Introduced mock functions for the 'sharp' library to facilitate image processing tests, including metadata retrieval and buffer conversion. - Enhanced mocking for '@librechat/api' to ensure consistent behavior in tests, particularly for the needsRefresh and getNewS3URL functions. --------- Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
|
8ba2bde5c1
|
📦 refactor: Consolidate DB models, encapsulating Mongoose usage in data-schemas (#11830)
* chore: move database model methods to /packages/data-schemas * chore: add TypeScript ESLint rule to warn on unused variables * refactor: model imports to streamline access - Consolidated model imports across various files to improve code organization and reduce redundancy. - Updated imports for models such as Assistant, Message, Conversation, and others to a unified import path. - Adjusted middleware and service files to reflect the new import structure, ensuring functionality remains intact. - Enhanced test files to align with the new import paths, maintaining test coverage and integrity. * chore: migrate database models to packages/data-schemas and refactor all direct Mongoose Model usage outside of data-schemas * test: update agent model mocks in unit tests - Added `getAgent` mock to `client.test.js` to enhance test coverage for agent-related functionality. - Removed redundant `getAgent` and `getAgents` mocks from `openai.spec.js` and `responses.unit.spec.js` to streamline test setup and reduce duplication. - Ensured consistency in agent mock implementations across test files. * fix: update types in data-schemas * refactor: enhance type definitions in transaction and spending methods - Updated type definitions in `checkBalance.ts` to use specific request and response types. - Refined `spendTokens.ts` to utilize a new `SpendTxData` interface for better clarity and type safety. - Improved transaction handling in `transaction.ts` by introducing `TransactionResult` and `TxData` interfaces, ensuring consistent data structures across methods. - Adjusted unit tests in `transaction.spec.ts` to accommodate new type definitions and enhance robustness. * refactor: streamline model imports and enhance code organization - Consolidated model imports across various controllers and services to a unified import path, improving code clarity and reducing redundancy. - Updated multiple files to reflect the new import structure, ensuring all functionalities remain intact. - Enhanced overall code organization by removing duplicate import statements and optimizing the usage of model methods. * feat: implement loadAddedAgent and refactor agent loading logic - Introduced `loadAddedAgent` function to handle loading agents from added conversations, supporting multi-convo parallel execution. - Created a new `load.ts` file to encapsulate agent loading functionalities, including `loadEphemeralAgent` and `loadAgent`. - Updated the `index.ts` file to export the new `load` module instead of the deprecated `loadAgent`. - Enhanced type definitions and improved error handling in the agent loading process. - Adjusted unit tests to reflect changes in the agent loading structure and ensure comprehensive coverage. * refactor: enhance balance handling with new update interface - Introduced `IBalanceUpdate` interface to streamline balance update operations across the codebase. - Updated `upsertBalanceFields` method signatures in `balance.ts`, `transaction.ts`, and related tests to utilize the new interface for improved type safety. - Adjusted type imports in `balance.spec.ts` to include `IBalanceUpdate`, ensuring consistency in balance management functionalities. - Enhanced overall code clarity and maintainability by refining type definitions related to balance operations. * feat: add unit tests for loadAgent functionality and enhance agent loading logic - Introduced comprehensive unit tests for the `loadAgent` function, covering various scenarios including null and empty agent IDs, loading of ephemeral agents, and permission checks. - Enhanced the `initializeClient` function by moving `getConvoFiles` to the correct position in the database method exports, ensuring proper functionality. - Improved test coverage for agent loading, including handling of non-existent agents and user permissions. * chore: reorder memory method exports for consistency - Moved `deleteAllUserMemories` to the correct position in the exported memory methods, ensuring a consistent and logical order of method exports in `memory.ts`. |
||
|
|
54fc9c2c99
|
♾️ fix: Permanent Ban Cache and Expired Ban Cleanup Defects (#12324)
* fix: preserve ban data object in checkBan to prevent permanent cache The !! operator on line 108 coerces the ban data object to a boolean, losing the expiresAt property. This causes: 1. Number(true.expiresAt) = NaN → expired bans never cleaned from banLogs 2. banCache.set(key, true, NaN) → Keyv stores with expires: null (permanent) 3. IP-based cache entries persist indefinitely, blocking unrelated users Fix: replace isBanned (boolean) with banData (original object) so that expiresAt is accessible for TTL calculation and proper cache expiry. * fix: address checkBan cleanup defects exposed by ban data fix The prior commit correctly replaced boolean coercion with the ban data object, but activated previously-dead cleanup code with several defects: - IP-only expired bans fell through cleanup without returning next(), re-caching with negative TTL (permanent entry) and blocking the user - Redis deployments used cache-prefixed keys for banLogs.delete(), silently failing since bans are stored at raw keys - banCache.set() calls were fire-and-forget, silently dropping errors - No guard for missing/invalid expiresAt reproduced the NaN TTL bug on legacy ban records Consolidate expired-ban cleanup into a single block that always returns next(), use raw keys (req.ip, userId) for banLogs.delete(), add an expiresAt validity guard, await cache writes with error logging, and parallelize independent I/O with Promise.all. Add 25 tests covering all checkBan code paths including the specific regressions for IP-only cleanup, Redis key mismatch, missing expiresAt, and cache write failures. --------- Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
|
7692fa837e
|
🪣 fix: S3 path-style URL support for MinIO, R2, and custom endpoints (#11894)
* 🪣 fix: S3 path-style URL support for MinIO, R2, and custom endpoints `extractKeyFromS3Url` now uses `AWS_BUCKET_NAME` to automatically detect and strip the bucket prefix from path-style URLs, fixing `NoSuchKey` errors on URL refresh for any S3-compatible provider using a custom endpoint (MinIO, Cloudflare R2, Hetzner, Backblaze B2, etc.). No additional configuration required — the bucket name is already a required env var for S3 to function. `initializeS3` now passes `forcePathStyle: true` to the S3Client constructor when `AWS_FORCE_PATH_STYLE=true` is set. Required for providers whose SSL certificates do not support virtual-hosted-style bucket subdomains (e.g. Hetzner Object Storage), which previously caused 401 / SignatureDoesNotMatch on upload. Additional fixes: - Suppress error log noise in `extractKeyFromS3Url` catch path: plain S3 keys no longer log as errors, only inputs that start with http(s):// do - Fix test env var ordering so module-level constants pick up `AWS_BUCKET_NAME` and `S3_URL_EXPIRY_SECONDS` correctly before the module is required - Add missing `deleteRagFile` mock and assertion in `deleteFileFromS3` tests - Add `AWS_BUCKET_NAME` cleanup to `afterEach` to prevent cross-test pollution - Add `initializeS3` unit tests covering endpoint, forcePathStyle, credentials, singleton, and IRSA code paths - Document `AWS_FORCE_PATH_STYLE` in `.env.example`, `dotenv.mdx`, and `s3.mdx` * 🪣 fix: Enhance S3 URL key extraction for custom endpoints Updated `extractKeyFromS3Url` to support precise key extraction when using custom endpoints with path-style URLs. The logic now accounts for the `AWS_ENDPOINT_URL` and `AWS_FORCE_PATH_STYLE` environment variables, ensuring correct key handling for various S3-compatible providers. Added unit tests to verify the new functionality, including scenarios for endpoints with base paths. This improves compatibility and reduces potential errors when interacting with S3-like services. |
||
|
|
5d2b7fa4d5
|
🪣 fix: Proper Key Extraction from S3 URL (#11241)
* ✨ feat: Enhance S3 URL handling and add comprehensive tests for CRUD operations * 🔒 fix: Improve S3 URL key extraction with enhanced logging and additional test cases * chore: removed some duplicate testcases and fixed incorrect apostrophes * fix: Log error for malformed URLs * test: Add additional test case for extracting keys from S3 URLs * fix: Enhance S3 URL key extraction logic and improve error handling with additional test cases * test: Add test case for stripping bucket from custom endpoint URLs with forcePathStyle enabled * refactor: Update S3 path style handling and enhance environment configuration for S3-compatible services * refactor: Remove S3_FORCE_PATH_STYLE dependency and streamline S3 URL key extraction logic --------- Co-authored-by: Danny Avila <danny@librechat.ai> |
||
|
|
b9792160e2
|
💾 feat: Add Memory Configuration Options for CI unit tests (#10567)
* 💾 feat: Add Memory Configuration Options for CI unit tests - configured GitHub Actions workflows with configurable Node.js heap allocation, defaults to 6144 MiB - added heap usage logging for memory monitoring and debugging - increased Docker frontend build memory allocation to ensure consistent memory limits - optimized Jest timeout for tokenSplit test * 💾 feat: Add Memory Configuration Options for CI unit tests - responding to PR feedback from Copilot |
||
|
|
b7ea340769
|
🏞️ feat: Modifiable OpenAI Image Gen Model Environment Variable (#11082) | ||
|
|
04a4a2aa44
|
🧵 refactor: Migrate Endpoint Initialization to TypeScript (#10794)
* refactor: move endpoint initialization methods to typescript * refactor: move agent init to packages/api - Introduced `initialize.ts` for agent initialization, including file processing and tool loading. - Updated `resources.ts` to allow optional appConfig parameter. - Enhanced endpoint configuration handling in various initialization files to support model parameters. - Added new artifacts and prompts for React component generation. - Refactored existing code to improve type safety and maintainability. * refactor: streamline endpoint initialization and enhance type safety - Updated initialization functions across various endpoints to use a consistent request structure, replacing `unknown` types with `ServerResponse`. - Simplified request handling by directly extracting keys from the request body. - Improved type safety by ensuring user IDs are safely accessed with optional chaining. - Removed unnecessary parameters and streamlined model options handling for better clarity and maintainability. * refactor: moved ModelService and extractBaseURL to packages/api - Added comprehensive tests for the models fetching functionality, covering scenarios for OpenAI, Anthropic, Google, and Ollama models. - Updated existing endpoint index to include the new models module. - Enhanced utility functions for URL extraction and model data processing. - Improved type safety and error handling across the models fetching logic. * refactor: consolidate utility functions and remove unused files - Merged `deriveBaseURL` and `extractBaseURL` into the `@librechat/api` module for better organization. - Removed redundant utility files and their associated tests to streamline the codebase. - Updated imports across various client files to utilize the new consolidated functions. - Enhanced overall maintainability by reducing the number of utility modules. * refactor: replace ModelService references with direct imports from @librechat/api and remove ModelService file * refactor: move encrypt/decrypt methods and key db methods to data-schemas, use `getProviderConfig` from `@librechat/api` * chore: remove unused 'res' from options in AgentClient * refactor: file model imports and methods - Updated imports in various controllers and services to use the unified file model from '~/models' instead of '~/models/File'. - Consolidated file-related methods into a new file methods module in the data-schemas package. - Added comprehensive tests for file methods including creation, retrieval, updating, and deletion. - Enhanced the initializeAgent function to accept dependency injection for file-related methods. - Improved error handling and logging in file methods. * refactor: streamline database method references in agent initialization * refactor: enhance file method tests and update type references to IMongoFile * refactor: consolidate database method imports in agent client and initialization * chore: remove redundant import of initializeAgent from @librechat/api * refactor: move checkUserKeyExpiry utility to @librechat/api and update references across endpoints * refactor: move updateUserPlugins logic to user.ts and simplify UserController * refactor: update imports for user key management and remove UserService * refactor: remove unused Anthropics and Bedrock endpoint files and clean up imports * refactor: consolidate and update encryption imports across various files to use @librechat/data-schemas * chore: update file model mock to use unified import from '~/models' * chore: import order * refactor: remove migrated to TS agent.js file and its associated logic from the endpoints * chore: add reusable function to extract imports from source code in unused-packages workflow * chore: enhance unused-packages workflow to include @librechat/api dependencies and improve dependency extraction * chore: improve dependency extraction in unused-packages workflow with enhanced error handling and debugging output * chore: add detailed debugging output to unused-packages workflow for better visibility into unused dependencies and exclusion lists * chore: refine subpath handling in unused-packages workflow to correctly process scoped and non-scoped package imports * chore: clean up unused debug output in unused-packages workflow and reorganize type imports in initialize.ts |
||
|
|
656e1abaea
|
🪦 refactor: Remove Legacy Code (#10533)
* 🗑️ chore: Remove unused Legacy Provider clients and related helpers * Deleted OpenAIClient and GoogleClient files along with their associated tests. * Removed references to these clients in the clients index file. * Cleaned up typedefs by removing the OpenAISpecClient export. * Updated chat controllers to use the OpenAI SDK directly instead of the removed client classes. * chore/remove-openapi-specs * 🗑️ chore: Remove unused mergeSort and misc utility functions * Deleted mergeSort.js and misc.js files as they are no longer needed. * Removed references to cleanUpPrimaryKeyValue in messages.js and adjusted related logic. * Updated mongoMeili.ts to eliminate local implementations of removed functions. * chore: remove legacy endpoints * chore: remove all plugins endpoint related code * chore: remove unused prompt handling code and clean up imports * Deleted handleInputs.js and instructions.js files as they are no longer needed. * Removed references to these files in the prompts index.js. * Updated docker-compose.yml to simplify reverse proxy configuration. * chore: remove unused LightningIcon import from Icons.tsx * chore: clean up translation.json by removing deprecated and unused keys * chore: update Jest configuration and remove unused mock file * Simplified the setupFiles array in jest.config.js by removing the fetchEventSource mock. * Deleted the fetchEventSource.js mock file as it is no longer needed. * fix: simplify endpoint type check in Landing and ConversationStarters components * Updated the endpoint type check to use strict equality for better clarity and performance. * Ensured consistency in the handling of the azureOpenAI endpoint across both components. * chore: remove unused dependencies from package.json and package-lock.json * chore: remove legacy EditController, associated routes and imports * chore: update banResponse logic to refine request handling for banned users * chore: remove unused validateEndpoint middleware and its references * chore: remove unused 'res' parameter from initializeClient in multiple endpoint files * chore: remove unused 'isSmallScreen' prop from BookmarkNav and NewChat components; clean up imports in ArchivedChatsTable and useSetIndexOptions hooks; enhance localization in PromptVersions * chore: remove unused import of Constants and TMessage from MobileNav; retain only necessary QueryKeys import * chore: remove unused TResPlugin type and related references; clean up imports in types and schemas |
||
|
|
ef3bf0a932
|
🆔 feat: Add OpenID Connect Federated Provider Token Support (#9931)
* feat: Add OpenID Connect federated provider token support
Implements support for passing federated provider tokens (Cognito, Azure AD, Auth0)
as variables in LibreChat's librechat.yaml configuration for both custom endpoints
and MCP servers.
Features:
- New LIBRECHAT_OPENID_* template variables for federated provider tokens
- JWT claims parsing from ID tokens without verification (for claim extraction)
- Token validation with expiration checking
- Support for multiple token storage locations (federatedTokens, openidTokens)
- Integration with existing template variable system
- Comprehensive test suite with Cognito-specific scenarios
- Provider-agnostic design supporting Cognito, Azure AD, Auth0, etc.
Security:
- Server-side only token processing
- Automatic token expiration validation
- Graceful fallbacks for missing/invalid tokens
- No client-side token exposure
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: Add federated token propagation to OIDC authentication strategies
Adds federatedTokens object to user during authentication to enable
federated provider token template variables in LibreChat configuration.
Changes:
- OpenID JWT Strategy: Extract raw JWT from Authorization header and
attach as federatedTokens.access_token to enable {{LIBRECHAT_OPENID_TOKEN}}
placeholder resolution
- OpenID Strategy: Attach tokenset tokens as federatedTokens object to
standardize token access across both authentication strategies
This enables proper token propagation for custom endpoints and MCP
servers that require federated provider tokens for authorization.
Resolves missing token issue reported by @ramden in PR #9931
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Denis Ramic <denis.ramic@nfon.com>
Co-Authored-By: Claude <noreply@anthropic.com>
* test: Add federatedTokens validation tests for OIDC strategies
Adds comprehensive test coverage for the federated token propagation
feature implemented in the authentication strategies.
Tests added:
- Verify federatedTokens object is attached to user with correct structure
(access_token, refresh_token, expires_at)
- Verify both tokenset and federatedTokens are present in user object
- Ensure tokens from OIDC provider are correctly propagated
Also fixes existing test suite by adding missing mocks:
- isEmailDomainAllowed function mock
- findOpenIDUser function mock
These tests validate the fix from commit
|
||
|
|
49c57b27fd
|
➿ fix: createFileSearchTool to return tuples for error messages (#10547)
|
||
|
|
5b1a31ef4d
|
🔄 refactor: Optimize MCP Tool Initialization
🔄 refactor: Optimize MCP Tool Initialization
fix: update tool caching to use separated mcp logic
refactor: Replace `req.user` with `userId` in MCP handling functions
refactor: Replace `req` parameter with `userId` in file search tool functions
fix: Update user connection parameter to use object format in reinitMCPServer
refactor: Simplify MCP tool creation logic and improve handling of tool configurations to avoid capturing too much in closures
refactor: ensure MCP available tools are fetched from cache only when needed
|
||
|
|
9a210971f5
|
🛜 refactor: Streamline App Config Usage (#9234)
* WIP: app.locals refactoring
WIP: appConfig
fix: update memory configuration retrieval to use getAppConfig based on user role
fix: update comment for AppConfig interface to clarify purpose
🏷️ refactor: Update tests to use getAppConfig for endpoint configurations
ci: Update AppService tests to initialize app config instead of app.locals
ci: Integrate getAppConfig into remaining tests
refactor: Update multer storage destination to use promise-based getAppConfig and improve error handling in tests
refactor: Rename initializeAppConfig to setAppConfig and update related tests
ci: Mock getAppConfig in various tests to provide default configurations
refactor: Update convertMCPToolsToPlugins to use mcpManager for server configuration and adjust related tests
chore: rename `Config/getAppConfig` -> `Config/app`
fix: streamline OpenAI image tools configuration by removing direct appConfig dependency and using function parameters
chore: correct parameter documentation for imageOutputType in ToolService.js
refactor: remove `getCustomConfig` dependency in config route
refactor: update domain validation to use appConfig for allowed domains
refactor: use appConfig registration property
chore: remove app parameter from AppService invocation
refactor: update AppConfig interface to correct registration and turnstile configurations
refactor: remove getCustomConfig dependency and use getAppConfig in PluginController, multer, and MCP services
refactor: replace getCustomConfig with getAppConfig in STTService, TTSService, and related files
refactor: replace getCustomConfig with getAppConfig in Conversation and Message models, update tempChatRetention functions to use AppConfig type
refactor: update getAppConfig calls in Conversation and Message models to include user role for temporary chat expiration
ci: update related tests
refactor: update getAppConfig call in getCustomConfigSpeech to include user role
fix: update appConfig usage to access allowedDomains from actions instead of registration
refactor: enhance AppConfig to include fileStrategies and update related file strategy logic
refactor: update imports to use normalizeEndpointName from @librechat/api and remove redundant definitions
chore: remove deprecated unused RunManager
refactor: get balance config primarily from appConfig
refactor: remove customConfig dependency for appConfig and streamline loadConfigModels logic
refactor: remove getCustomConfig usage and use app config in file citations
refactor: consolidate endpoint loading logic into loadEndpoints function
refactor: update appConfig access to use endpoints structure across various services
refactor: implement custom endpoints configuration and streamline endpoint loading logic
refactor: update getAppConfig call to include user role parameter
refactor: streamline endpoint configuration and enhance appConfig usage across services
refactor: replace getMCPAuthMap with getUserMCPAuthMap and remove unused getCustomConfig file
refactor: add type annotation for loadedEndpoints in loadEndpoints function
refactor: move /services/Files/images/parse to TS API
chore: add missing FILE_CITATIONS permission to IRole interface
refactor: restructure toolkits to TS API
refactor: separate manifest logic into its own module
refactor: consolidate tool loading logic into a new tools module for startup logic
refactor: move interface config logic to TS API
refactor: migrate checkEmailConfig to TypeScript and update imports
refactor: add FunctionTool interface and availableTools to AppConfig
refactor: decouple caching and DB operations from AppService, make part of consolidated `getAppConfig`
WIP: fix tests
* fix: rebase conflicts
* refactor: remove app.locals references
* refactor: replace getBalanceConfig with getAppConfig in various strategies and middleware
* refactor: replace appConfig?.balance with getBalanceConfig in various controllers and clients
* test: add balance configuration to titleConvo method in AgentClient tests
* chore: remove unused `openai-chat-tokens` package
* chore: remove unused imports in initializeMCPs.js
* refactor: update balance configuration to use getAppConfig instead of getBalanceConfig
* refactor: integrate configMiddleware for centralized configuration handling
* refactor: optimize email domain validation by removing unnecessary async calls
* refactor: simplify multer storage configuration by removing async calls
* refactor: reorder imports for better readability in user.js
* refactor: replace getAppConfig calls with req.config for improved performance
* chore: replace getAppConfig calls with req.config in tests for centralized configuration handling
* chore: remove unused override config
* refactor: add configMiddleware to endpoint route and replace getAppConfig with req.config
* chore: remove customConfig parameter from TTSService constructor
* refactor: pass appConfig from request to processFileCitations for improved configuration handling
* refactor: remove configMiddleware from endpoint route and retrieve appConfig directly in getEndpointsConfig if not in `req.config`
* test: add mockAppConfig to processFileCitations tests for improved configuration handling
* fix: pass req.config to hasCustomUserVars and call without await after synchronous refactor
* fix: type safety in useExportConversation
* refactor: retrieve appConfig using getAppConfig in PluginController and remove configMiddleware from plugins route, to avoid always retrieving when plugins are cached
* chore: change `MongoUser` typedef to `IUser`
* fix: Add `user` and `config` fields to ServerRequest and update JSDoc type annotations from Express.Request to ServerRequest
* fix: remove unused setAppConfig mock from Server configuration tests
|
||
|
|
fc8fd489d6
|
🔗 fix: File Citation Processing to Use Tool Artifacts | ||
|
|
949682ef0f
|
🏪 feat: Agent Marketplace
bugfix: Enhance Agent and AgentCategory schemas with new fields for category, support contact, and promotion status refactored and moved agent category methods and schema to data-schema package 🔧 fix: Merge and Rebase Conflicts - Move AgentCategory from api/models to @packages/data-schemas structure - Add schema, types, methods, and model following codebase conventions - Implement auto-seeding of default categories during AppService startup - Update marketplace controller to use new data-schemas methods - Remove old model file and standalone seed script refactor: unify agent marketplace to single endpoint with cursor pagination - Replace multiple marketplace routes with unified /marketplace endpoint - Add query string controls: category, search, limit, cursor, promoted, requiredPermission - Implement cursor-based pagination replacing page-based system - Integrate ACL permissions for proper access control - Fix ObjectId constructor error in Agent model - Update React components to use unified useGetMarketplaceAgentsQuery hook - Enhance type safety and remove deprecated useDynamicAgentQuery - Update tests for new marketplace architecture -Known issues: see more button after category switching + Unit tests feat: add icon property to ProcessedAgentCategory interface - Add useMarketplaceAgentsInfiniteQuery and useGetAgentCategoriesQuery to client/src/data-provider/Agents/ - Replace manual pagination in AgentGrid with infinite query pattern - Update imports to use local data provider instead of librechat-data-provider - Add proper permission handling with PERMISSION_BITS.VIEW/EDIT constants - Improve agent access control by adding requiredPermission validation in backend - Remove manual cursor/state management in favor of infinite query built-ins - Maintain existing search and category filtering functionality refactor: consolidate agent marketplace endpoints into main agents API and improve data management consistency - Remove dedicated marketplace controller and routes, merging functionality into main agents v1 API - Add countPromotedAgents function to Agent model for promoted agents count - Enhance getListAgents handler with marketplace filtering (category, search, promoted status) - Move getAgentCategories from marketplace to v1 controller with same functionality - Update agent mutations to invalidate marketplace queries and handle multiple permission levels - Improve cache management by updating all agent query variants (VIEW/EDIT permissions) - Consolidate agent data access patterns for better maintainability and consistency - Remove duplicate marketplace route definitions and middleware selected view only agents injected in the drop down fix: remove minlength validation for support contact name in agent schema feat: add validation and error messages for agent name in AgentConfig and AgentPanel fix: update agent permission check logic in AgentPanel to simplify condition Fix linting WIP Fix Unit tests WIP ESLint fixes eslint fix refactor: enhance isDuplicateVersion function in Agent model for improved comparison logic - Introduced handling for undefined/null values in array and object comparisons. - Normalized array comparisons to treat undefined/null as empty arrays. - Added deep comparison for objects and improved handling of primitive values. - Enhanced projectIds comparison to ensure consistent MongoDB ObjectId handling. refactor: remove redundant properties from IAgent interface in agent schema chore: update localization for agent detail component and clean up imports ci: update access middleware tests chore: remove unused PermissionTypes import from Role model ci: update AclEntry model tests ci: update button accessibility labels in AgentDetail tests refactor: update exhaustive dep. lint warning 🔧 fix: Fixed agent actions access feat: Add role-level permissions for agent sharing people picker - Add PEOPLE_PICKER permission type with VIEW_USERS and VIEW_GROUPS permissions - Create custom middleware for query-aware permission validation - Implement permission-based type filtering in PeoplePicker component - Hide people picker UI when user lacks permissions, show only public toggle - Support granular access: users-only, groups-only, or mixed search modes refactor: Replace marketplace interface config with permission-based system - Add MARKETPLACE permission type to handle marketplace access control - Update interface configuration to use role-based marketplace settings (admin/user) - Replace direct marketplace boolean config with permission-based checks - Modify frontend components to use marketplace permissions instead of interface config - Update agent query hooks to use marketplace permissions for determining permission levels - Add marketplace configuration structure similar to peoplePicker in YAML config - Backend now sets MARKETPLACE permissions based on interface configuration - When marketplace enabled: users get agents with EDIT permissions in dropdown lists (builder mode) - When marketplace disabled: users get agents with VIEW permissions in dropdown lists (browse mode) 🔧 fix: Redirect to New Chat if No Marketplace Access and Required Agent Name Placeholder (#8213) * Fix: Fix the redirect to new chat page if access to marketplace is denied * Fixed the required agent name placeholder --------- Co-authored-by: Atef Bellaaj <slalom.bellaaj@external.daimlertruck.com> chore: fix tests, remove unnecessary imports refactor: Implement permission checks for file access via agents - Updated `hasAccessToFilesViaAgent` to utilize permission checks for VIEW and EDIT access. - Replaced project-based access validation with permission-based checks. - Enhanced tests to cover new permission logic and ensure proper access control for files associated with agents. - Cleaned up imports and initialized models in test files for consistency. refactor: Enhance test setup and cleanup for file access control - Introduced modelsToCleanup array to track models added during tests for proper cleanup. - Updated afterAll hooks in test files to ensure all collections are cleared and only added models are deleted. - Improved consistency in model initialization across test files. - Added comments for clarity on cleanup processes and test data management. chore: Update Jest configuration and test setup for improved timeout handling - Added a global test timeout of 30 seconds in jest.config.js. - Configured jest.setTimeout in jestSetup.js to allow individual test overrides if needed. - Enhanced test reliability by ensuring consistent timeout settings across all tests. refactor: Implement file access filtering based on agent permissions - Introduced `filterFilesByAgentAccess` function to filter files based on user access through agents. - Updated `getFiles` and `primeFiles` functions to utilize the new filtering logic. - Moved `hasAccessToFilesViaAgent` function from the File model to permission services, adjusting imports accordingly - Enhanced tests to ensure proper access control and filtering behavior for files associated with agents. fix: make support_contact field a nested object rather than a sub-document refactor: Update support_contact field initialization in agent model - Removed handling for empty support_contact object in createAgent function. - Changed default value of support_contact in agent schema to undefined. test: Add comprehensive tests for support_contact field handling and versioning refactor: remove unused avatar upload mutation field and add informational toast for success chore: add missing SidePanelProvider for AgentMarketplace and organize imports fix: resolve agent selection race condition in marketplace HandleStartChat - Set agent in localStorage before newConversation to prevent useSelectorEffects from auto-selecting previous agent fix: resolve agent dropdown showing raw ID instead of agent info from URL - Add proactive agent fetching when agent_id is present in URL parameters - Inject fetched agent into agents cache so dropdowns display proper name/avatar - Use useAgentsMap dependency to ensure proper cache initialization timing - Prevents raw agent IDs from showing in UI when visiting shared agent links Fix: Agents endpoint renamed to "My Agent" for less confusion with the Marketplace agents. chore: fix ESLint issues and Test Mocks ci: update permissions structure in loadDefaultInterface tests - Refactored permissions for MEMORY and added new permissions for MARKETPLACE and PEOPLE_PICKER. - Ensured consistent structure for permissions across different types. feat: support_contact validation to allow empty email strings |
||
|
|
52e59e40be
|
📚 feat: Add Source Citations for File Search in Agents (#8652)
* feat: Source Citations for file_search in Agents * Fix: Added citation limits and relevance score to app service. Removed duplicate tests * ✨ feat: implement Role-level toggle to optionally disable file Source Citation in Agents * 🐛 fix: update mock for librechat-data-provider to include PermissionTypes and SystemRoles --------- Co-authored-by: “Praneeth <praneeth.goparaju@slalom.com> |
||
|
|
283c9cff6f
|
ℹ️ fix: Add back Removed Icons for MCP Servers in Tools Dialog (#8636)
* Bug: Fix icons for MCP servers * Add `OPENAI_API_KEY` to `jestSetup.js` to fix tests |
||
|
|
5f2d1c5dc9
|
👁️ feat: Azure Mistral OCR Strategy (#7888)
* 👁️ feat: Add Azure Mistral OCR strategy and endpoint integration This commit introduces a new OCR strategy named 'azure_mistral_ocr', allowing the use of a Mistral OCR endpoint deployed on Azure. The configuration, schemas, and file upload strategies have been updated to support this integration, enabling seamless OCR processing via Azure-hosted Mistral services. * 🗑️ chore: Clean up .gitignore by removing commented-out uncommon directory name * chore: remove unused vars * refactor: Move createAxiosInstance to packages/api/utils and update imports - Removed the createAxiosInstance function from the config module and relocated it to a new utils module for better organization. - Updated import paths in relevant files to reflect the new location of createAxiosInstance. - Added tests for createAxiosInstance to ensure proper functionality and proxy configuration handling. * chore: move axios helpers to packages/api - Added logAxiosError function to @librechat/api for centralized error logging. - Updated imports across various files to use the new logAxiosError function. - Removed the old axios.js utility file as it is no longer needed. * chore: Update Jest moduleNameMapper for improved path resolution - Added a new mapping for '~/' to resolve module paths in Jest configuration, enhancing import handling for the project. * feat: Implement Mistral OCR API integration in TS * chore: Update MistralOCR tests based on new imports * fix: Enhance MistralOCR configuration handling and tests - Introduced helper functions for resolving configuration values from environment variables or hardcoded settings. - Updated the uploadMistralOCR and uploadAzureMistralOCR functions to utilize the new configuration resolution logic. - Improved test cases to ensure correct behavior when mixing environment variables and hardcoded values. - Mocked file upload and signed URL responses in tests to validate functionality without external dependencies. * feat: Enhance MistralOCR functionality with improved configuration and error handling - Introduced helper functions for loading authentication configuration and resolving values from environment variables. - Updated uploadMistralOCR and uploadAzureMistralOCR functions to utilize the new configuration logic. - Added utility functions for processing OCR results and creating error messages. - Improved document type determination and result aggregation for better OCR processing. * refactor: Reorganize OCR type imports in Mistral CRUD file - Moved OCRResult, OCRResultPage, and OCRImage imports to a more logical grouping for better readability and maintainability. * feat: Add file exports to API and create files index * chore: Update OCR types for enhanced structure and clarity - Redesigned OCRImage interface to include mandatory fields and improved naming conventions. - Added PageDimensions interface for better representation of page metrics. - Updated OCRResultPage to include dimensions and mandatory images array. - Refined OCRResult to include document annotation and usage information. * refactor: use TS counterpart of uploadOCR methods * ci: Update MistralOCR tests to reflect new OCR result structure * chore: Bump version of @librechat/api to 1.2.3 in package.json and package-lock.json * chore: Update CONFIG_VERSION to 1.2.8 * chore: remove unused sendEvent function from config module (now imported from '@librechat/api') * chore: remove MistralOCR service files and tests (now in '@librechat/api') * ci: update logger import in ModelService tests to use @librechat/data-schemas --------- Co-authored-by: arthurolivierfortin <arthurolivier.fortin@gmail.com> |
||
|
|
bf80cf30b3
|
*️⃣ feat: Reuse OpenID Auth Tokens (#7397)
* feat: integrate OpenID Connect support with token reuse
- Added `jwks-rsa` and `new-openid-client` dependencies for OpenID Connect functionality.
- Implemented OpenID token refresh logic in `AuthController`.
- Enhanced `LogoutController` to handle OpenID logout and session termination.
- Updated JWT authentication middleware to support OpenID token provider.
- Modified OAuth routes to accommodate OpenID authentication and token management.
- Created `setOpenIDAuthTokens` function to manage OpenID tokens in cookies.
- Upgraded OpenID strategy with user info fetching and token exchange protocol.
- Introduced `openIdJwtLogin` strategy for handling OpenID JWT tokens.
- Added caching mechanism for exchanged OpenID tokens.
- Updated configuration to include OpenID exchanged tokens cache key.
- updated .env.example to include the new env variables needed for the feature.
* fix: update return type in downloadImage documentation for clarity and fixed openIdJwtLogin env variables
* fix: update Jest configuration and tests for OpenID strategy integration
* fix: update OpenID strategy to include callback URL in setup
* fix: fix optionalJwtAuth middleware to support OpenID token reuse and improve currentUrl method in CustomOpenIDStrategy to override the dynamic host issue related to proxy (e.g. cloudfront)
* fix: fixed code formatting
* Fix: Add mocks for openid-client and passport strategy in Jest configuration to fix unit tests
* fix eslint errors: Format mock file openid-client.
* ✨ feat: Add PKCE support for OpenID and default handling in strategy setup
---------
Co-authored-by: Atef Bellaaj <slalom.bellaaj@external.daimlertruck.com>
Co-authored-by: Ruben Talstra <RubenTalstra1211@outlook.com>
|
||
|
|
353adceb0c
|
💽 fix: Exclude index page / from static cache settings (#7382)
* Disable default static caching for app's index page * Update index.html related environment variables in `.env.example` * Fix linting * Update index.spec.js --------- Co-authored-by: Danny Avila <danacordially@gmail.com> |
||
|
|
64bd373bc8
|
🔧 fix: Keyv and Proxy Issues, and More Memory Optimizations (#6867)
* chore: update @librechat/agents dependency to version 2.4.15
* refactor: Prevent memory leaks by nullifying boundModel.client in disposeClient function
* fix: use of proxy, use undici
* chore: update @librechat/agents dependency to version 2.4.16
* Revert "fix: use of proxy, use undici"
This reverts commit
|
||
|
|
37964975c1
|
🤖 refactor: Improve Agents Memory Usage, Bump Keyv, Grok 3 (#6850)
* chore: remove unused redis file * chore: bump keyv dependencies, and update related imports * refactor: Implement IoRedis client for rate limiting across middleware, as node-redis via keyv not compatible * fix: Set max listeners to expected amount * WIP: memory improvements * refactor: Simplify getAbortData assignment in createAbortController * refactor: Update getAbortData to use WeakRef for content management * WIP: memory improvements in agent chat requests * refactor: Enhance memory management with finalization registry and cleanup functions * refactor: Simplify domainParser calls by removing unnecessary request parameter * refactor: Update parameter types for action tools and agent loading functions to use minimal configs * refactor: Simplify domainParser tests by removing unnecessary request parameter * refactor: Simplify domainParser call by removing unnecessary request parameter * refactor: Enhance client disposal by nullifying additional properties to improve memory management * refactor: Improve title generation by adding abort controller and timeout handling, consolidate request cleanup * refactor: Update checkIdleConnections to skip current user when checking for idle connections if passed * refactor: Update createMCPTool to derive userId from config and handle abort signals * refactor: Introduce createTokenCounter function and update tokenCounter usage; enhance disposeClient to reset Graph values * refactor: Update getMCPManager to accept userId parameter for improved idle connection handling * refactor: Extract logToolError function for improved error handling in AgentClient * refactor: Update disposeClient to clear handlerRegistry and graphRunnable references in client.run * refactor: Extract createHandleNewToken function to streamline token handling in initializeClient * chore: bump @librechat/agents * refactor: Improve timeout handling in addTitle function for better error management * refactor: Introduce createFetch instead of using class method * refactor: Enhance client disposal and request data handling in AskController and EditController * refactor: Update import statements for AnthropicClient and OpenAIClient to use specific paths * refactor: Use WeakRef for response handling in SplitStreamHandler to prevent memory leaks * refactor: Simplify client disposal and rename getReqData to processReqData in AskController and EditController * refactor: Improve logging structure and parameter handling in OpenAIClient * refactor: Remove unused GraphEvents and improve stream event handling in AnthropicClient and OpenAIClient * refactor: Simplify client initialization in AskController and EditController * refactor: Remove unused mock functions and implement in-memory store for KeyvMongo * chore: Update dependencies in package-lock.json to latest versions * refactor: Await token usage recording in OpenAIClient to ensure proper async handling * refactor: Remove handleAbort route from multiple endpoints and enhance client disposal logic * refactor: Enhance abort controller logic by managing abortKey more effectively * refactor: Add newConversation handling in useEventHandlers for improved conversation management * fix: dropparams * refactor: Use optional chaining for safer access to request properties in BaseClient * refactor: Move client disposal and request data processing logic to cleanup module for better organization * refactor: Remove aborted request check from addTitle function for cleaner logic * feat: Add Grok 3 model pricing and update tests for new models * chore: Remove trace warnings and inspect flags from backend start script used for debugging * refactor: Replace user identifier handling with userId for consistency across controllers, use UserId in clientRegistry * refactor: Enhance client disposal logic to prevent memory leaks by clearing additional references * chore: Update @librechat/agents to version 2.4.14 in package.json and package-lock.json |
||
|
|
ded3cd8876
|
🔍 feat: Mistral OCR API / Upload Files as Text (#6274)
* refactor: move `loadAuthValues` to `~/services/Tools/credentials` * feat: add createAxiosInstance function to configure axios with proxy support * WIP: First pass mistral ocr * refactor: replace getConvoFiles with getToolFiles for improved file retrieval logic * refactor: improve document formatting in encodeAndFormat function * refactor: remove unused resendFiles parameter from buildOptions function (this option comes from the agent config) * fix: update getFiles call to include files with `text` property as well * refactor: move file handling to `initializeAgentOptions` * refactor: enhance addImageURLs method to handle OCR text and improve message formatting * refactor: update message formatting to handle OCR text in various content types * refactor: remove unused resendFiles property from compactAgentsSchema * fix: add error handling for Mistral OCR document upload and logging * refactor: integrate OCR capability into file upload options and configuration * refactor: skip processing for text source files in delete request, as they are directly tied to database * feat: add metadata field to ExtendedFile type and update PanelColumns and PanelTable components for localization and metadata handling * fix: source icon styling * wip: first pass, frontend file context agent resources * refactor: add hover card with contextual information for File Context (OCR) in FileContext component * feat: enhance file processing by integrating file retrieval for OCR resources in agent initialization * feat: implement OCR config; fix: agent resource deletion for ocr files * feat: enhance agent initialization by adding OCR capability check in resource priming * ci: fix `~/config` module mock * ci: add OCR property expectation in AppService tests * refactor: simplify OCR config loading by removing environment variable extraction, to be done when OCR is actually performed * ci: add unit test to ensure environment variable references are not parsed in OCR config * refactor: disable base64 image inclusion in OCR request * refactor: enhance OCR configuration handling by validating environment variables and providing defaults * refactor: use file stream from disk for mistral ocr api |
||
|
|
c26b54c74d
|
🔄 refactor: Consolidate Tokenizer; Fix Jest Open Handles (#5175)
* refactor: consolidate tokenizer to singleton * fix: remove legacy tokenizer code, add Tokenizer singleton tests * ci: fix jest open handles |
||
|
|
ab6fbe48f1
|
📥 feat: Import Conversations from LibreChat, ChatGPT, Chatbot UI (#2355)
* Basic implementation of ChatGPT conversation import * remove debug code * Handle citations * Fix updatedAt in import * update default model * Use job scheduler to handle import requests * import job status endpoint * Add wrapper around Agenda * Rate limits for import endpoint * rename import api path * Batch save import to mongo * Improve naming * Add documenting comments * Test for importers * Change button for importing conversations * Frontend changes * Import job status endpoint * Import endpoint response * Add translations to new phrases * Fix conversations refreshing * cleanup unused functions * set timeout for import job status polling * Add documentation * get extra spaces back * Improve error message * Fix translation files after merge * fix translation files 2 * Add zh translation for import functionality * Sync mailisearch index after import * chore: add dummy uri for jest tests, as MONGO_URI should only be real for E2E tests * docs: fix links * docs: fix conversationsImport section * fix: user role issue for librechat imports * refactor: import conversations from json - organize imports - add additional jsdocs - use multer with diskStorage to avoid loading file into memory outside of job - use filepath instead of loading data string for imports - replace console logs and some logger.info() with logger.debug - only use multer for import route * fix: undefined metadata edge case and replace ChatGtp -> ChatGpt * Refactor importChatGptConvo function to handle undefined metadata edge case and replace ChatGtp with ChatGpt * fix: chatgpt importer * feat: maintain tree relationship for librechat messages * chore: use enum * refactor: saveMessage to use single object arg, replace console logs, add userId to log message * chore: additional comment * chore: multer edge case * feat: first pass, maintain tree relationship * chore: organize * chore: remove log * ci: add heirarchy test for chatgpt * ci: test maintaining of heirarchy for librechat * wip: allow non-text content type messages * refactor: import content part object json string * refactor: more content types to format * chore: consolidate messageText formatting * docs: update on changes, bump data-provider/config versions, update readme * refactor(indexSync): singleton pattern for MeiliSearchClient * refactor: debug log after batch is done * chore: add back indexSync error handling --------- Co-authored-by: jakubmieszczak <jakub.mieszczak@zendesk.com> Co-authored-by: Danny Avila <danny@librechat.ai> |