Commit graph

15 commits

Author SHA1 Message Date
Danny Avila
f13b0eaef4
🚚 chore: Point Repository and Image References at LibreChat-AI (#16005)
Some checks failed
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Docker Dev Images Build / publish (push) Waiting to run
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
Publish `@librechat/client` to NPM / pack (push) Has been cancelled
Publish `librechat-data-provider` to NPM / pack (push) Has been cancelled
Publish `@librechat/data-schemas` to NPM / pack (push) Has been cancelled
Publish `@librechat/client` to NPM / publish-npm (push) Has been cancelled
Publish `librechat-data-provider` to NPM / publish-npm (push) Has been cancelled
Publish `@librechat/data-schemas` to NPM / publish-npm (push) Has been cancelled
Update package repository and bugs URLs, GitHub links in docs, templates and comments, and image references (compose files, deployed-update, Helm values and renovate hints) from danny-avila to LibreChat-AI / librechat-ai. Covers the rag-api repository and its images, which moved to the organization. Sponsor links and the sealed star-history chart are unchanged.
2026-09-23 23:08:36 -04:00
Ravi Kumar L
c7665ab1ae
🗾 fix: Route Cross-Region Attachments Through CloudFront (#16082)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
* fix(files): read cross-region CloudFront attachments

* fix(files): propagate CloudFront stream failures and cancellation
2026-09-18 16:29:52 -04:00
lia-by-librechat[bot]
be54587d14
🪒 fix: Strip Cache-Bust Suffixes From Every Local File Read (#15991)
* 🧼 fix: Strip Cache-Bust Query Before Local Vision Encode

Reused code-interpreter images persist filepath with a ?v= suffix.
prepareImagesLocal now strips it before disk encode, matching crud/share.

Co-authored-by: Zsanz3 <Zsanz3@users.noreply.github.com>

* 🧪 test: Isolate prepareImagesLocal query-strip coverage

Mock sharp and resize so the local vision-encode test does not load
the rest of the file-strategy graph.

Co-authored-by: Zsanz3 <Zsanz3@users.noreply.github.com>

* 🪒 fix: Strip Cache-Bust Suffixes From Every Local File Read

Reused code-interpreter outputs persist a `?v=<timestamp>` suffix on the file
document's filepath, and local storage resolves that field into a filesystem path.
prepareImagesLocal already handled it after the cherry-picked commits; getLocalFileStream
did not, so every download-stream consumer (the download route, provisioning, agent and
skill file reads) still hit ENOENT for a regenerated image.

Move the strip into one documented helper and call it from all three local read paths.

* 🧱 refactor: Move Cache-Bust Stripping Into the Storage Package

`/api` holds wiring, not behavior, so `stripCacheBust` moves out of a new CJS helper and
into `packages/api/src/storage/path.ts`, beside `resolveDownloadPath` — the function that
already decides what a stored filepath means to a download. Its doc comment now states why
the two differ: a remote strategy's query string can carry a presigned signature, so only
the local paths may strip.

The local storage modules keep just the call into it. Semantics are covered by the storage
package's own test; the `/api` specs pin the wiring.

Addresses the Codex P1 finding on 5f8dbf3cf.

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Zsanz3 <Zsanz3@users.noreply.github.com>
Co-authored-by: Lia <lia@librechat.ai>
2026-09-16 08:28:32 -04:00
Hao
bd08a363c4
🔣 fix: Encode Images From Storage Keys (#15851)
* fix: Prefer storage keys for image encoding

* fix: Sanitize assistant image errors

* fix: Sanitize agent image storage errors

* fix: Move blob image encoding into API package

* fix: Complete image encoder TypeScript migration

* fix: Keep request-boundary error logs diagnosable

getSafeErrorMetadata reduces an error to { type: 'Error' } and an optional
status. That is the right bound where an error may echo submitted content, and
it stays in place at getS3FileStream, getAzureFileStream and
getFirebaseFileStream, where a signed URL reaches the log.

Applied at ResumableAgentController's initialization catch and the chatV1
generic branch it blinded the whole route: a missing key, an agent that fails to
load and a database error all logged as { type: 'Error' }, with no message,
name, code or stack.

getSafeErrorText keeps the error's own description and stack, reduces every URL
to its origin so an object path and signature cannot ride along, and strips
bearer credentials. It returns text so the two handlers log it inside the
message and pass no winston metadata, which is what stops an SDK error's own
enumerable properties (request.url, config, response.headers) from being
promoted onto the record.

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-09-12 12:39:26 -04:00
Danny Avila
ec15075dc8
🚚 feat: Route Attachments and Provision Tools Lazily (#15763) 2026-09-08 23:28:52 -04:00
Danny Avila
58de77619e
🗝️ refactor: Prefer the Recorded Storage Key for Every Download Stream (#15695)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
S3 and CloudFront records have carried `storageKey` since #12987, yet six
readers still handed `file.filepath` to `getDownloadStream` and re-derived
the key by parsing a presigned or CDN URL, while four others had grown
their own `storageKey || filepath` expression. One resolver now serves all
ten: `resolveDownloadPath` returns the recorded key when present and the
path otherwise, so records without a key (local, Firebase, Azure, code
output) behave exactly as before, and the share route keeps its local-only
query-string strip on top.

`resolveStoredS3Key` reuses the same `StoredFileRef` type. Covered by unit
cases for the resolver and an S3 case proving a record with a key streams
correctly even when its stored URL no longer parses.

Closes #15693
2026-09-07 01:28:35 -04:00
Sigurður Guðbrandsson
ea3c61d69a
🪣 fix: Decode S3 Object Keys from URLs (#15426)
* 🔤 fix: Decode percent-encoded S3 keys so non-ASCII filenames are readable

extractKeyFromS3Url returns `URL.pathname` as the S3 object key without
decoding it. The AWS SDK percent-encodes the Key again when it signs the
request, so a file stored under `Ársreikningur.pdf` is fetched as
`%C3%81rsreikningur.pdf` and every read fails with NoSuchKey. ASCII keys are
byte-identical either way, which is why this only appears for non-English
filenames.

Decode keys derived from a URL path in all three branches (path-style
endpoint, bucket-in-path, virtual-hosted). A key passed in raw — not a URL —
still returns untouched, and a malformed escape sequence falls back to the
raw value with a warning rather than throwing.

* 🔗 fix: Percent-encode CloudFront URL keys so both URL forms agree

buildCloudFrontUrl interpolated the raw S3 key into the URL while SDK-generated
S3 URLs carry an encoded one, so the two forms of `file.filepath` disagreed
about what a `%` means. `assertS3FileName` permits `%`, so a key containing the
literal text `report%20final.pdf` produced a CloudFront URL indistinguishable
from one for a key containing a space — and decoding on extraction would then
target the wrong object on read, re-sign, and delete.

Encoding each path segment here (separators stay literal) makes both producers
consistent, which is what lets extractKeyFromS3Url decode unconditionally.

* style: sort imports in cloudfront/crud.ts (pre-existing drift)

The changed-file import-sort gate flags this file; the drift predates this PR
(the untouched upstream version fails the same check). Kept as its own commit
so it does not obscure the fix.

* 🔏 fix: Encode the CloudFront Invalidation Path Like the Viewer URL

`buildCloudFrontUrl` now percent-encodes each key segment, so the cached
viewer path for a key with a literal `%` or a non-ASCII character is the
encoded form. `deleteFileFromCloudFront` still handed the raw key to
`CreateInvalidationCommand`, so the invalidation no longer matched the
cached object and deleted content stayed served until the entry expired.
Both producers now share one `encodeKeyPath` helper.

Also asserts that `getS3FileStream` sends the decoded key to
`GetObjectCommand`, which is the call that actually failed with
`NoSuchKey`, rather than only checking the extractor.

Co-authored-by: dinershtein <228485+dinershtein@users.noreply.github.com>

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
Co-authored-by: dinershtein <228485+dinershtein@users.noreply.github.com>
2026-09-07 00:23:36 -04:00
Danny Avila
ad0f72dede
🌀 ci: Deterministic Circular Dependency Checks (#14579)
* 🌀 ci: Deterministic Circular Dependency Checks

* 🌀 ci: Enforce Type-Level Edges in Circular Dependency Scan

* 🌀 ci: Materialize Import-Type Expression Edges in Cycle Scan

* 🌀 ci: Collect Inline Type-Only Specifier Edges in Cycle Scan
2026-08-01 14:43:26 -04:00
Danny Avila
b15d40e3e4
🪣 refactor: Rate-Limit Token Routes and Cap Remote File Downloads (#13978)
* harden token and remote file handling

* sort s3 storage imports

* split token submission rate limits
2026-06-26 12:19:03 -04:00
Danny Avila
6bc75d24c8
⚡️ refactor: Migrate @librechat/api build to tsdown (#13595)
* ⚡️ refactor: Migrate @librechat/api build to tsdown

Replace Rollup with tsdown (rolldown + oxc isolated-declarations) for the
@librechat/api package build, mirroring the merged data-schemas migration.

- Add tsdown.config.mjs (cjs output, oxc dts, externalize all bare deps,
  bundle first-party `~/` + relative imports)
- Annotate exports for isolatedDeclarations (codefix-driven). Collapse the
  tokens.ts model->token maps to Record<string, Record<string, number>> and
  switch validation.ts's runtime `files` field from z.any() to z.unknown()
  so no explicit `any` is introduced
- Repoint package.json main/types/exports to tsdown's .cjs/.d.cts output
- Add src/telemetry.ts entry shim so the two index.ts entries don't collide
  in oxc's flat dts output (stable dist/telemetry.{cjs,d.cts})
- Delete rollup.config.js

Build time ~36s -> ~0.5s. No runtime behavior change: 5712 unit tests pass,
both entries load via require(), legacy /api consumes them unchanged.

* 👷 ci: Hash packages/api/tsdown.config.mjs in build-api cache keys

The build-api cache keys hashed `packages/api/server-rollup.config.js`,
which never existed (api used `rollup.config.js`, now removed) — a copy-paste
artifact from the data-provider key that matched no file. Replace it with the
new `packages/api/tsdown.config.mjs` so edits to the build config (entry,
format, externals) bust the api build cache, matching the data-schemas key.
2026-06-08 10:54:48 -04:00
Ravi Kumar L
05d4e90f91
🌩️ feat: Strict CloudFront signed cookie enforcement via requireSignedAccess (#13078)
* feat(cloudfront): add requireSignedAccess to enforce strict signed access

Introduces cloudfront.requireSignedAccess (default false). When enabled,
initializeCloudFront requires both CLOUDFRONT_KEY_PAIR_ID and
CLOUDFRONT_PRIVATE_KEY, rejects the unimplemented imageSigning="url"
mode, and initializeFileStorage throws to block startup on any
CloudFront init failure. OSS path is unchanged: missing keys still
log-and-continue when requireSignedAccess is false.

Adds low-noise startup and cookie-issuance logs without leaking signed
URLs, policies, signatures, private keys, or cookie values.

* fix(cloudfront): reject requireSignedAccess unless imageSigning is "cookies"

Previously requireSignedAccess=true was accepted with imageSigning="none"
or "url", but setCloudFrontCookies() only runs for "cookies" — leaving
strict mode toothless: CloudFront stayed publicly accessible, or image
delivery broke on a distribution that actually requires signed access.

Adds a Zod refinement plus a runtime guard in initializeCloudFront so
the only currently-functional strict configuration is imageSigning
"cookies". Signed URL mode can lift this restriction once implemented.

* fix(cloudfront): resolve strict access type checks

* chore(cloudfront): reduce strict startup log noise

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-05-11 23:30:01 -04:00
Danny Avila
1bc2692a15
🌥️ feat: Add Optional Region-aware S3/CloudFront Storage Keys (#12987)
* feat(files): add optional region-aware storage keys

* test(files): fix region storage CI fixtures

* feat(files): finalize inline CloudFront asset namespaces

* fix(files): allow wildcard region CloudFront cookies

* fix(files): preserve legacy storage key compatibility

* fix(files): align CloudFront clear cookie cleanup

* fix(files): clear legacy CloudFront cookie scopes

* chore(files): clean up storage review nits

* fix(files): keep inline namespaces CloudFront-only
2026-05-06 23:16:56 -04:00
Danny Avila
9c81792d25
🔐 feat: Add Signed CloudFront File Downloads (#12970)
* feat: add signed CloudFront downloads

* fix: preserve local IdP avatar paths

* fix: address signed download review findings

* fix: harden CloudFront cookie scope validation

* fix: preserve URL save API compatibility

* fix: store CDN SSO avatars under shared prefix

* fix: Harden CloudFront tenant file access

* fix: Preserve CloudFront download compatibility

* fix: Address CloudFront review follow-ups

* fix: Preserve file URL fallback user paths

* fix: Address download review hardening

* fix: Use file owner for S3 RAG cleanup

* fix: Address final download review nits

* fix: Clear stale avatar CloudFront cookies

* fix: Align download filename helpers with dev

* fix: Address final CloudFront review follow-ups

* fix: Stream S3 URL uploads

* fix: Set S3 stream upload length

* fix: Preserve download metadata filepath

* fix: Avoid remote content length for stream uploads

* fix: Use bounded multipart URL uploads

* fix: Harden S3 filename boundaries
2026-05-06 19:48:30 -04:00
Atef Bellaaj
187ab787da
🌩️ feat: CloudFront CDN File Strategy (#12193)
* 🌩️ feat: CloudFront CDN File Strategy + signed cookies

Squashed from PR #12193:
- feat(storage): add CloudFront CDN file strategy
- feat(auth): add CloudFront signed cookie support

Note: package.json/package-lock.json dependency additions are intentionally
omitted from this commit and will be re-added via `npm install` after rebase
to avoid lock-file merge conflicts. The two new peer deps that need to be
re-installed are:
  - @aws-sdk/client-cloudfront@^3.1032.0
  - @aws-sdk/cloudfront-signer@^3.1012.0

Also fixes 4 missing destructured names in AuthService.spec.js
(getUserById, generateToken, generateRefreshToken, createSession) that
were referenced in tests but not imported from the mocked '~/models'.

* 📦 chore: install CloudFront SDK deps for PR #12193

Adds the two AWS CloudFront packages required by the rebased
CloudFront CDN strategy:
  - @aws-sdk/client-cloudfront
  - @aws-sdk/cloudfront-signer

Following the @aws-sdk/client-s3 pattern:
  - api/package.json: regular dependency (runtime resolution)
  - packages/api/package.json: peerDependency

Generated by `npm install` against the freshly rebased lock file
to avoid the merge conflicts that came from the original PR's
lock-file edits being made against an older base of dev.

* 🐛 fix: CI failures + review findings on CloudFront PR #12193

CI fixes
- Rename packages/data-provider/src/__tests__/cloudfront-config.test.ts
  → src/cloudfront-config.spec.ts. Jest's default testMatch picks up
  __tests__/ directories even inside dist/, so the compiled .d.ts shell
  was being executed as an empty test suite. Moving to .spec.ts (matching
  the rest of the package) avoids the dist/ pickup.
- Add cookieExpiry: 1800 to CloudFront crud.test makeConfig: the schema
  applies a default so CloudFrontFullConfig requires it.

Review findings addressed
- #1 (Codex + comprehensive): Normalize CloudFront domain with /\/+$/
  regex (and key with /^\/+/ regex) in buildCloudFrontUrl, matching the
  cookie code so resource policy and file URLs stay aligned even when
  the configured domain has multiple trailing slashes. Added tests.
- #2: Move DEFAULT_BASE_PATH out of s3Config into shared
  packages/api/src/storage/constants.ts. ImageService no longer imports
  S3-specific config.
- #3: getCloudFrontConfig() returns Readonly<CloudFrontFullConfig> | null
  to discourage mutation of the cached signing config.
- #4: Add cross-field refinement tests for cloudfrontConfigSchema
  (invalidateOnDelete-without-distributionId,
  imageSigning="cookies"-without-cookieDomain).
- #6: Revert unrelated MCP comment re-indentation in
  librechat.example.yaml.
- #7: Add azure_blob to the strategy list comment.

Skipped
- #5 (extractKeyFromS3Url with CloudFront URLs): existing
  deleteFileFromCloudFront tests already cover the path-equivalence
  assumption; renaming the helper is real refactor work beyond this
  PR's scope.
- #8, #9 (NIT, low confidence): leaving for author judgement.

* 🧹 chore: drop dead DEFAULT_BASE_PATH from s3Config test mock

After moving DEFAULT_BASE_PATH to ~/storage/constants, crud.ts no longer
reads it from s3Config — so the entry in the s3Config jest mock was
misleading dead config. The tests still pass because the unmocked real
constants module provides the value.

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-05-05 13:21:05 -04:00
Atef Bellaaj
a0fed6173c
🗂️ refactor: Migrate S3 Storage to TypeScript in packages/api (#11947)
* Migrate S3 storage module with unit and integration tests

  - Migrate S3 CRUD and image operations to packages/api/src/storage/s3/
  - Add S3ImageService class with dependency injection
  - Add unit tests using aws-sdk-client-mock
  - Add integration tests with real s3 bucket (condition presence of  AWS_TEST_BUCKET_NAME)

* AI Review Findings Fixes

* chore: tests and refactor S3 storage types

- Added mock implementations for the 'sharp' library in various test files to improve image processing testing.
- Updated type references in S3 storage files from MongoFile to TFile for consistency and type safety.
- Refactored S3 CRUD operations to ensure proper handling of file types and improve code clarity.
- Enhanced integration tests to validate S3 file operations and error handling more effectively.

* chore: rename test file

* Remove duplicate import of refreshS3Url

* chore: imports order

* fix: remove duplicate imports for S3 URL handling in UserController

* fix: remove duplicate import of refreshS3FileUrls in files.js

* test: Add mock implementations for 'sharp' and '@librechat/api' in UserController tests

- Introduced mock functions for the 'sharp' library to facilitate image processing tests, including metadata retrieval and buffer conversion.
- Enhanced mocking for '@librechat/api' to ensure consistent behavior in tests, particularly for the needsRefresh and getNewS3URL functions.

---------

Co-authored-by: Danny Avila <danny@librechat.ai>
2026-03-21 14:28:55 -04:00