LibreChat/api/server/services/Files
Danny Avila 2f8bc3dfcd
🎞️ fix: Deliver Opted-In Custom Endpoint Media Safely (#15937)
* feat: send video/audio to custom endpoints that opt in via supportedMimeTypes

The video and audio encoders only emitted OpenAI-format parts (video_url,
input_audio) for OpenRouter, and Google's media part for Google/Vertex. For
every other provider the file was validated and then silently dropped, so a
custom OpenAI-compatible endpoint (vLLM, LiteLLM, ...) never received the
attachment even though the wire format is identical to OpenRouter's.

Emit the OpenAI-format part for OpenAI-like providers when the endpoint's
fileConfig supportedMimeTypes explicitly allows the file's type. The inherited
default list does not count as opting in (isExplicitMimeConfig, keyed on
referential identity like the client's picker), so endpoints whose gateway
cannot handle media are unaffected.

* feat: let a configured custom endpoint pick and attach video/audio

The attach menu's Upload to Provider filter and the drag-drop viability check
only opened video/audio for Google and OpenRouter, or for a custom endpoint
with a fully permissive (.*) supportedMimeTypes. Honor any admin-configured
allowlist on a custom endpoint instead, so a finite list that includes
video/.* or audio/.* is enough, matching the server-side encoders.

* fix: canonicalize input_audio format from MIME type

The `input_audio.format` value was derived from the filename extension via
`filename.split('.').pop()`, which produces values providers reject:

- `clip.wave` (audio/wave) emitted `wave`, but the accepted format is `wav`
- a file with no extension (`recording`) emitted the whole filename, since
  splitting a dotless string yields a single-element array, so the existing
  non-empty guard never fired
- `clip.mpeg` (audio/mpeg) emitted `mpeg` rather than `mp3`

All of these pass MIME validation and then fail at the provider. Derive the
format from the MIME type instead, mirroring the canonicalization already in
STTService, and fall back to the filename extension only when it is itself a
supported format. Throws when neither source yields one rather than sending an
unsupported value.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: route opted-in custom endpoint media to the provider path

In unified upload mode an upload carries no tool resource, so its delivery path is
inferred by resolveDefaultLLMDeliveryPath. That resolver judged audio and video for any
named endpoint with isMediaSupportedProvider, which lists only Google, Vertex and
OpenRouter, so a custom endpoint that opted into media through supportedMimeTypes still
recorded video as none and audio as text, and the encoder branch was never reached.

The upload resolver already holds the endpoint's file config, so it now passes the
endpoint's supportedMimeTypes through, and an explicit match on an OpenAI-compatible or
custom endpoint counts as provider-capable, mirroring isConfiguredProviderMediaType on
the encoder side. The inherited default list is still not an opt-in, and known providers
whose encoders emit nothing for media are unchanged. BaseClient re-resolves per turn with
the same function, so the turn path follows.

* fix: scope the media opt-in to custom endpoints that run as OpenAI

The opt-in applied to the built-in openAI and azureOpenAI endpoints too, but the picker
and drag-drop only open media for custom endpoints, so that was a route the client could
not send to. It is now limited to endpoints that are not a known provider.

A custom endpoint may also declare provider: anthropic, in which case initialization runs
it as Anthropic and the encoders emit no media part for it. getCustomEndpointProvider
reads that declaration off the app config, the upload callers pass it as endpointProvider,
the agent encoder and BaseClient pass the agent's resolved provider, and an endpoint known
to run as something other than OpenAI keeps its previous route: video stays off the model
path and audio still reaches transcription.

* fix: reject unsupported provider audio during upload

---------

Co-authored-by: Renato Garita Figueiredo <renato.garitafigueiredo@uni-osnabrueck.de>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 16:38:51 -04:00
..
Audio 🚚 feat: Route Attachments and Provision Tools Lazily (#15763) 2026-09-08 23:28:52 -04:00
Azure 🔣 fix: Encode Images From Storage Keys (#15851) 2026-09-12 12:39:26 -04:00
Citations 🛤️ fix: Align File Search Citation Paths (#15650) 2026-09-06 17:25:47 -04:00
Code 🦘 feat: Share Run-Scoped Files With Subagents (#15848) 2026-09-12 20:14:22 -04:00
Firebase 🔣 fix: Encode Images From Storage Keys (#15851) 2026-09-12 12:39:26 -04:00
images 🔣 fix: Encode Images From Storage Keys (#15851) 2026-09-12 12:39:26 -04:00
Local 🪣 refactor: Rate-Limit Token Routes and Cap Remote File Downloads (#13978) 2026-06-26 12:19:03 -04:00
OpenAI
VectorDB 🚚 feat: Route Attachments and Provision Tools Lazily (#15763) 2026-09-08 23:28:52 -04:00
index.js
permissions.js 🧱 fix: Enforce Agent Runtime File Trust Boundaries (#14577) 2026-08-02 14:18:28 -04:00
permissions.spec.js 🧱 fix: Enforce Agent Runtime File Trust Boundaries (#14577) 2026-08-02 14:18:28 -04:00
process.integration.spec.js 🧹 fix: Clean Up Orphaned Agent File Stubs After Deletion (#12781) 2026-04-22 11:35:48 -07:00
process.js 🎞️ fix: Deliver Opted-In Custom Endpoint Media Safely (#15937) 2026-09-14 16:38:51 -04:00
process.spec.js 👑 fix: Elect a Single Expired-File Sweeper (#15909) 2026-09-14 02:05:52 -04:00
provision.js 🚚 feat: Route Attachments and Provision Tools Lazily (#15763) 2026-09-08 23:28:52 -04:00
provisionCallback.js 🚚 feat: Route Attachments and Provision Tools Lazily (#15763) 2026-09-08 23:28:52 -04:00
retention.js ⏳ fix: Separate Temporary and General Chat Retention (#15734) 2026-09-08 11:22:54 -04:00
retention.spec.js ⏳ fix: Separate Temporary and General Chat Retention (#15734) 2026-09-08 11:22:54 -04:00
routing.js 🚚 feat: Route Attachments and Provision Tools Lazily (#15763) 2026-09-08 23:28:52 -04:00
saveBase64Image.spec.js 🏷️ fix: Record the Media Type of Saved Images, Not the Declared One (#15606) 2026-09-04 11:51:28 -04:00
strategies.js 🗄️ refactor: Honor All-Data Retention for Agent Files (#13424) 2026-05-30 22:32:22 -04:00