🗺️ fix: Resolve Custom-Endpoint Providers for Summarization (#12739)

* 🔧 fix: resolve custom-endpoint providers for summarization

When `summarization.provider` in `librechat.yaml` is set to a custom-endpoint
name (e.g. `Ollama`), the string was passed verbatim to the agents SDK, which
only knows a fixed set of provider names and threw
`Unsupported LLM provider: Ollama`.

Before shaping the summarization config for the SDK, resolve the provider
through `getProviderConfig`: custom-endpoint labels are remapped to the
underlying SDK provider (e.g. `openAI`) and the endpoint's baseURL/apiKey are
injected into `parameters` so the summarization model reaches the right
backend, even when summarization targets a different custom endpoint than the
main agent.

Unknown names and names that appear with no matching endpoint flow through
unchanged so the SDK can surface a clear error. User-provided credentials and
unresolved env-var references are skipped rather than forwarded, letting the
SDK's self-summarize path reuse the agent's own clientOptions.

Ref: LibreChat Discussion #12614

* address: widen unresolved-env-var guard, fix test naming

- Reject summarization overrides when the extracted baseURL/apiKey still
  contains any `${...}` placeholder, including prefix/suffix patterns like
  `https://${UNSET}.example.com` that `envVarRegex` (exact-match) missed.
- Rename the "case-insensitive" test to reflect that only `Ollama` is
  normalized via `normalizeEndpointName`; add coverage proving other
  custom-endpoint names match case-sensitively.

* address: use req.config in responses.js; forward full endpoint options

- `responses.js` relied on a module-level `appConfig` set via `setAppConfig`,
  which is never called anywhere. Use `req.config` directly so the
  summarization provider resolver actually runs on the responses route.
- Route the custom endpoint config through `getOpenAIConfig` so summarization
  inherits the same `headers`, `defaultQuery`, `addParams`/`dropParams`, and
  `customParams` transforms (Anthropic/Google/etc.) that `initializeCustom`
  applies for the main agent flow. Strip the stale `model`/`modelName`
  defaults so `summarization.model` still wins.

* address: skip overrides when summarization matches agent endpoint

When `summarization.provider` resolves to the same custom endpoint as the
main agent, rely on the SDK's self-summarize path (which reuses
`agentContext.clientOptions` unchanged) rather than injecting overrides.
Otherwise the shallow spread of `clientOverrides.configuration` would
replace the agent's request-resolved state (dynamic headers, proxy/fetch
options) with yaml-only config.

Only applies when summarization targets a *different* endpoint from the
agent; the yaml config is all we have in that case, so overrides still
flow through.

* address: preserve raw provider when overrides cannot be built

When summarization points at a different custom endpoint than the agent
and we can't resolve the endpoint's credentials (user_provided, or a
still-unresolved `${VAR}` after env extraction), remapping to `openAI`
without overrides would silently route summaries to the default OpenAI
client. Preserve the raw provider name so the SDK raises a clear
"Unsupported LLM provider" error (now also logged, via the agents SDK
defense-in-depth fix) instead of sending traffic to the wrong backend.

* address: resolve endpoint headers and forward PROXY to summarization

- Custom-endpoint `headers` now flow through `resolveHeaders` before
  reaching `getOpenAIConfig`, matching the main agent path. This ensures
  templated values like `\${PORTKEY_API_KEY}` or `{{LIBRECHAT_BODY_...}}`
  are substituted for summarization requests instead of being forwarded
  literally.
- `PROXY` env var is now passed into `getOpenAIConfig` so cross-endpoint
  summarization honors outbound proxy dispatchers configured for the rest
  of the deployment.

* address: user summarization parameters win over endpoint defaults

Flip the merge order so `summarization.parameters` from yaml override
`clientOverrides` defaults (which come from `getOpenAIConfig` and always
include `streaming: true` etc.). A user who sets `parameters.streaming:
false` in their config should still see non-streaming summarization for
providers that require it.

* address: review feedback (logging, dead code, DRY, types, deep-merge)

- Log error in the resolveSummarizationProvider catch-all so programming
  bugs in getProviderConfig/getOpenAIConfig/resolveHeaders surface in
  operator logs instead of falling through silently.
- Drop dead `setAppConfig`/`appConfig` infrastructure in responses.js and
  fix adjacent `allowedProviders` reference that also relied on the
  never-initialized module-level appConfig. Uses `req.config` directly.
- Import canonical `normalizeEndpointName` from librechat-data-provider
  instead of duplicating it locally.
- Replace `SummarizationClientOverrides = Record<string, unknown>` with
  an explicit interface covering the known fields.
- Deep-merge `configuration` when user-supplied `summarization.parameters.
  configuration` overlaps the resolved endpoint configuration, so user
  additions (e.g. `defaultQuery`) don't wipe out `baseURL`/`defaultHeaders`.
- Wrap `process.env` mutations in test in `try/finally` so a failed
  assertion doesn't leak env state into subsequent tests.
- Drop `as unknown as AppConfig` in test helper; fixture now matches the
  `AppConfig` shape directly using a `Partial<TEndpoint>` union.
- Trim JSDoc that restated the name it was attached to.

* address: review nits — import order, local test type, conflict test

- Move `import { logger }` up into the package value-imports section so
  it no longer sits between `import type` blocks.
- Replace `as unknown as SummarizationConfig['parameters']` in the
  deep-merge test with a named `TestSummarizationParameters` type and a
  single narrowing cast at the call site, making intent explicit.
- Add a test proving that user-supplied `configuration.baseURL` wins
  over the resolved endpoint baseURL, locking in the deep-merge's
  user-wins-on-conflict semantics that the previous suite only exercised
  additively.
This commit is contained in:
Danny Avila 2026-04-20 09:00:46 -07:00 committed by GitHub
parent d2cbd551b7
commit 8e866e6010
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 708 additions and 18 deletions

View file

@ -836,6 +836,7 @@ class AgentClient extends BaseClient {
requestBody: config.configurable.requestBody,
user: createSafeUser(this.options.req?.user),
summarizationConfig: appConfig?.summarization,
appConfig,
tokenCounter,
});

View file

@ -584,6 +584,7 @@ const OpenAIChatCompletionController = async (req, res) => {
initialSummary,
runId: responseId,
summarizationConfig,
appConfig,
signal: abortController.signal,
customHandlers: handlers,
requestBody: {

View file

@ -53,17 +53,6 @@ const { getModelsConfig } = require('~/server/controllers/ModelController');
const { logViolation } = require('~/cache');
const db = require('~/models');
/** @type {import('@librechat/api').AppConfig | null} */
let appConfig = null;
/**
* Set the app config for the controller
* @param {import('@librechat/api').AppConfig} config
*/
function setAppConfig(config) {
appConfig = config;
}
/**
* Creates a tool loader function for the agent.
* @param {AbortSignal} signal - The abort signal
@ -345,7 +334,7 @@ const createResponse = async (req, res) => {
// Build allowed providers set
const allowedProviders = new Set(
appConfig?.endpoints?.[EModelEndpoint.agents]?.allowedProviders,
req.config?.endpoints?.[EModelEndpoint.agents]?.allowedProviders,
);
// Create tool loader
@ -599,6 +588,7 @@ const createResponse = async (req, res) => {
initialSummary,
runId: responseId,
summarizationConfig,
appConfig: req.config,
signal: abortController.signal,
customHandlers: handlers,
requestBody: {
@ -764,6 +754,7 @@ const createResponse = async (req, res) => {
initialSummary,
runId: responseId,
summarizationConfig,
appConfig: req.config,
signal: abortController.signal,
customHandlers: handlers,
requestBody: {
@ -1048,5 +1039,4 @@ module.exports = {
createResponse,
getResponse,
listModels,
setAppConfig,
};