mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-08-27 04:07:05 +00:00
🗺️ fix: Resolve Custom-Endpoint Providers for Summarization (#12739)
* 🔧 fix: resolve custom-endpoint providers for summarization When `summarization.provider` in `librechat.yaml` is set to a custom-endpoint name (e.g. `Ollama`), the string was passed verbatim to the agents SDK, which only knows a fixed set of provider names and threw `Unsupported LLM provider: Ollama`. Before shaping the summarization config for the SDK, resolve the provider through `getProviderConfig`: custom-endpoint labels are remapped to the underlying SDK provider (e.g. `openAI`) and the endpoint's baseURL/apiKey are injected into `parameters` so the summarization model reaches the right backend, even when summarization targets a different custom endpoint than the main agent. Unknown names and names that appear with no matching endpoint flow through unchanged so the SDK can surface a clear error. User-provided credentials and unresolved env-var references are skipped rather than forwarded, letting the SDK's self-summarize path reuse the agent's own clientOptions. Ref: LibreChat Discussion #12614 * address: widen unresolved-env-var guard, fix test naming - Reject summarization overrides when the extracted baseURL/apiKey still contains any `${...}` placeholder, including prefix/suffix patterns like `https://${UNSET}.example.com` that `envVarRegex` (exact-match) missed. - Rename the "case-insensitive" test to reflect that only `Ollama` is normalized via `normalizeEndpointName`; add coverage proving other custom-endpoint names match case-sensitively. * address: use req.config in responses.js; forward full endpoint options - `responses.js` relied on a module-level `appConfig` set via `setAppConfig`, which is never called anywhere. Use `req.config` directly so the summarization provider resolver actually runs on the responses route. - Route the custom endpoint config through `getOpenAIConfig` so summarization inherits the same `headers`, `defaultQuery`, `addParams`/`dropParams`, and `customParams` transforms (Anthropic/Google/etc.) that `initializeCustom` applies for the main agent flow. Strip the stale `model`/`modelName` defaults so `summarization.model` still wins. * address: skip overrides when summarization matches agent endpoint When `summarization.provider` resolves to the same custom endpoint as the main agent, rely on the SDK's self-summarize path (which reuses `agentContext.clientOptions` unchanged) rather than injecting overrides. Otherwise the shallow spread of `clientOverrides.configuration` would replace the agent's request-resolved state (dynamic headers, proxy/fetch options) with yaml-only config. Only applies when summarization targets a *different* endpoint from the agent; the yaml config is all we have in that case, so overrides still flow through. * address: preserve raw provider when overrides cannot be built When summarization points at a different custom endpoint than the agent and we can't resolve the endpoint's credentials (user_provided, or a still-unresolved `${VAR}` after env extraction), remapping to `openAI` without overrides would silently route summaries to the default OpenAI client. Preserve the raw provider name so the SDK raises a clear "Unsupported LLM provider" error (now also logged, via the agents SDK defense-in-depth fix) instead of sending traffic to the wrong backend. * address: resolve endpoint headers and forward PROXY to summarization - Custom-endpoint `headers` now flow through `resolveHeaders` before reaching `getOpenAIConfig`, matching the main agent path. This ensures templated values like `\${PORTKEY_API_KEY}` or `{{LIBRECHAT_BODY_...}}` are substituted for summarization requests instead of being forwarded literally. - `PROXY` env var is now passed into `getOpenAIConfig` so cross-endpoint summarization honors outbound proxy dispatchers configured for the rest of the deployment. * address: user summarization parameters win over endpoint defaults Flip the merge order so `summarization.parameters` from yaml override `clientOverrides` defaults (which come from `getOpenAIConfig` and always include `streaming: true` etc.). A user who sets `parameters.streaming: false` in their config should still see non-streaming summarization for providers that require it. * address: review feedback (logging, dead code, DRY, types, deep-merge) - Log error in the resolveSummarizationProvider catch-all so programming bugs in getProviderConfig/getOpenAIConfig/resolveHeaders surface in operator logs instead of falling through silently. - Drop dead `setAppConfig`/`appConfig` infrastructure in responses.js and fix adjacent `allowedProviders` reference that also relied on the never-initialized module-level appConfig. Uses `req.config` directly. - Import canonical `normalizeEndpointName` from librechat-data-provider instead of duplicating it locally. - Replace `SummarizationClientOverrides = Record<string, unknown>` with an explicit interface covering the known fields. - Deep-merge `configuration` when user-supplied `summarization.parameters. configuration` overlaps the resolved endpoint configuration, so user additions (e.g. `defaultQuery`) don't wipe out `baseURL`/`defaultHeaders`. - Wrap `process.env` mutations in test in `try/finally` so a failed assertion doesn't leak env state into subsequent tests. - Drop `as unknown as AppConfig` in test helper; fixture now matches the `AppConfig` shape directly using a `Partial<TEndpoint>` union. - Trim JSDoc that restated the name it was attached to. * address: review nits — import order, local test type, conflict test - Move `import { logger }` up into the package value-imports section so it no longer sits between `import type` blocks. - Replace `as unknown as SummarizationConfig['parameters']` in the deep-merge test with a named `TestSummarizationParameters` type and a single narrowing cast at the call site, making intent explicit. - Add a test proving that user-supplied `configuration.baseURL` wins over the resolved endpoint baseURL, locking in the deep-merge's user-wins-on-conflict semantics that the previous suite only exercised additively.
This commit is contained in:
parent
d2cbd551b7
commit
8e866e6010
5 changed files with 708 additions and 18 deletions
|
|
@ -836,6 +836,7 @@ class AgentClient extends BaseClient {
|
|||
requestBody: config.configurable.requestBody,
|
||||
user: createSafeUser(this.options.req?.user),
|
||||
summarizationConfig: appConfig?.summarization,
|
||||
appConfig,
|
||||
tokenCounter,
|
||||
});
|
||||
|
||||
|
|
|
|||
|
|
@ -584,6 +584,7 @@ const OpenAIChatCompletionController = async (req, res) => {
|
|||
initialSummary,
|
||||
runId: responseId,
|
||||
summarizationConfig,
|
||||
appConfig,
|
||||
signal: abortController.signal,
|
||||
customHandlers: handlers,
|
||||
requestBody: {
|
||||
|
|
|
|||
|
|
@ -53,17 +53,6 @@ const { getModelsConfig } = require('~/server/controllers/ModelController');
|
|||
const { logViolation } = require('~/cache');
|
||||
const db = require('~/models');
|
||||
|
||||
/** @type {import('@librechat/api').AppConfig | null} */
|
||||
let appConfig = null;
|
||||
|
||||
/**
|
||||
* Set the app config for the controller
|
||||
* @param {import('@librechat/api').AppConfig} config
|
||||
*/
|
||||
function setAppConfig(config) {
|
||||
appConfig = config;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a tool loader function for the agent.
|
||||
* @param {AbortSignal} signal - The abort signal
|
||||
|
|
@ -345,7 +334,7 @@ const createResponse = async (req, res) => {
|
|||
|
||||
// Build allowed providers set
|
||||
const allowedProviders = new Set(
|
||||
appConfig?.endpoints?.[EModelEndpoint.agents]?.allowedProviders,
|
||||
req.config?.endpoints?.[EModelEndpoint.agents]?.allowedProviders,
|
||||
);
|
||||
|
||||
// Create tool loader
|
||||
|
|
@ -599,6 +588,7 @@ const createResponse = async (req, res) => {
|
|||
initialSummary,
|
||||
runId: responseId,
|
||||
summarizationConfig,
|
||||
appConfig: req.config,
|
||||
signal: abortController.signal,
|
||||
customHandlers: handlers,
|
||||
requestBody: {
|
||||
|
|
@ -764,6 +754,7 @@ const createResponse = async (req, res) => {
|
|||
initialSummary,
|
||||
runId: responseId,
|
||||
summarizationConfig,
|
||||
appConfig: req.config,
|
||||
signal: abortController.signal,
|
||||
customHandlers: handlers,
|
||||
requestBody: {
|
||||
|
|
@ -1048,5 +1039,4 @@ module.exports = {
|
|||
createResponse,
|
||||
getResponse,
|
||||
listModels,
|
||||
setAppConfig,
|
||||
};
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue