mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-08-27 04:07:05 +00:00
📨 feat: Custom Headers on Built-in Provider Endpoints (#13742)
* 📨 feat: Custom Headers on Built-in Provider Endpoints Add a `headers` config option to the built-in `openAI`, `anthropic`, and `google` endpoints (incl. Anthropic/Google Vertex), mirroring the custom endpoint header mechanism. Values support the same placeholder resolution (env vars, `{{LIBRECHAT_USER_*}}`, `{{LIBRECHAT_BODY_CONVERSATIONID}}`) and are resolved at request time so dynamic values like conversationId resolve against the live request — without losing provider-native request shaping. Closes #13082. Covers #13713: forwarding conversationId to a reverse proxy is now `X-Conversation-Id: '{{LIBRECHAT_BODY_CONVERSATIONID}}'` — an unknown header is ignored by the native Anthropic API, so no 400 and no metadata gating needed. - Schema: `headers` on `baseEndpointSchema` (openAI/google/anthropic/all). - New `mergeHeaders`/`resolveConfigHeaders` utils centralize the per-provider header locations (`configuration.defaultHeaders`, Anthropic `clientOptions.defaultHeaders`, Google `customHeaders`); provider-managed headers (auth, `anthropic-beta`) always win on collision. - Each initializer threads configured headers (endpoint over `all`) into the right place; request-time resolution runs across all locations in the main and title flows. * 🩹 fix: Cast endpoints.all to TEndpoint for headers DeepPartial widening Adding `headers` (a Record) to `baseEndpointSchema` makes `DeepPartial<TCustomConfig>` widen its value type to `string | undefined`, which is not assignable to the concrete `TEndpoint['headers']: Record<string, string>` at the `loadedEndpoints.all` assignment. Cast at the assignment site, mirroring the existing `anthropicConfig as TAnthropicEndpoint` cast in the same function. * 🛡️ fix: Harden built-in endpoint custom headers (Codex review) Address Codex P2 findings on the custom-headers feature: - Anthropic title requests: `omitTitleOptions` strips the `clientOptions` carrier, which dropped its `defaultHeaders`. Preserve just the header carrier so gateway/reverse-proxy metadata still reaches title generation. - mergeHeaders: match header names case-insensitively so an override (e.g. a provider-managed `Authorization`/`anthropic-beta`) replaces/uniones a case-variant from the base instead of emitting two names a client may collapse. - OpenAI: withhold admin-configured headers when the user supplies the base URL (`user_provided`), since values may carry `${SECRET}`/token placeholders that must not reach a user-controlled endpoint — mirrors the custom-endpoint guard. - Azure: honor global `endpoints.all` headers (same OpenAI carrier) while keeping Azure-managed `api-key`/version headers authoritative. Adds tests for each. * 🔐 fix: Resolve-once + provider-managed header safety (Codex review round 2) Address Codex P2 findings: - Azure: keep global `endpoints.all` headers unresolved at init and let request-time `resolveConfigHeaders` resolve them once, avoiding a second-order env expansion of already-substituted user values. - Google: `resolveConfigHeaders` no longer template-resolves the provider-managed `Authorization` header (built from a possibly user-provided key), so a user key like `${ENV}` can't leak server environment values. - Model fetches: thread configured headers (endpoint over `all`) + user object through `getOpenAIModels`/`getAnthropicModels` → `fetchModels`, so a gateway-fronted built-in provider receives the header on `/models` too. Fixed `fetchModels` to merge custom headers for Anthropic instead of overwriting them (managed `x-api-key`/version still win). Adds/updates tests for each. * 🧯 fix: Header provenance, memory/title coverage, idempotency (Codex round 3) Address Codex P2 findings, including two regressions from the prior round: - Google auth (findings 6 & 8): move native Google header resolution to init (`initializeGoogle`), resolving admin templates BEFORE the key-derived auth header is built. resolveConfigHeaders no longer touches Google `customHeaders`, so admin `Authorization` templates resolve again (fixes the round-2 regression) while the SDK auth header (possibly a user-provided key) is never env-expanded. - Memory runs: memory extraction now calls `resolveConfigHeaders`, so native Anthropic (and OpenAI) headers resolve for memory requests too. - Vertex titles: restore the ORIGINAL `clientOptions` object reference (not a copy) when preserving headers across `omitTitleOptions`, so the Vertex `createClient` closure and the resolved headers stay on the same object. - Reuse: `resolveConfigHeaders` is now idempotent (resolve-once per header map), preventing a second pass from env-expanding values already substituted with user/body data when an agent object flows through buildAgentInput twice. Adds/updates tests for each.
This commit is contained in:
Danny Avila
GitHub
parent
7c071e244b
commit
2350ebb24a
26 changed files with 950 additions and 76 deletions
|
|
@ -9,9 +9,9 @@ const {
|
|||
logToolError,
|
||||
sanitizeTitle,
|
||||
payloadParser,
|
||||
resolveHeaders,
|
||||
createSafeUser,
|
||||
initializeAgent,
|
||||
resolveConfigHeaders,
|
||||
countTokens,
|
||||
getBalanceConfig,
|
||||
omitTitleOptions,
|
||||
|
|
@ -1624,12 +1624,25 @@ class AgentClient extends BaseClient {
|
|||
delete clientOptions.modelKwargs.max_output_tokens;
|
||||
}
|
||||
|
||||
/** `omitTitleOptions` drops the Anthropic `clientOptions` carrier (thinking,
|
||||
* streaming, etc.), which would also drop its `defaultHeaders` — preserve the
|
||||
* original `clientOptions` object so gateway/reverse-proxy metadata still
|
||||
* reaches title requests (the proxy may require it for auth/routing). Restore
|
||||
* the SAME object reference, not a copy: the Vertex `createClient` closure from
|
||||
* `getLLMConfig` closes over this object, so `resolveConfigHeaders` must mutate
|
||||
* the very object the client is built from. */
|
||||
const anthropicClientOptions = clientOptions?.clientOptions;
|
||||
|
||||
clientOptions = Object.assign(
|
||||
Object.fromEntries(
|
||||
Object.entries(clientOptions).filter(([key]) => !omitTitleOptions.has(key)),
|
||||
),
|
||||
);
|
||||
|
||||
if (anthropicClientOptions?.defaultHeaders != null && clientOptions.clientOptions == null) {
|
||||
clientOptions.clientOptions = anthropicClientOptions;
|
||||
}
|
||||
|
||||
if (
|
||||
provider === Providers.GOOGLE &&
|
||||
(endpointConfig?.titleMethod === TitleMethod.FUNCTIONS ||
|
||||
|
|
@ -1638,20 +1651,19 @@ class AgentClient extends BaseClient {
|
|||
clientOptions.json = true;
|
||||
}
|
||||
|
||||
/** Resolve request-based headers for Custom Endpoints. Note: if this is added to
|
||||
* non-custom endpoints, needs consideration of varying provider header configs.
|
||||
/** Resolve request-based headers across provider-specific header locations:
|
||||
* OpenAI `configuration.defaultHeaders`, Anthropic `clientOptions.defaultHeaders`
|
||||
* (preserved above), and Google `customHeaders`.
|
||||
*/
|
||||
if (clientOptions?.configuration?.defaultHeaders != null) {
|
||||
clientOptions.configuration.defaultHeaders = resolveHeaders({
|
||||
headers: clientOptions.configuration.defaultHeaders,
|
||||
user: createSafeUser(this.options.req?.user),
|
||||
body: {
|
||||
messageId: this.responseMessageId,
|
||||
conversationId: this.conversationId,
|
||||
parentMessageId: this.parentMessageId,
|
||||
},
|
||||
});
|
||||
}
|
||||
resolveConfigHeaders({
|
||||
llmConfig: clientOptions,
|
||||
user: createSafeUser(this.options.req?.user),
|
||||
body: {
|
||||
messageId: this.responseMessageId,
|
||||
conversationId: this.conversationId,
|
||||
parentMessageId: this.parentMessageId,
|
||||
},
|
||||
});
|
||||
|
||||
try {
|
||||
const titleResult = await this.run.generateTitle({
|
||||
|
|
|
|||
|
|
@ -225,6 +225,51 @@ describe('AgentClient - titleConvo', () => {
|
|||
expect(generateTitleCall.clientOptions.model).toBe('gpt-3.5-turbo');
|
||||
});
|
||||
|
||||
it('preserves Anthropic custom headers on title requests despite omitTitleOptions', async () => {
|
||||
const prevKey = process.env.ANTHROPIC_API_KEY;
|
||||
process.env.ANTHROPIC_API_KEY = 'sk-ant-test';
|
||||
try {
|
||||
const req = {
|
||||
user: { id: 'user-123' },
|
||||
body: { model: 'claude-sonnet-4-5', endpoint: EModelEndpoint.anthropic, key: null },
|
||||
config: {
|
||||
endpoints: {
|
||||
[EModelEndpoint.anthropic]: {
|
||||
headers: { 'X-Conversation-Id': '{{LIBRECHAT_BODY_CONVERSATIONID}}' },
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
const agent = {
|
||||
id: 'agent-anthropic',
|
||||
endpoint: EModelEndpoint.anthropic,
|
||||
provider: EModelEndpoint.anthropic,
|
||||
model_parameters: { model: 'claude-sonnet-4-5' },
|
||||
};
|
||||
const anthropicClient = new AgentClient({ req, res: {}, agent, endpointTokenConfig: {} });
|
||||
anthropicClient.run = mockRun;
|
||||
anthropicClient.responseMessageId = 'response-123';
|
||||
anthropicClient.conversationId = 'convo-123';
|
||||
anthropicClient.contentParts = [{ type: 'text', text: 'Test content' }];
|
||||
anthropicClient.recordCollectedUsage = jest.fn().mockResolvedValue();
|
||||
|
||||
await anthropicClient.titleConvo({ text: 'Hello', abortController: new AbortController() });
|
||||
|
||||
const defaultHeaders =
|
||||
mockRun.generateTitle.mock.calls[0][0].clientOptions?.clientOptions?.defaultHeaders;
|
||||
// Custom header survives the `omitTitleOptions` strip and resolves the conversationId
|
||||
expect(defaultHeaders?.['X-Conversation-Id']).toBe('convo-123');
|
||||
// Provider-managed beta header is preserved alongside it
|
||||
expect(defaultHeaders?.['anthropic-beta']).toBeDefined();
|
||||
} finally {
|
||||
if (prevKey === undefined) {
|
||||
delete process.env.ANTHROPIC_API_KEY;
|
||||
} else {
|
||||
process.env.ANTHROPIC_API_KEY = prevKey;
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it('should handle missing endpoint config gracefully', async () => {
|
||||
// Remove endpoint config
|
||||
mockReq.config = { endpoints: {} };
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
const { logger } = require('@librechat/data-schemas');
|
||||
const { EModelEndpoint } = require('librechat-data-provider');
|
||||
const {
|
||||
mergeHeaders,
|
||||
getAnthropicModels,
|
||||
getBedrockModels,
|
||||
getOpenAIModels,
|
||||
|
|
@ -25,18 +26,35 @@ async function loadDefaultModels(req) {
|
|||
}));
|
||||
const vertexConfig = appConfig?.endpoints?.[EModelEndpoint.anthropic]?.vertexConfig;
|
||||
|
||||
/** Forward configured custom headers (endpoint over global `all`) so model
|
||||
* fetches reach a gateway-fronted provider the same as chat requests. */
|
||||
const allHeaders = appConfig?.endpoints?.all?.headers;
|
||||
const openAIHeaders = mergeHeaders(
|
||||
allHeaders,
|
||||
appConfig?.endpoints?.[EModelEndpoint.openAI]?.headers,
|
||||
);
|
||||
const anthropicHeaders = mergeHeaders(
|
||||
allHeaders,
|
||||
appConfig?.endpoints?.[EModelEndpoint.anthropic]?.headers,
|
||||
);
|
||||
|
||||
const [openAI, anthropic, azureOpenAI, assistants, azureAssistants, google, bedrock] =
|
||||
await Promise.all([
|
||||
getOpenAIModels({ user: req.user.id }).catch((error) => {
|
||||
logger.error('Error fetching OpenAI models:', error);
|
||||
return [];
|
||||
}),
|
||||
getAnthropicModels({ user: req.user.id, vertexModels: vertexConfig?.modelNames }).catch(
|
||||
getOpenAIModels({ user: req.user.id, headers: openAIHeaders, userObject: req.user }).catch(
|
||||
(error) => {
|
||||
logger.error('Error fetching Anthropic models:', error);
|
||||
logger.error('Error fetching OpenAI models:', error);
|
||||
return [];
|
||||
},
|
||||
),
|
||||
getAnthropicModels({
|
||||
user: req.user.id,
|
||||
vertexModels: vertexConfig?.modelNames,
|
||||
headers: anthropicHeaders,
|
||||
userObject: req.user,
|
||||
}).catch((error) => {
|
||||
logger.error('Error fetching Anthropic models:', error);
|
||||
return [];
|
||||
}),
|
||||
getOpenAIModels({ user: req.user.id, azure: true }).catch((error) => {
|
||||
logger.error('Error fetching Azure OpenAI models:', error);
|
||||
return [];
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue