From 27acf97ad91595ed054e2c9790cdfdbea7515b3a Mon Sep 17 00:00:00 2001 From: Joseph Licata <54822374+usnavy13@users.noreply.github.com> Date: Mon, 14 Sep 2026 15:06:35 -0400 Subject: [PATCH] =?UTF-8?q?=F0=9F=93=AB=20fix:=20Route=20Azure=20Astra=20T?= =?UTF-8?q?hrough=20Responses=20API=20(#15845)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix: route Azure Astra through Responses API * fix: keep Azure Astra deployment alias out of summarizer overrides The Azure deployment alias in `modelKwargs.model` is spread after `model` by langchain, so the SDK's same-provider summarizer, which copies the agent's client options and overrides only `model`, still reached the Astra deployment when configured with a different model. Hand the summarizer the agent's kwargs without the alias whenever its model differs. Also derive the Astra routing decision once, reuse the first-party OpenAI predicate in the GPT-5.6 rule, pin the opt-out shape in the Azure spec, and expect `firstPartyEndpoint` for canonical Azure hosts in the backward-compat spec. Co-Authored-By: Claude Fable 5.1 * fix: resolve Azure request URLs and summarizer deployments * fix: honor explicit OpenAI summarization for Azure agents * ๐Ÿงญ fix: Degrade Unresolvable Azure Summarizers Without Failing the Run Summarization target resolution for Azure agents threw inside createRun, so a summarization setting that could not be resolved failed every agent run before any model call, not just compaction. The documented `summarization.provider: openAI` block with the `.env.example` default `OPENAI_API_KEY=user_provided` broke all Azure agents, and a summary model missing from the Azure configuration did the same. - An explicit OpenAI summarizer without server credentials now disables summarization for that Azure agent with a one-time warning instead of throwing; the agent's own client is not a fallback because Azure Responses shares the `openAI` provider. - A summary model the Azure configuration cannot resolve warns once and keeps the agent's client, as before this branch. - A summary group without a base URL no longer inherits the agent group's `azureOpenAIBasePath`, which sent the summary key and deployment to the agent's resource. - Serverless summary groups without a version send no empty `api-version`. * ๐Ÿงช test: Assert No Summary Request Leaves a Disabled OpenAI Summarizer * ๐Ÿงฏ fix: Disable Summaries Whose Azure Deployment Cannot Be Resolved When the Azure configuration could not resolve a summary model, the summarizer fell back to the agent's own client. On an Azure Responses agent that stripped the deployment alias and sent the logical model name where Azure expects a deployment, so every compaction failed; on a chat agent the URL-pinned deployment silently served a different model than the one configured. An unresolvable Azure summary target now disables summarization for that agent with a one-time warning, matching the unreachable OpenAI target: the run proceeds on context pruning and no summary request is sent. * ๐Ÿ”‘ fix: Treat an Empty Azure Summary Key as Unavailable `azureBaseSchema` accepts `apiKey: ''`, and `mapModelToAzureConfig` passes it through, so a summary group with an empty key kept summarization enabled and built a client with no credential. The agent flow already rejects an empty Azure key during initialization; the summary resolver now disables summarization for that target with the same one-time warning as a user-provided key or base URL. * ๐Ÿงญ fix: Resolve Azure Summaries From Every Supported Azure Configuration Three Azure configuration shapes the summary resolver and URL builders did not cover: - Legacy `AZURE_*` environment credentials: without an `azureOpenAI` block the resolver skipped Azure, so an Astra agent's summarizer reused the agent's Responses client and sent the logical summary model where a deployment belongs. The target now resolves from the environment the way `initializeOpenAI` does (including `AZURE_USE_MODEL_AS_DEPLOYMENT_NAME`) and is disabled when those credentials are user-provided or missing. - `summarization.parameters` `baseURL` and `apiKey` overrides now apply to Azure summary targets, as they already did for explicit OpenAI targets, through one shared helper. - A full Azure hostname used as the instance name (as `genAzureEndpoint` accepts) is used as the host for both the default Responses URL and the chat deployments path, instead of being suffixed into `.openai.azure.com`. * ๐Ÿ” fix: Expand Summarizer Credential Placeholders and Bound Warnings Admin-authored `summarization.parameters` `apiKey` and `baseURL` values were forwarded verbatim, so `${SUMMARY_KEY}` reached the client as a literal key while Azure groups and custom endpoints expand the same syntax. They are now expanded once through `extractEnvVariable` (keeping its sensitive-variable denylist) before resolution and before the parameters are layered over the client, and a credential or base URL that still holds an unresolved placeholder disables summarization like a missing one. Azure base URL templates (`${INSTANCE_NAME}`, `${DEPLOYMENT_NAME}`) are not treated as unresolved. The one-time warning set is now keyed per tenant, so one tenant's report no longer hides another's, and resets after 256 distinct entries so changing configurations cannot grow it without bound. * ๐Ÿงฉ fix: Keep Azure URL Templates Out of Summarizer Env Expansion Expanding `summarization.parameters.baseURL` through `extractEnvVariable` also replaced Azure's reserved `${INSTANCE_NAME}` and `${DEPLOYMENT_NAME}` templates whenever the host environment happened to define those names, so a template override could route compaction to the host's instance or deployment instead of the summary group's. Expansion now skips those two templates and leaves them for `constructAzureURL`, and the unresolved-placeholder check reads the same segments. The warning deduplication set now evicts the least recently seen entry at its bound instead of clearing, so warnings for problems that are still recurring are not all logged again at once. * ๐Ÿ”€ fix: Reset Inherited API Mode for Custom-Endpoint Summarizers A custom OpenAI-compatible summarization endpoint resolves to the same `openAI` provider as an Azure Responses agent, so the SDK layered its overrides over the agent's client options and kept the agent's `useResponsesApi`, `firstPartyEndpoint`, reasoning and request kwargs. An Azure Astra agent therefore sent its compaction to the gateway's `/responses` with Astra shaping. When the resolved custom provider matches the agent's, those fields are now replaced from the target's own configuration, and explicit summarization parameters can still opt back in. Azure's reserved `${INSTANCE_NAME}`/`${DEPLOYMENT_NAME}` URL templates are now preserved only for Azure summarization targets, so an OpenAI or custom base URL can reference environment variables with those names. * fix: Resolve Azure Summary Identity, Transport, and Token Limits ๐Ÿงญ * fix: Resolve Headers Before Building Summary Clients ๐Ÿ“จ --------- Co-authored-by: Claude Fable 5.1 Co-authored-by: Danny Avila --- .../src/agents/__tests__/initialize.test.ts | 40 + .../__tests__/run-summarization.test.ts | 1040 ++++++++++++++++- packages/api/src/agents/initialize.ts | 3 + packages/api/src/agents/run.ts | 581 ++++++++- .../openai/config.backward-compat.spec.ts | 1 + .../api/src/endpoints/openai/config.spec.ts | 4 +- packages/api/src/endpoints/openai/config.ts | 17 +- .../api/src/endpoints/openai/initialize.ts | 5 +- packages/api/src/endpoints/openai/llm.spec.ts | 91 +- packages/api/src/endpoints/openai/llm.ts | 84 +- .../api/src/endpoints/openai/requests.spec.ts | 227 ++++ packages/api/src/types/endpoints.ts | 3 + packages/api/src/types/openai.ts | 2 +- packages/api/src/utils/azure.ts | 75 +- packages/data-provider/src/config.ts | 7 +- 15 files changed, 2090 insertions(+), 90 deletions(-) create mode 100644 packages/api/src/endpoints/openai/requests.spec.ts diff --git a/packages/api/src/agents/__tests__/initialize.test.ts b/packages/api/src/agents/__tests__/initialize.test.ts index 7ee63434de..51ea605bab 100644 --- a/packages/api/src/agents/__tests__/initialize.test.ts +++ b/packages/api/src/agents/__tests__/initialize.test.ts @@ -286,6 +286,46 @@ describe('initializeAgent โ€” execution context', () => { jest.clearAllMocks(); }); + it('carries request-resolved Azure identity to the run without changing persisted agent fields', async () => { + const { agent, req, res, loadTools, db } = createMocks({ + provider: EModelEndpoint.azureOpenAI, + model: 'gpt-6-astra', + }); + const azureOptions = { + azureOpenAIApiKey: 'request-key', + azureOpenAIApiInstanceName: 'request-instance', + azureOpenAIApiDeploymentName: 'deployment', + azureOpenAIApiVersion: '2024-10-21', + }; + mockGetProviderConfig.mockReturnValue({ + overrideProvider: Providers.AZURE, + getOptions: jest.fn().mockResolvedValue({ + azureOptions, + llmConfig: { + model: 'gpt-6-astra', + useResponsesApi: true, + apiKey: 'request-key', + modelKwargs: { model: 'deployment' }, + }, + }), + }); + const result = await initializeAgent( + { + req, + res, + agent, + loadTools, + endpointOption: { endpoint: EModelEndpoint.agents }, + allowedProviders: new Set([EModelEndpoint.azureOpenAI]), + isInitialAgent: true, + }, + db, + ); + expect(result.azureOptions).toEqual(azureOptions); + expect(result.model_parameters).not.toHaveProperty('azureOpenAIApiInstanceName'); + expect(agent).not.toHaveProperty('azureOptions'); + }); + it('initializes without Express request or response objects', async () => { const { agent, loadTools, db } = createMocks(); const previousReqDescriptor = Object.getOwnPropertyDescriptor(globalThis, 'req'); diff --git a/packages/api/src/agents/__tests__/run-summarization.test.ts b/packages/api/src/agents/__tests__/run-summarization.test.ts index a3ff8797b9..ede0868eb4 100644 --- a/packages/api/src/agents/__tests__/run-summarization.test.ts +++ b/packages/api/src/agents/__tests__/run-summarization.test.ts @@ -1,4 +1,5 @@ import { encryptV3, logger } from '@librechat/data-schemas'; +import { HumanMessage, AIMessage } from '@langchain/core/messages'; import { CallbackManager } from '@langchain/core/callbacks/manager'; import { EModelEndpoint, @@ -6,12 +7,16 @@ import { MAX_SUBAGENT_DEPTH, MAX_SUBAGENT_RUN_CONFIGS, } from 'librechat-data-provider'; -import type { CompactionSemanticIndex, SubagentTaskConfig } from '@librechat/agents'; +import type { CompactionSemanticIndex, SubagentTaskConfig, AgentInputs } from '@librechat/agents'; import type { SummarizationConfig, TEndpoint } from 'librechat-data-provider'; import type { AppConfig, IUser } from '@librechat/data-schemas'; import type { BaseMessage } from '@langchain/core/messages'; +import type { OpenAI } from 'openai'; import type { ModelBoundChatModelCallback } from '~/middleware/modelBoundContent'; +import type { OpenAIConfiguration, AzureOptions } from '~/types'; import { createRun, isAskUserQuestionAdminDisabled } from '~/agents/run'; +import { initializeOpenAI } from '~/endpoints/openai/initialize'; +import { getOpenAIConfig } from '~/endpoints/openai/config'; // Mock winston logger โ€” `format` must be callable so @librechat/data-schemas // dist module-load completes cleanly; see api/test/__mocks__/logger.js. @@ -190,6 +195,7 @@ async function callAndCapture( modelCallbacks?: readonly ModelBoundChatModelCallback[]; user?: IUser; tenantId?: string; + requestBody?: Parameters[0]['requestBody']; } = {}, ) { const agents = opts.agents ?? [makeAgent()]; @@ -209,6 +215,7 @@ async function callAndCapture( modelCallbacks: opts.modelCallbacks, user: opts.user, tenantId: opts.tenantId, + requestBody: opts.requestBody, streaming: true, streamUsage: true, }); @@ -762,12 +769,14 @@ const ADAPTIVE_CLAUDE_MODEL = 'anthropic/claude-sonnet-4.6'; /** Agent whose resolved client options already carry a reasoning configuration. */ function makeReasoningAgent(overrides: { + azureOptions?: AzureOptions; provider: string; endpoint: string; model: string; model_parameters: Record; }) { return makeAgent({ + ...overrides, provider: overrides.provider as never, endpoint: overrides.endpoint, model: overrides.model, @@ -971,6 +980,1035 @@ describe('summarization reasoning effort', () => { }); }); +type CapturedRequest = { + url: URL; + headers: Headers; + body: OpenAI.ChatCompletionCreateParams & OpenAI.Responses.ResponseCreateParams; +}; + +async function compactSummary( + agents: Array>, + requests: CapturedRequest[] = [], +) { + const summaryConfig = agents[0].summarizationConfig as NonNullable< + AgentInputs['summarizationConfig'] + >; + const clientOptions = agents[0].clientOptions as { configuration?: OpenAIConfiguration }; + const configuration = { + ...((summaryConfig.parameters?.configuration ?? + clientOptions.configuration) as OpenAIConfiguration), + }; + summaryConfig.parameters = { ...summaryConfig.parameters, configuration }; + configuration.fetch = async (url, init) => { + requests.push({ + url: new URL(String(url)), + headers: new Headers(init?.headers), + body: JSON.parse(String(init?.body)), + }); + const text = 'The user asked for arithmetic and the assistant calculated four.'; + return Response.json({ + id: 'summary-response', + model: 'summary-production', + object: 'response', + status: 'completed', + choices: [{ index: 0, message: { role: 'assistant', content: text }, finish_reason: 'stop' }], + output: [ + { + type: 'message', + id: 'msg_summary', + role: 'assistant', + status: 'completed', + content: [{ type: 'output_text', text, annotations: [] }], + }, + ], + usage: { + prompt_tokens: 20, + completion_tokens: 10, + input_tokens: 20, + output_tokens: 10, + total_tokens: 30, + }, + }); + }; + const actual = jest.requireActual('@librechat/agents'); + const runConfig = (Run.create as jest.Mock).mock.calls[0][0] as Parameters[0]; + const run = await actual.Run.create({ + ...runConfig, + graphConfig: { ...runConfig.graphConfig, agents: agents as unknown as AgentInputs[] }, + tokenCounter: (message) => String(message.content).length, + }); + await run.processStream( + { messages: [new HumanMessage('Compute 2 + 2.'), new AIMessage('4')] }, + { version: 'v2', configurable: { thread_id: 'azure-summary-test' } }, + ); + + return { requests }; +} + +describe('Azure deployment alias', () => { + /** `initializeAgent` maps an Azure Responses agent to the OpenAI provider. */ + const azureAstraAgent = () => { + const { llmConfig, configOptions } = getOpenAIConfig( + 'test-azure-key', + { + azure: { + azureOpenAIApiInstanceName: 'test-instance', + azureOpenAIApiDeploymentName: 'production-deployment', + azureOpenAIApiVersion: '2025-04-01-preview', + azureOpenAIApiKey: 'test-azure-key', + }, + modelOptions: { model: 'gpt-6-astra', max_tokens: 2048 }, + }, + EModelEndpoint.azureOpenAI, + ); + return makeReasoningAgent({ + provider: EModelEndpoint.openAI, + endpoint: EModelEndpoint.azureOpenAI, + model: 'gpt-6-astra', + model_parameters: { ...llmConfig, configuration: configOptions }, + }); + }; + + /** The SDK spreads `parameters` onto the agent's client options, then sets `model`. */ + const summaryRequestModel = ( + clientOptions: Record, + summaryConfig: Record, + ) => { + const summaryModel = new ChatOpenAI({ + ...clientOptions, + ...((summaryConfig.parameters as Record | undefined) ?? {}), + apiKey: 'test-key', + model: summaryConfig.model as string, + } as never); + return (summaryModel.invocationParams() as Record).model; + }; + + it("keeps the Astra agent's API mode and deployment alias out of a custom-endpoint summarizer", async () => { + const agents = await callAndCapture({ + agents: [azureAstraAgent()], + appConfig: makeAppConfig([ + { + name: 'Gateway', + apiKey: 'gateway-key', + baseURL: 'https://gateway.example/v1', + models: { default: ['gpt-4.1-mini'] }, + }, + ]), + summarizeOnly: true, + summarizationConfig: { + provider: 'Gateway', + model: 'gpt-4.1-mini', + parameters: { streaming: false }, + }, + }); + + expect((agents[0].clientOptions as Record).modelKwargs).toEqual({ + model: 'production-deployment', + max_output_tokens: 2048, + }); + const { requests } = await compactSummary(agents); + expect(requests).toHaveLength(1); + const { url, headers, body } = requests[0]; + expect(url.origin + url.pathname).toBe('https://gateway.example/v1/chat/completions'); + expect(headers.get('api-key')).toBeNull(); + expect(body.model).toBe('gpt-4.1-mini'); + expect(body).not.toHaveProperty('include'); + }); + + it.each([ + { useModelAsDeploymentName: undefined, deployment: 'env-deployment' }, + { useModelAsDeploymentName: 'true', deployment: 'gpt-41-mini' }, + ])( + 'resolves a different summary model through the legacy Azure environment to $deployment', + async ({ useModelAsDeploymentName, deployment }) => { + jest.replaceProperty(process, 'env', { + ...process.env, + AZURE_API_KEY: 'env-key', + AZURE_OPENAI_API_INSTANCE_NAME: 'env-instance', + AZURE_OPENAI_API_DEPLOYMENT_NAME: 'env-deployment', + AZURE_OPENAI_API_VERSION: '2024-10-21', + AZURE_USE_MODEL_AS_DEPLOYMENT_NAME: useModelAsDeploymentName, + }); + const agents = await callAndCapture({ + agents: [azureAstraAgent()], + appConfig: makeAppConfig([]), + summarizeOnly: true, + summarizationConfig: { model: 'gpt-4.1-mini', parameters: { streaming: false } }, + }); + + const { requests } = await compactSummary(agents); + expect(requests).toHaveLength(1); + const { url, headers, body } = requests[0]; + expect(url.origin + url.pathname).toBe( + `https://env-instance.openai.azure.com/openai/deployments/${deployment}/chat/completions`, + ); + expect(headers.get('api-key')).toBe('env-key'); + expect(body.model).toBe(deployment); + }, + ); + + it('applies summarization base URL and API key overrides to an Azure summary deployment', async () => { + const appConfig = makeAppConfig([]); + appConfig.endpoints![EModelEndpoint.azureOpenAI] = { + isValid: true, + errors: [], + modelNames: ['gpt-6-astra', 'gpt-4.1-mini'], + modelGroupMap: { 'gpt-6-astra': { group: 'main' }, 'gpt-4.1-mini': { group: 'summary' } }, + groupMap: { + main: { + apiKey: 'test-azure-key', + instanceName: 'test-instance', + version: '2025-04-01-preview', + models: { 'gpt-6-astra': { deploymentName: 'production-deployment' } }, + }, + summary: { + apiKey: 'summary-key', + instanceName: 'summary-instance', + version: '2024-10-21', + models: { 'gpt-4.1-mini': { deploymentName: 'summary-production' } }, + }, + }, + }; + const agents = await callAndCapture({ + agents: [azureAstraAgent()], + appConfig, + summarizeOnly: true, + summarizationConfig: { + model: 'gpt-4.1-mini', + parameters: { + streaming: false, + apiKey: 'gateway-key', + baseURL: 'https://summary-gateway.example/openai/deployments/${DEPLOYMENT_NAME}', + }, + }, + }); + + const { requests } = await compactSummary(agents); + expect(requests).toHaveLength(1); + const { url, headers } = requests[0]; + expect(url.origin + url.pathname).toBe( + 'https://summary-gateway.example/openai/deployments/summary-production/chat/completions', + ); + expect(headers.get('api-key')).toBe('gateway-key'); + }); + + it.each([ + { model: 'gpt-6-astra', initialResponses: true, useResponsesApi: true }, + { model: 'gpt-4.1', initialResponses: false, useResponsesApi: false }, + { model: 'gpt-6-astra', initialResponses: true, useResponsesApi: false }, + { model: 'gpt-4.1', initialResponses: false, useResponsesApi: true }, + ])( + 'uses same-model Azure transport overrides for $model', + async ({ model, initialResponses, useResponsesApi }) => { + const { llmConfig, configOptions } = getOpenAIConfig( + 'resolved-user-key', + { + azure: { + azureOpenAIApiKey: 'resolved-user-key', + azureOpenAIApiInstanceName: 'user-instance', + azureOpenAIApiDeploymentName: 'user-deployment', + azureOpenAIApiVersion: '2024-10-21', + }, + modelOptions: { model, max_tokens: 1536 }, + headers: { 'X-Request': 'resolved-user-header' }, + }, + EModelEndpoint.azureOpenAI, + ); + const agents = await callAndCapture({ + agents: [ + makeReasoningAgent({ + endpoint: EModelEndpoint.azureOpenAI, + provider: initialResponses ? Providers.OPENAI : Providers.AZURE, + model, + model_parameters: { ...llmConfig, configuration: configOptions }, + }), + ], + appConfig: makeAppConfig([]), + summarizeOnly: true, + summarizationConfig: { + model, + parameters: { + streaming: false, + apiKey: 'override-key', + useResponsesApi, + baseURL: + 'https://summary-instance.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}', + }, + }, + }); + expect(agents[0].summarizationEnabled).toBe(true); + const { requests } = await compactSummary(agents); + expect(requests).toHaveLength(1); + expect(requests[0].url.origin + requests[0].url.pathname).toBe( + useResponsesApi + ? 'https://summary-instance.openai.azure.com/openai/v1/responses' + : 'https://summary-instance.openai.azure.com/openai/deployments/user-deployment/chat/completions', + ); + expect(requests[0].headers.get('api-key')).toBe('override-key'); + expect(requests[0].headers.get('X-Request')).toBe('resolved-user-header'); + expect(requests[0].body.model).toBe('user-deployment'); + const chatTokenKey = model === 'gpt-6-astra' ? 'max_completion_tokens' : 'max_tokens'; + expect(requests[0].body[useResponsesApi ? 'max_output_tokens' : chatTokenKey]).toBe(1536); + expect((agents[0].clientOptions as Record).configuration).toEqual( + configOptions, + ); + }, + ); + + it.each(['root', 'lazy'])( + 'retains Azure identity and resolved headers for a %s self-summary', + async (kind) => { + jest.replaceProperty(process, 'env', { + ...process.env, + AZURE_API_KEY: 'user_provided', + SUMMARY_HEADER_SECRET: 'must-not-leak', + }); + const user = { id: 'user-1', username: '${SUMMARY_HEADER_SECRET}' }; + const appConfig = makeAppConfig([]); + appConfig.endpoints!.all = { + headers: { + 'X-Conversation': '{{LIBRECHAT_BODY_CONVERSATIONID}}', + 'X-Tenant': '{{LIBRECHAT_USER_TENANTID}}', + 'X-User': '{{LIBRECHAT_USER_USERNAME}}', + }, + }; + const getUserKeyValues = jest.fn().mockResolvedValue({ + apiKey: JSON.stringify({ + azureOpenAIApiKey: 'user-key', + azureOpenAIApiInstanceName: 'user-instance', + azureOpenAIApiDeploymentName: 'user-deployment', + azureOpenAIApiVersion: '2024-10-21', + }), + }); + const options = await initializeOpenAI({ + endpoint: EModelEndpoint.azureOpenAI, + model_parameters: { model: 'gpt-6-astra' }, + runtime: { appConfig, user, requestBody: {} }, + db: { getUserKeyValues }, + } as unknown as Parameters[0]); + const azureAgent = { + ...makeReasoningAgent({ + endpoint: EModelEndpoint.azureOpenAI, + provider: Providers.OPENAI, + model: 'gpt-6-astra', + azureOptions: options.azureOptions, + model_parameters: { ...options.llmConfig, configuration: options.configOptions }, + }), + id: 'azure-child', + }; + const lazyParent = makeAgent({ + id: 'parent', + summarization: { enabled: false }, + subagents: { enabled: true, allowSelf: false }, + lazySubagentConfigs: [ + { + id: azureAgent.id, + name: 'Azure child', + description: 'Lazy Azure child', + configId: 'azure-child:1:test', + resolve: jest.fn().mockResolvedValue(azureAgent), + }, + ], + }); + const captured = await callAndCapture({ + agents: kind === 'lazy' ? [lazyParent] : [azureAgent], + appConfig, + user: user as IUser, + tenantId: 'tenant-1', + requestBody: { conversationId: 'conversation-1' }, + summarizeOnly: true, + summarizationConfig: { + parameters: { + streaming: false, + baseURL: + 'https://${INSTANCE_NAME}.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}', + }, + }, + }); + let selected = captured[0]; + if (kind === 'lazy') { + const [child] = captured[0].subagentConfigs as Array>; + selected = await ( + child.resolveAgentInputs as (context: never) => Promise> + )({ signal: new AbortController().signal } as never); + selected.summarizeOnly = true; + } + const agents = [selected]; + const { requests } = await compactSummary(agents); + expect(requests).toHaveLength(1); + expect(requests[0].url.origin + requests[0].url.pathname).toBe( + 'https://user-instance.openai.azure.com/openai/v1/responses', + ); + expect(requests[0].body.model).toBe('user-deployment'); + expect(requests[0].headers.get('api-key')).toBe('user-key'); + expect(requests[0].headers.get('X-Conversation')).toBe('conversation-1'); + expect(requests[0].headers.get('X-Tenant')).toBe('tenant-1'); + expect(requests[0].headers.get('X-User')).toBe('${SUMMARY_HEADER_SECRET}'); + const mainConfiguration = (agents[0].clientOptions as Record) + .configuration as OpenAIConfiguration; + expect(new Headers(mainConfiguration?.defaultHeaders as HeadersInit).get('X-User')).toBe( + '${SUMMARY_HEADER_SECRET}', + ); + expect(getUserKeyValues).toHaveBeenCalledTimes(1); + }, + ); + + it.each([ + { parameterCap: undefined, expected: 512 }, + { parameterCap: '128', expected: 512 }, + { parameterCap: 0, expected: 512 }, + { parameterCap: 256, expected: 256 }, + ])( + 'honors the summary token cap with parameter cap $parameterCap', + async ({ parameterCap, expected }) => { + const agents = await callAndCapture({ + agents: [azureAstraAgent()], + summarizeOnly: true, + summarizationConfig: { + maxSummaryTokens: 512, + parameters: { + streaming: false, + ...(parameterCap !== undefined ? { maxSummaryTokens: parameterCap } : {}), + }, + }, + }); + const { requests } = await compactSummary(agents); + expect(requests).toHaveLength(1); + expect(requests[0].body.max_output_tokens).toBe(expected); + expect(requests[0].body.model).toBe('production-deployment'); + }, + ); + + it.each([ + { nested: false, useResponsesApi: true }, + { nested: true, useResponsesApi: true }, + { nested: false, useResponsesApi: false }, + ])( + 'normalizes final Azure transport (nested: $nested, Responses: $useResponsesApi)', + async ({ nested, useResponsesApi }) => { + const model = 'gpt-4.1-mini'; + const appConfig = makeAppConfig([]); + appConfig.endpoints!.azureOpenAI = { + isValid: true, + errors: [], + modelNames: [model], + modelGroupMap: { [model]: { group: 'summary' } }, + groupMap: { + summary: { + apiKey: 'summary-key', + instanceName: 'summary-instance', + version: '2024-10-21', + models: { [model]: { deploymentName: 'summary-production' } }, + addParams: { useResponsesApi: !useResponsesApi }, + }, + }, + }; + const baseURL = + 'https://${INSTANCE_NAME}.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}?api-version=2025-04-01-preview'; + const parameters = { + streaming: false, + useResponsesApi, + ...(nested + ? { configuration: { baseURL, defaultHeaders: { 'X-Override': 'yes' } } } + : { baseURL }), + } as unknown as SummarizationConfig['parameters']; + const agents = await callAndCapture({ + agents: [azureAstraAgent()], + appConfig, + summarizeOnly: true, + summarizationConfig: { model, parameters }, + }); + const { requests } = await compactSummary(agents); + expect(requests).toHaveLength(1); + expect(requests[0].url.origin + requests[0].url.pathname).toBe( + useResponsesApi + ? 'https://summary-instance.openai.azure.com/openai/v1/responses' + : 'https://summary-instance.openai.azure.com/openai/deployments/summary-production/chat/completions', + ); + expect(requests[0].headers.get('api-key')).toBe('summary-key'); + expect(requests[0].body.model).toBe('summary-production'); + if (nested) expect(requests[0].headers.get('X-Override')).toBe('yes'); + if (useResponsesApi) + expect(requests[0].url.searchParams.get('api-version')).toBe('2025-04-01-preview'); + }, + ); + + it.each([EModelEndpoint.azureOpenAI, EModelEndpoint.openAI])( + 'disables an invalid %s summary URL before constructing a run client', + async (provider) => { + jest.replaceProperty(process, 'env', { ...process.env, OPENAI_API_KEY: 'summary-key' }); + const agents = await callAndCapture({ + agents: [azureAstraAgent()], + appConfig: makeAppConfig([]), + summarizeOnly: true, + summarizationConfig: { + provider, + model: 'gpt-6-astra', + parameters: { streaming: false, baseURL: 'not a URL' }, + }, + }); + expect(agents[0].summarizationEnabled).toBe(false); + const requests: CapturedRequest[] = []; + await expect(compactSummary(agents, requests)).rejects.toThrow( + 'Compaction skipped: summarization is not enabled for this agent', + ); + expect(requests).toHaveLength(0); + }, + ); + + it('keeps the deployment alias when the summarizer runs the agent model', async () => { + const agents = await callAndCapture({ agents: [azureAstraAgent()] }); + + const mainClientOptions = agents[0].clientOptions as Record; + const summaryConfig = agents[0].summarizationConfig as Record; + + expect(summaryConfig.parameters).toBeUndefined(); + expect(summaryRequestModel(mainClientOptions, summaryConfig)).toBe('production-deployment'); + }); + + it.each<{ + model: string; + deployed: boolean; + proxy?: string; + parameters?: SummarizationConfig['parameters']; + }>([ + { model: 'gpt-4.1-mini', deployed: false, proxy: undefined }, + { model: 'gpt-4.1-mini', deployed: true, proxy: undefined }, + { model: 'gpt-6-astra', deployed: true, proxy: undefined }, + { model: 'gpt-4.1-mini', deployed: false, proxy: 'https://summary-gateway.example/v1' }, + { + model: 'gpt-6-astra', + deployed: true, + parameters: { useResponsesApi: false, apiKey: 'override-key' }, + }, + { + model: 'gpt-4.1-mini', + deployed: false, + parameters: { baseURL: 'https://per-summary.example/v1', useResponsesApi: true }, + }, + ])( + 'honors explicit OpenAI for $model (Azure deployed: $deployed, proxy: $proxy)', + async ({ model, deployed, proxy, parameters }) => { + jest.replaceProperty(process, 'env', { + ...process.env, + OPENAI_API_KEY: 'openai-summary-key', + OPENAI_REVERSE_PROXY: proxy, + }); + const appConfig = makeAppConfig([]); + appConfig.endpoints!.all = { headers: { 'X-Global': 'global' } }; + appConfig.endpoints!.openAI = { headers: { 'X-Summary': 'openai' } }; + appConfig.endpoints!.azureOpenAI = { + isValid: true, + errors: [], + modelNames: deployed ? [model] : [], + modelGroupMap: deployed ? { [model]: { group: 'azure' } } : {}, + groupMap: { + azure: { + apiKey: 'test-azure-key', + instanceName: 'test-instance', + version: '2025-04-01-preview', + models: { [model]: { deploymentName: 'production-deployment' } }, + }, + }, + }; + const agents = await callAndCapture({ + agents: [azureAstraAgent()], + appConfig, + summarizeOnly: true, + summarizationConfig: { + provider: EModelEndpoint.openAI, + model, + parameters: { streaming: false, ...parameters }, + }, + }); + const { requests } = await compactSummary(agents); + expect(requests).toHaveLength(1); + const { url, headers, body } = requests[0]; + const baseURL = parameters?.baseURL ?? proxy; + expect(url.origin).toBe( + typeof baseURL === 'string' ? new URL(baseURL).origin : 'https://api.openai.com', + ); + const usesResponses = parameters?.useResponsesApi ?? model === 'gpt-6-astra'; + expect(url.pathname).toBe(usesResponses ? '/v1/responses' : '/v1/chat/completions'); + expect(url.search).toBe(''); + expect(headers.get('authorization')).toBe( + `Bearer ${parameters?.apiKey ?? 'openai-summary-key'}`, + ); + expect(headers.has('api-key')).toBe(false); + expect(headers.get('X-Global')).toBe('global'); + expect(headers.get('X-Summary')).toBe('openai'); + expect(body.model).toBe(model); + expect(body).not.toHaveProperty('max_output_tokens', 2048); + expect((agents[0].clientOptions as OpenAIConfiguration)?.apiKey).toBe('test-azure-key'); + }, + ); + + it.each([ + { setting: 'OPENAI_API_KEY', value: undefined, model: 'gpt-4.1-mini' }, + { setting: 'OPENAI_API_KEY', value: 'user_provided', model: 'gpt-4.1-nano' }, + { setting: 'OPENAI_REVERSE_PROXY', value: 'user_provided', model: 'gpt-4o-mini' }, + ])( + 'runs the agent without summarization, not through Azure, when $setting is $value', + async ({ setting, value, model }) => { + jest.replaceProperty(process, 'env', { + ...process.env, + OPENAI_API_KEY: 'openai-summary-key', + [setting]: value, + }); + const agents = await callAndCapture({ + agents: [azureAstraAgent()], + appConfig: makeAppConfig([]), + summarizeOnly: true, + summarizationConfig: { + provider: EModelEndpoint.openAI, + model, + parameters: { streaming: false }, + }, + }); + + expect(agents[0].summarizationEnabled).toBe(false); + expect(logger.warn).toHaveBeenCalledWith( + `[createRun] Summarization with OpenAI model "${model}" is disabled for Azure OpenAI agents: it needs a server-configured OpenAI API key and base URL.`, + ); + const requests: CapturedRequest[] = []; + await expect(compactSummary(agents, requests)).rejects.toThrow( + 'Compaction skipped: summarization is not enabled for this agent', + ); + expect(requests).toHaveLength(0); + }, + ); + + it('reports an unreachable OpenAI summarizer once per tenant across runs', async () => { + jest.replaceProperty(process, 'env', { ...process.env, OPENAI_API_KEY: undefined }); + const run = (tenantId?: string) => + callAndCapture({ + agents: [azureAstraAgent()], + appConfig: makeAppConfig([]), + tenantId, + summarizationConfig: { provider: EModelEndpoint.openAI, model: 'o4-mini' }, + }); + + for (const tenantId of [undefined, undefined, 'tenant-a', 'tenant-a', 'tenant-b']) { + (Run.create as jest.Mock).mockClear(); + await run(tenantId); + } + + const warnings = (logger.warn as jest.Mock).mock.calls.filter(([message]) => + String(message).includes('"o4-mini"'), + ); + expect(warnings.map(([, meta]) => meta)).toEqual([ + undefined, + { tenantId: 'tenant-a' }, + { tenantId: 'tenant-b' }, + ]); + }); + + it('expands environment placeholders in OpenAI and Azure summarizer credentials', async () => { + jest.replaceProperty(process, 'env', { + ...process.env, + OPENAI_API_KEY: 'openai-summary-key', + SUMMARY_OPENAI_KEY: 'expanded-openai-key', + SUMMARY_OPENAI_URL: 'https://expanded-gateway.example/v1', + SUMMARY_AZURE_KEY: 'expanded-azure-key', + }); + const appConfig = makeAppConfig([]); + appConfig.endpoints![EModelEndpoint.azureOpenAI] = { + isValid: true, + errors: [], + modelNames: ['gpt-6-astra', 'gpt-4.1-mini'], + modelGroupMap: { 'gpt-6-astra': { group: 'main' }, 'gpt-4.1-mini': { group: 'summary' } }, + groupMap: { + main: { + apiKey: 'test-azure-key', + instanceName: 'test-instance', + version: '2025-04-01-preview', + models: { 'gpt-6-astra': { deploymentName: 'production-deployment' } }, + }, + summary: { + apiKey: 'summary-key', + instanceName: 'summary-instance', + version: '2024-10-21', + models: { 'gpt-4.1-mini': { deploymentName: 'summary-production' } }, + }, + }, + }; + const compact = async (summarizationConfig: SummarizationConfig) => { + (Run.create as jest.Mock).mockClear(); + const agents = await callAndCapture({ + agents: [azureAstraAgent()], + appConfig, + summarizeOnly: true, + summarizationConfig, + }); + const { requests } = await compactSummary(agents); + expect(requests).toHaveLength(1); + return requests[0]; + }; + + const openAI = await compact({ + provider: EModelEndpoint.openAI, + model: 'gpt-4.1-mini', + parameters: { + streaming: false, + apiKey: '${SUMMARY_OPENAI_KEY}', + baseURL: '${SUMMARY_OPENAI_URL}', + }, + }); + expect(openAI.url.origin).toBe('https://expanded-gateway.example'); + expect(openAI.headers.get('authorization')).toBe('Bearer expanded-openai-key'); + + const azure = await compact({ + model: 'gpt-4.1-mini', + parameters: { streaming: false, apiKey: '${SUMMARY_AZURE_KEY}' }, + }); + expect(azure.url.origin).toBe('https://summary-instance.openai.azure.com'); + expect(azure.headers.get('api-key')).toBe('expanded-azure-key'); + }); + + it("leaves Azure's reserved URL templates to the summary deployment", async () => { + jest.replaceProperty(process, 'env', { + ...process.env, + INSTANCE_NAME: 'host-instance', + DEPLOYMENT_NAME: 'host-deployment', + SUMMARY_GATEWAY_PATH: 'openai', + }); + const appConfig = makeAppConfig([]); + appConfig.endpoints![EModelEndpoint.azureOpenAI] = { + isValid: true, + errors: [], + modelNames: ['gpt-6-astra', 'gpt-4.1-mini'], + modelGroupMap: { 'gpt-6-astra': { group: 'main' }, 'gpt-4.1-mini': { group: 'summary' } }, + groupMap: { + main: { + apiKey: 'test-azure-key', + instanceName: 'test-instance', + version: '2025-04-01-preview', + models: { 'gpt-6-astra': { deploymentName: 'production-deployment' } }, + }, + summary: { + apiKey: 'summary-key', + instanceName: 'summary-instance', + version: '2024-10-21', + models: { 'gpt-4.1-mini': { deploymentName: 'summary-production' } }, + }, + }, + }; + const agents = await callAndCapture({ + agents: [azureAstraAgent()], + appConfig, + summarizeOnly: true, + summarizationConfig: { + model: 'gpt-4.1-mini', + parameters: { + streaming: false, + baseURL: + 'https://${INSTANCE_NAME}.openai.azure.com/${SUMMARY_GATEWAY_PATH}/deployments/${DEPLOYMENT_NAME}', + }, + }, + }); + + const { requests } = await compactSummary(agents); + expect(requests).toHaveLength(1); + const { url } = requests[0]; + expect(url.origin + url.pathname).toBe( + 'https://summary-instance.openai.azure.com/openai/deployments/summary-production/chat/completions', + ); + }); + + it('expands Azure-reserved names as ordinary variables for an OpenAI summarizer', async () => { + jest.replaceProperty(process, 'env', { + ...process.env, + OPENAI_API_KEY: 'openai-summary-key', + INSTANCE_NAME: 'gateway-host', + }); + const agents = await callAndCapture({ + agents: [azureAstraAgent()], + appConfig: makeAppConfig([]), + summarizeOnly: true, + summarizationConfig: { + provider: EModelEndpoint.openAI, + model: 'gpt-4.1-mini', + parameters: { streaming: false, baseURL: 'https://${INSTANCE_NAME}.example/v1' }, + }, + }); + + expect(agents[0].summarizationEnabled).toBe(true); + const { requests } = await compactSummary(agents); + expect(requests).toHaveLength(1); + expect(requests[0].url.origin).toBe('https://gateway-host.example'); + }); + + it('disables summarization when a credential placeholder has no environment value', async () => { + jest.replaceProperty(process, 'env', { ...process.env, OPENAI_API_KEY: 'openai-summary-key' }); + const agents = await callAndCapture({ + agents: [azureAstraAgent()], + appConfig: makeAppConfig([]), + summarizeOnly: true, + summarizationConfig: { + provider: EModelEndpoint.openAI, + model: 'gpt-4.1', + parameters: { streaming: false, apiKey: '${UNSET_SUMMARY_KEY}' }, + }, + }); + + expect(agents[0].summarizationEnabled).toBe(false); + const requests: CapturedRequest[] = []; + await expect(compactSummary(agents, requests)).rejects.toThrow( + 'Compaction skipped: summarization is not enabled for this agent', + ); + expect(requests).toHaveLength(0); + }); + + it.each([ + { + name: 'an inherited summary model absent from the Azure map', + agent: () => azureAstraAgent(), + summarizationConfig: { model: 'gpt-5.4-nano' }, + env: {}, + reason: 'Model named "gpt-5.4-nano" not found in configuration.', + }, + { + name: 'an explicit Azure summary model absent from the map on a non-Azure agent', + agent: () => makeAgent(), + summarizationConfig: { provider: EModelEndpoint.azureOpenAI, model: 'gpt-5.4-mini' }, + env: {}, + reason: 'Model named "gpt-5.4-mini" not found in configuration.', + }, + { + name: 'a mapped summary group whose base URL is user-provided', + agent: () => azureAstraAgent(), + summarizationConfig: { model: 'gpt-4.1' }, + env: { AZURE_OPENAI_BASEURL: 'user_provided' }, + reason: 'it needs a server-configured Azure OpenAI API key and base URL.', + }, + { + name: 'a mapped summary group with an empty API key', + agent: () => azureAstraAgent(), + summarizationConfig: { model: 'gpt-4.1-nano' }, + env: {}, + summaryApiKey: '', + reason: 'it needs a server-configured Azure OpenAI API key and base URL.', + }, + { + name: 'a legacy Azure environment whose key is user-provided', + agent: () => azureAstraAgent(), + summarizationConfig: { model: 'gpt-4o' }, + env: { AZURE_API_KEY: 'user_provided' }, + legacyEnvironment: true, + reason: 'it needs a server-configured Azure OpenAI API key and base URL.', + }, + ])('disables summarization for $name', async (target) => { + const { agent, summarizationConfig, env, reason } = target; + jest.replaceProperty(process, 'env', { ...process.env, ...env }); + const appConfig = makeAppConfig([]); + appConfig.endpoints![EModelEndpoint.azureOpenAI] = target.legacyEnvironment + ? undefined + : { + isValid: true, + errors: [], + modelNames: ['gpt-6-astra', 'gpt-4.1', 'gpt-4.1-nano'], + modelGroupMap: { + 'gpt-6-astra': { group: 'main' }, + 'gpt-4.1': { group: 'summary' }, + 'gpt-4.1-nano': { group: 'summary' }, + }, + groupMap: { + main: { + apiKey: 'test-azure-key', + instanceName: 'test-instance', + version: '2025-04-01-preview', + models: { 'gpt-6-astra': { deploymentName: 'production-deployment' } }, + }, + summary: { + apiKey: target.summaryApiKey ?? 'summary-key', + instanceName: 'summary-instance', + version: '2024-10-21', + models: { + 'gpt-4.1': { deploymentName: 'summary-production' }, + 'gpt-4.1-nano': { deploymentName: 'summary-nano' }, + }, + }, + }, + }; + const agents = await callAndCapture({ + agents: [agent()], + appConfig, + summarizeOnly: true, + summarizationConfig: { ...summarizationConfig, parameters: { streaming: false } }, + }); + + expect(agents[0].summarizationEnabled).toBe(false); + expect(logger.warn).toHaveBeenCalledWith( + `[createRun] Summarization with Azure OpenAI model "${summarizationConfig.model}" is disabled: ${reason}`, + ); + const requests: CapturedRequest[] = []; + await expect(compactSummary(agents, requests)).rejects.toThrow( + 'Compaction skipped: summarization is not enabled for this agent', + ); + expect(requests).toHaveLength(0); + }); + + it("sends a summary deployment to its own resource when only the agent's group sets a base URL", async () => { + const agentBaseURL = + 'https://agent-instance.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}'; + const appConfig = makeAppConfig([]); + appConfig.endpoints![EModelEndpoint.azureOpenAI] = { + isValid: true, + errors: [], + modelNames: ['gpt-4.1', 'gpt-4.1-mini'], + modelGroupMap: { 'gpt-4.1': { group: 'main' }, 'gpt-4.1-mini': { group: 'summary' } }, + groupMap: { + main: { + apiKey: 'test-azure-key', + instanceName: 'agent-instance', + baseURL: agentBaseURL, + version: '2024-10-21', + models: { 'gpt-4.1': { deploymentName: 'agent-deployment' } }, + }, + summary: { + apiKey: 'summary-key', + instanceName: 'summary-instance', + version: '2024-10-21', + models: { 'gpt-4.1-mini': { deploymentName: 'summary-production' } }, + }, + }, + }; + const { llmConfig, configOptions } = getOpenAIConfig( + 'test-azure-key', + { + reverseProxyUrl: agentBaseURL, + azure: { + azureOpenAIApiInstanceName: 'agent-instance', + azureOpenAIApiDeploymentName: 'agent-deployment', + azureOpenAIApiVersion: '2024-10-21', + azureOpenAIApiKey: 'test-azure-key', + }, + modelOptions: { model: 'gpt-4.1' }, + }, + EModelEndpoint.azureOpenAI, + ); + const agents = await callAndCapture({ + agents: [ + makeReasoningAgent({ + provider: EModelEndpoint.azureOpenAI, + endpoint: EModelEndpoint.azureOpenAI, + model: 'gpt-4.1', + model_parameters: { ...llmConfig, configuration: configOptions }, + }), + ], + appConfig, + summarizeOnly: true, + summarizationConfig: { model: 'gpt-4.1-mini', parameters: { streaming: false } }, + }); + + const { requests } = await compactSummary(agents); + expect(requests).toHaveLength(1); + const { url, headers, body } = requests[0]; + expect(url.origin + url.pathname).toBe( + 'https://summary-instance.openai.azure.com/openai/deployments/summary-production/chat/completions', + ); + expect(headers.get('api-key')).toBe('summary-key'); + expect(body.model).toBe('summary-production'); + }); + + it('sends no empty api-version to a serverless summary group without a version', async () => { + const appConfig = makeAppConfig([]); + appConfig.endpoints![EModelEndpoint.azureOpenAI] = { + isValid: true, + errors: [], + modelNames: ['gpt-6-astra', 'Phi-4'], + modelGroupMap: { 'gpt-6-astra': { group: 'main' }, 'Phi-4': { group: 'serverless' } }, + groupMap: { + main: { + apiKey: 'test-azure-key', + instanceName: 'test-instance', + version: '2025-04-01-preview', + models: { 'gpt-6-astra': { deploymentName: 'production-deployment' } }, + }, + serverless: { + apiKey: 'serverless-key', + baseURL: 'https://phi-instance.services.ai.azure.com/models', + serverless: true, + models: { 'Phi-4': true }, + }, + }, + }; + const agents = await callAndCapture({ + agents: [azureAstraAgent()], + appConfig, + summarizeOnly: true, + summarizationConfig: { model: 'Phi-4', parameters: { streaming: false } }, + }); + + const { requests } = await compactSummary(agents); + expect(requests).toHaveLength(1); + const { url, headers } = requests[0]; + expect(url.origin + url.pathname).toBe( + 'https://phi-instance.services.ai.azure.com/models/chat/completions', + ); + expect(url.search).toBe(''); + expect(headers.get('api-key')).toBe('serverless-key'); + }); + + it.each([ + { summaryModel: 'gpt-4.1-mini', instance: 'test-instance', provider: undefined }, + { + summaryModel: 'gpt-4.1-mini', + instance: 'summary-instance', + provider: EModelEndpoint.azureOpenAI, + }, + { summaryModel: 'gpt-6-astra-2026-09-03', instance: 'summary-instance', provider: undefined }, + ])( + 'compacts with $summaryModel on $instance using its configured deployment', + async ({ summaryModel, instance, provider }) => { + const appConfig = makeAppConfig([]); + appConfig.endpoints![EModelEndpoint.azureOpenAI] = { + isValid: true, + errors: [], + modelNames: ['gpt-6-astra', summaryModel], + modelGroupMap: { 'gpt-6-astra': { group: 'main' }, [summaryModel]: { group: 'summary' } }, + groupMap: { + main: { + apiKey: 'test-azure-key', + instanceName: 'test-instance', + version: '2025-04-01-preview', + models: { 'gpt-6-astra': { deploymentName: 'production-deployment' } }, + }, + summary: { + apiKey: 'summary-key', + instanceName: instance, + baseURL: `https://${instance}.openai.azure.com`, + version: '2025-04-01-preview', + additionalHeaders: { 'X-Summary-Group': 'summary' }, + models: { [summaryModel]: { deploymentName: 'summary-production' } }, + }, + }, + }; + const agents = await callAndCapture({ + agents: [azureAstraAgent()], + appConfig, + summarizeOnly: true, + summarizationConfig: { model: summaryModel, provider, parameters: { streaming: false } }, + }); + const { requests } = await compactSummary(agents); + expect(requests).toHaveLength(1); + const { url, headers, body } = requests[0]; + const usesResponses = summaryModel.startsWith('gpt-6-astra'); + expect(url.origin).toBe(`https://${instance}.openai.azure.com`); + expect(url.pathname).toBe( + usesResponses + ? '/openai/v1/responses' + : '/openai/deployments/summary-production/chat/completions', + ); + expect(headers.get('api-key')).toBe('summary-key'); + expect(headers.get('X-Summary-Group')).toBe('summary'); + expect(body.model).toBe('summary-production'); + expect(body).not.toHaveProperty('max_output_tokens', 2048); + expect((agents[0].clientOptions as OpenAIConfiguration)?.apiKey).toBe('test-azure-key'); + }, + ); +}); + // --------------------------------------------------------------------------- // Suite 5: Multi-agent + per-agent overrides // --------------------------------------------------------------------------- diff --git a/packages/api/src/agents/initialize.ts b/packages/api/src/agents/initialize.ts index c1e720392e..f36ffc49f5 100644 --- a/packages/api/src/agents/initialize.ts +++ b/packages/api/src/agents/initialize.ts @@ -605,6 +605,8 @@ function resolveProviderToolConflicts({ * Extended agent type with additional fields needed after initialization */ export type InitializedAgent = Agent & { + /** Request-resolved Azure identity for self-summarization; never persisted on the agent. */ + azureOptions?: InitializeResultBase['azureOptions']; tools: GenericTool[]; /** @deprecated use requestAttachments or agentContextAttachments based on sharing semantics. */ attachments: IMongoFile[]; @@ -2363,6 +2365,7 @@ export async function initializeAgent( const initializedAgent: InitializedAgent = { ...agent, + azureOptions: options.azureOptions, resendFiles, imageDetail, toolRegistry, diff --git a/packages/api/src/agents/run.ts b/packages/api/src/agents/run.ts index 2c95d4d5e6..c6287521cc 100644 --- a/packages/api/src/agents/run.ts +++ b/packages/api/src/agents/run.ts @@ -11,6 +11,7 @@ import { extractEnvVariable, providerEndpointMap, normalizeEndpointName, + mapModelToAzureConfig, resolveUseResponsesApi, } from 'librechat-data-provider'; import type { @@ -37,6 +38,7 @@ import type { import type { Agent, ImageDetail, + TAzureConfig, CodeApprovalMode, TAgentsEndpoint, AgentModelParameters, @@ -106,11 +108,12 @@ import { } from '~/agents/config'; import { applyCustomHandoffPromptKeyCompatibility } from '~/agents/handoffPromptKeyCompatibility'; import { stripIntentFromToolRegistry, stripIntentFromToolDefinitions } from '~/agents/intent'; +import { resolveConfigHeaders, resolveModelHeaders, mergeHeaders } from '~/utils/headers'; import { extractDefaultParams, resolveReasoningParams } from '~/endpoints/openai/llm'; import { getLLMConfig as getAnthropicLLMConfig } from '~/endpoints/anthropic/llm'; import { CREATE_FILE_TOOL_NAME, EDIT_FILE_TOOL_NAME } from '~/agents/tools'; -import { resolveConfigHeaders, resolveModelHeaders } from '~/utils/headers'; import { buildAgentInitialToolSessions } from '~/agents/codeFilesSession'; +import { getAzureCredentials, constructAzureURL } from '~/utils/azure'; import { getBuiltInBaseURL } from '~/endpoints/openai/initialize'; import { getProviderConfig } from '~/endpoints/config/providers'; import { buildToolApprovalHooks } from '~/agents/hitl/hooks'; @@ -413,6 +416,7 @@ export function shouldReplayReasoningContent( } type RunAgent = Omit & { + azureOptions?: t.AzureOptions; tools?: GenericTool[]; maxContextTokens?: number; /** Pre-ratio context budget from initializeAgent. */ @@ -577,20 +581,68 @@ function normalizeAgentModelParameters( * Merges user-supplied summarization parameters on top of endpoint-resolved * overrides. User params win for top-level keys; `configuration` is * deep-merged so user additions (e.g. `defaultQuery`) don't wipe out the - * resolved `baseURL`/`defaultHeaders`/`fetchOptions`. + * resolved `baseURL`/`defaultHeaders`/`fetchOptions`. When transport resolution already + * consumed the user's URL, retain its normalized form instead of restoring the raw template. */ function mergeParameters( overrides: SummarizationClientOverrides, userParams: SummarizationConfig['parameters'], + resolvedTransport = false, ): Record { const merged: Record = { ...overrides, ...(userParams ?? {}) }; const userConfiguration = (userParams as Record | undefined)?.configuration; if (isPlainObject(overrides.configuration) && isPlainObject(userConfiguration)) { - merged.configuration = { ...overrides.configuration, ...userConfiguration }; + merged.configuration = { + ...overrides.configuration, + ...userConfiguration, + ...(resolvedTransport ? { baseURL: overrides.configuration.baseURL } : {}), + defaultHeaders: mergeHeaders( + overrides.configuration.defaultHeaders as Record | undefined, + userConfiguration.defaultHeaders as Record | undefined, + ), + defaultQuery: { + ...(isPlainObject(overrides.configuration.defaultQuery) + ? overrides.configuration.defaultQuery + : {}), + ...(isPlainObject(userConfiguration.defaultQuery) ? userConfiguration.defaultQuery : {}), + }, + }; } return merged; } +/** `model_parameters` is the agent's resolved `llmConfig`, whose kwargs the schema type omits. */ +function agentModelKwargs( + modelParameters: AgentModelParameters | undefined, +): Record | undefined { + if (modelParameters == null || !('modelKwargs' in modelParameters)) { + return undefined; + } + return isPlainObject(modelParameters.modelKwargs) ? modelParameters.modelKwargs : undefined; +} + +/** + * `getOpenAILLMConfig` carries an Azure Astra deployment alias in + * `modelKwargs.model`, which langchain spreads after `model`. The SDK's + * same-provider summarizer copies the agent's client options and overrides only + * `model`, so a summarizer on another model would still reach the agent's + * deployment. Hand it the agent's kwargs without the alias. + */ +function summarizationModelKwargs( + agentKwargs: Record | undefined, + agentModel: string | undefined, + summarizationModel: string | undefined, +): Record | undefined { + if (agentKwargs == null || !('model' in agentKwargs)) { + return undefined; + } + if (!isNonEmptyString(summarizationModel) || summarizationModel === agentModel) { + return undefined; + } + const { model: _alias, ...kwargs } = agentKwargs; + return kwargs; +} + /** * Mirrors `getOpenAIConfig`'s `llmConfig` shape (plus its `configOptions` * assigned to `configuration`). Index signature covers fields that the @@ -701,6 +753,370 @@ function resolveBuiltInClientOverrides( return Object.keys(shaping).length > 0 ? shaping : undefined; } +/** + * Memory bound for the warning deduplication below, not an operator setting: it only decides when a + * warning for one of more than this many distinct, concurrently recurring problems is logged again. + */ +const MAX_UNRESOLVED_SUMMARIZATION_WARNINGS = 256; +/** Reported summarization misconfigurations per tenant, least recently seen first. */ +const unresolvedSummarizationWarnings = new Set(); + +function warnUnresolvedSummarization(message: string, tenantId?: string): void { + const key = `${tenantId ?? ''}\n${message}`; + if (unresolvedSummarizationWarnings.delete(key)) { + unresolvedSummarizationWarnings.add(key); + return; + } + if (unresolvedSummarizationWarnings.size >= MAX_UNRESOLVED_SUMMARIZATION_WARNINGS) { + const [leastRecentlySeen] = unresolvedSummarizationWarnings; + unresolvedSummarizationWarnings.delete(leastRecentlySeen); + } + unresolvedSummarizationWarnings.add(key); + if (tenantId == null) { + logger.warn(`[createRun] ${message}`); + return; + } + logger.warn(`[createRun] ${message}`, { tenantId }); +} + +/** Azure base URL templates the client fills from its own options rather than the environment. */ +const AZURE_URL_TEMPLATE = /(\$\{(?:INSTANCE_NAME|DEPLOYMENT_NAME)\})/; + +/** The URL's segments around Azure's reserved templates, which sit at the odd indexes. */ +function splitAzureURLTemplates(url: string): string[] { + return url.split(AZURE_URL_TEMPLATE); +} + +/** Expands environment references in a URL, leaving Azure's reserved templates for an Azure client. */ +function expandTransportURL(url: string, targetsAzure: boolean): string { + if (!targetsAzure) { + return extractEnvVariable(url); + } + return splitAzureURLTemplates(url) + .map((segment, index) => (index % 2 === 1 ? segment : extractEnvVariable(segment))) + .join(''); +} + +/** + * Admin-authored transport values may reference environment variables, as endpoint credentials + * do. Expanded once, before resolution and before the parameters are layered over the client. + */ +function expandSummarizationTransport( + parameters: SummarizationConfig['parameters'], + targetsAzure: boolean, +): SummarizationConfig['parameters'] { + if (!isPlainObject(parameters)) { + return parameters; + } + const params = parameters as Record; + const expanded: Record = { ...params }; + if (typeof params.apiKey === 'string') { + expanded.apiKey = extractEnvVariable(params.apiKey); + } + if (typeof params.baseURL === 'string') { + expanded.baseURL = expandTransportURL(params.baseURL, targetsAzure); + } + if (isPlainObject(params.configuration) && typeof params.configuration.baseURL === 'string') { + expanded.configuration = { + ...params.configuration, + baseURL: expandTransportURL(params.configuration.baseURL, targetsAzure), + }; + } + return expanded as SummarizationConfig['parameters']; +} + +/** The base URL and API key a summarization target's own parameters set, which replace the resolved ones. */ +function summarizationTransportOverrides(parameters: SummarizationConfig['parameters']): { + baseURL?: string; + apiKey?: string; +} { + const configuration = (parameters as Record | undefined)?.configuration; + const configurationBaseURL = + isPlainObject(configuration) && typeof configuration.baseURL === 'string' + ? configuration.baseURL + : undefined; + return { + baseURL: + configurationBaseURL ?? + (typeof parameters?.baseURL === 'string' ? parameters.baseURL : undefined), + apiKey: typeof parameters?.apiKey === 'string' ? parameters.apiKey : undefined, + }; +} + +/** Reject unusable transports while shaping the run, before a compaction client can fail it. */ +function isSummarizationURLAvailable(baseURL: string | undefined, azure?: t.AzureOptions): boolean { + if (baseURL == null) { + return azure == null || isNonEmptyString(azure.azureOpenAIApiInstanceName); + } + try { + const url = new URL(constructAzureURL({ baseURL, azureOptions: azure })); + return ( + (url.protocol === 'https:' || url.protocol === 'http:') && !hasUnresolvedPlaceholder(url.href) + ); + } catch { + return false; + } +} + +/** + * Azure Responses shares OpenAI's SDK provider, so switching endpoints must replace its transport. + * Returns `undefined` when the OpenAI credentials cannot be resolved without a per-user lookup. + */ +function resolveOpenAISummarization( + model: string, + appConfig: AppConfig | undefined, + parameters: SummarizationConfig['parameters'], + headerContext: { user?: IUser; tenantId?: string; requestBody?: t.RequestBody }, +): { provider: string; clientOverrides: SummarizationClientOverrides } | undefined { + const overrides = summarizationTransportOverrides(parameters); + const baseURL = overrides.baseURL ?? getBuiltInBaseURL(EModelEndpoint.openAI); + const apiKey = overrides.apiKey ?? process.env.OPENAI_API_KEY; + if ( + !apiKey || + !isSummarizationURLAvailable(baseURL) || + isUserProvided(baseURL) || + isUserProvided(apiKey) || + hasUnresolvedPlaceholder(apiKey) || + (baseURL != null && hasUnresolvedPlaceholder(baseURL)) + ) { + warnUnresolvedSummarization( + `Summarization with OpenAI model "${model}" is disabled for Azure OpenAI agents: it needs a server-configured OpenAI API key and base URL.`, + headerContext.tenantId, + ); + return undefined; + } + const headers = mergeHeaders( + appConfig?.endpoints?.all?.headers, + appConfig?.endpoints?.openAI?.headers, + ); + const { llmConfig, configOptions } = getOpenAIConfig( + apiKey, + { + modelOptions: { + model, + reasoning_effort: summarizationReasoningEffort(parameters), + useResponsesApi: + typeof parameters?.useResponsesApi === 'boolean' ? parameters.useResponsesApi : undefined, + }, + reverseProxyUrl: baseURL, + proxy: process.env.PROXY ?? undefined, + headers: resolveModelHeaders({ + headers: headers ?? {}, + user: createSafeUser(headerContext.user), + tenantId: headerContext.tenantId, + body: headerContext.requestBody, + }), + }, + EModelEndpoint.openAI, + ); + return { + provider: Providers.OPENAI, + clientOverrides: { + ...llmConfig, + apiKey, + useResponsesApi: llmConfig.useResponsesApi ?? false, + firstPartyEndpoint: llmConfig.firstPartyEndpoint ?? false, + reasoning: llmConfig.reasoning, + modelKwargs: llmConfig.modelKwargs ?? {}, + configuration: configOptions, + }, + }; +} + +type AzureSummarizationTarget = Omit, 'azureOptions'> & { + azureOptions: t.AzureOptions; + group?: TAzureConfig['groupMap'][string]; +}; + +/** + * The summary model's Azure credentials, resolved the way `initializeOpenAI` resolves an agent's: + * the `azureOpenAI` configuration when present, otherwise the legacy environment credentials. + * Returns `undefined` (reported once) when the configuration does not define the model. + */ +function resolveAzureSummarizationTarget( + model: string, + azureConfig: TAzureConfig | undefined, + tenantId: string | undefined, +): AzureSummarizationTarget | undefined { + if (!azureConfig) { + return { azureOptions: getAzureCredentials() }; + } + try { + const groupName = azureConfig.modelGroupMap[model]?.group; + return { + ...mapModelToAzureConfig({ + modelName: model, + modelGroupMap: azureConfig.modelGroupMap, + groupMap: azureConfig.groupMap, + }), + group: groupName ? azureConfig.groupMap[groupName] : undefined, + }; + } catch (error) { + warnUnresolvedSummarization( + `Summarization with Azure OpenAI model "${model}" is disabled: ${(error as Error).message}`, + tenantId, + ); + return undefined; + } +} + +/** Reuse request-resolved credentials for self-summaries, including user-provided Azure keys. */ +function azureSummarizationSource(agent: RunAgent): AzureSummarizationTarget { + const options = agent.model_parameters as Partial & t.AzureOptions; + const configuration = options.configuration; + const baseURL = configuration?.baseURL ?? options.azureOpenAIBasePath; + const deployment = agentModelKwargs(agent.model_parameters)?.model; + return { + serverless: agent.useLegacyContent === true, + baseURL: baseURL ?? undefined, + azureOptions: { + azureOpenAIApiKey: + options.azureOpenAIApiKey ?? + (typeof options.apiKey === 'string' ? options.apiKey : undefined), + azureOpenAIApiInstanceName: + options.azureOpenAIApiInstanceName ?? agent.azureOptions?.azureOpenAIApiInstanceName, + azureOpenAIApiDeploymentName: + options.azureOpenAIApiDeploymentName ?? + (typeof deployment === 'string' ? deployment : options.model), + azureOpenAIApiVersion: + options.azureOpenAIApiVersion ?? + agent.azureOptions?.azureOpenAIApiVersion ?? + configuration?.defaultQuery?.['api-version'] ?? + undefined, + }, + }; +} + +/** + * Resolve the summary model's deployment and transport before the SDK inherits agent options. + * A same-provider summarizer layers these over the agent's client options, so a summary group + * without a base path clears the agent's instead of sending its deployment to that resource. + * Returns `undefined` when the model cannot be resolved without a per-user lookup. + */ +function resolveAzureSummarization( + model: string, + appConfig: AppConfig | undefined, + parameters: SummarizationConfig['parameters'], + headerContext: { user?: IUser; tenantId?: string; requestBody?: t.RequestBody }, + source?: RunAgent, +): { provider: string; clientOverrides: SummarizationClientOverrides } | undefined { + const sourceOptions = source?.model_parameters as Partial | undefined; + const target = source + ? azureSummarizationSource(source) + : resolveAzureSummarizationTarget( + model, + appConfig?.endpoints?.[EModelEndpoint.azureOpenAI], + headerContext.tenantId, + ); + if (!target) { + return undefined; + } + const { baseURL, headers, serverless, group } = target; + const overrides = summarizationTransportOverrides(parameters); + const azureOptions: t.AzureOptions = { + ...target.azureOptions, + azureOpenAIApiKey: overrides.apiKey ?? target.azureOptions.azureOpenAIApiKey, + }; + const resolvedBaseURL = + overrides.baseURL ?? baseURL ?? getBuiltInBaseURL(EModelEndpoint.azureOpenAI); + if ( + !azureOptions.azureOpenAIApiKey || + !isSummarizationURLAvailable(resolvedBaseURL, serverless ? undefined : azureOptions) || + Object.values(azureOptions).some( + (value) => + typeof value === 'string' && (isUserProvided(value) || hasUnresolvedPlaceholder(value)), + ) || + isUserProvided(resolvedBaseURL) || + (resolvedBaseURL != null && + splitAzureURLTemplates(resolvedBaseURL).some( + (segment, index) => index % 2 === 0 && hasUnresolvedPlaceholder(segment), + )) + ) { + warnUnresolvedSummarization( + `Summarization with Azure OpenAI model "${model}" is disabled: it needs a server-configured Azure OpenAI API key and base URL.`, + headerContext.tenantId, + ); + return undefined; + } + const resolvedHeaders = sourceOptions + ? (sourceOptions.configuration?.defaultHeaders as Record | undefined) + : resolveModelHeaders({ + headers: mergeHeaders(appConfig?.endpoints?.all?.headers, headers) ?? {}, + user: createSafeUser(headerContext.user), + tenantId: headerContext.tenantId, + body: headerContext.requestBody, + }); + const { llmConfig, configOptions } = getOpenAIConfig( + azureOptions.azureOpenAIApiKey, + { + azure: serverless ? undefined : azureOptions, + reverseProxyUrl: resolvedBaseURL, + proxy: process.env.PROXY ?? undefined, + headers: + serverless || sourceOptions != null + ? { ...resolvedHeaders, 'api-key': azureOptions.azureOpenAIApiKey } + : resolvedHeaders, + defaultQuery: + serverless && azureOptions.azureOpenAIApiVersion + ? { 'api-version': azureOptions.azureOpenAIApiVersion } + : undefined, + modelOptions: { + model: source?.model ?? model, + reasoning_effort: summarizationReasoningEffort(parameters), + useResponsesApi: + typeof parameters?.useResponsesApi === 'boolean' + ? parameters.useResponsesApi + : sourceOptions?.useResponsesApi, + }, + addParams: { + ...group?.addParams, + ...(typeof parameters?.useResponsesApi === 'boolean' + ? { useResponsesApi: parameters.useResponsesApi } + : {}), + }, + dropParams: group?.dropParams?.filter( + (key) => key !== 'useResponsesApi' || typeof parameters?.useResponsesApi !== 'boolean', + ), + }, + EModelEndpoint.azureOpenAI, + ); + const sourceKwargs = agentModelKwargs(source?.model_parameters); + const preservesApiMode = !!sourceOptions?.useResponsesApi === !!llmConfig.useResponsesApi; + const sourceTokenLimit = sourceKwargs?.max_output_tokens ?? sourceKwargs?.max_completion_tokens; + const inheritedTokenLimit = + typeof sourceTokenLimit === 'number' + ? { + [llmConfig.useResponsesApi ? 'max_output_tokens' : 'max_completion_tokens']: + sourceTokenLimit, + } + : {}; + return { + provider: !serverless && !llmConfig.useResponsesApi ? Providers.AZURE : Providers.OPENAI, + clientOverrides: { + ...sourceOptions, + azureOpenAIBasePath: undefined, + azureOpenAIApiKey: undefined, + azureOpenAIApiInstanceName: undefined, + azureOpenAIApiDeploymentName: undefined, + azureOpenAIApiVersion: undefined, + ...llmConfig, + apiKey: azureOptions.azureOpenAIApiKey, + useResponsesApi: llmConfig.useResponsesApi ?? false, + firstPartyEndpoint: llmConfig.firstPartyEndpoint ?? false, + reasoning: llmConfig.reasoning ?? (preservesApiMode ? sourceOptions?.reasoning : undefined), + modelKwargs: { + ...(preservesApiMode ? sourceKwargs : undefined), + ...inheritedTokenLimit, + ...llmConfig.modelKwargs, + model: llmConfig.modelKwargs?.model ?? llmConfig.model, + }, + configuration: sourceOptions + ? { ...sourceOptions.configuration, ...configOptions } + : configOptions, + }, + }; +} + /** * Resolves a summarization provider string (which may be a custom-endpoint name * like "Ollama") into the SDK-recognized provider and any client-option @@ -861,10 +1277,19 @@ function resolveSummarizationProvider( * agent flow builds for that endpoint (`initializeAgent` applies the same * precedence). */ - return { - provider: detectedProvider ?? overrideProvider, - clientOverrides, - }; + const provider = detectedProvider ?? overrideProvider; + /** + * On the agent's provider the SDK layers these over the agent's own client options, so this + * different endpoint replaces the agent's API mode, first-party declaration, reasoning and + * request kwargs (an Azure Astra agent's Responses routing, for one) instead of inheriting them. + */ + if (provider === target.agentProvider) { + clientOverrides.useResponsesApi ??= false; + clientOverrides.firstPartyEndpoint ??= false; + clientOverrides.modelKwargs ??= {}; + clientOverrides.reasoning ??= undefined; + } + return { provider, clientOverrides }; } catch (error) { logger.warn( `[resolveSummarizationProvider] failed to resolve "${rawProvider}"; falling back to raw provider`, @@ -882,6 +1307,7 @@ function shapeSummarizationConfig( appConfig: AppConfig | undefined, agentEndpoint: string | undefined, headerContext: { user?: IUser; tenantId?: string; requestBody?: t.RequestBody }, + agent?: RunAgent, ) { const rawProvider = config?.provider ?? fallbackProvider; /** @@ -897,14 +1323,61 @@ function shapeSummarizationConfig( normalizeEndpointName(rawProvider) === normalizeEndpointName(agentEndpoint); const model = config?.model ?? fallbackModel; + const targetsAzure = + rawProvider === EModelEndpoint.azureOpenAI || + (agentEndpoint === EModelEndpoint.azureOpenAI && config?.provider == null); + const userParameters = expandSummarizationTransport(config?.parameters, targetsAzure); - const { provider, clientOverrides } = isSameEndpointAsAgent - ? { provider: fallbackProvider, clientOverrides: undefined } - : resolveSummarizationProvider(rawProvider, appConfig, headerContext, { + const selfAzureModel = + agentEndpoint === EModelEndpoint.azureOpenAI && + (model === fallbackModel || model === agent?.model); + const transportOverrides = summarizationTransportOverrides(userParameters); + const overridesAzureTransport = + transportOverrides.baseURL != null || + transportOverrides.apiKey != null || + typeof userParameters?.useResponsesApi === 'boolean'; + const selectsAzureDeployment = + targetsAzure && + isNonEmptyString(model) && + config?.enabled !== false && + (agentEndpoint !== EModelEndpoint.azureOpenAI || + model !== fallbackModel || + overridesAzureTransport); + const azureOverrides = selectsAzureDeployment + ? resolveAzureSummarization( model, - parameters: config?.parameters, - agentProvider: fallbackProvider, - }); + appConfig, + userParameters, + headerContext, + selfAzureModel ? agent : undefined, + ) + : undefined; + const selectsOpenAIForAzureAgent = + agentEndpoint === EModelEndpoint.azureOpenAI && + config?.provider === EModelEndpoint.openAI && + config.enabled !== false && + isNonEmptyString(model); + const openAIOverrides = selectsOpenAIForAzureAgent + ? resolveOpenAISummarization(model, appConfig, userParameters, headerContext) + : undefined; + /** + * A target resolved here is not handed to another client when resolution fails. Azure Responses + * shares the `openAI` provider, so the SDK would summarize an OpenAI target through the agent's + * Azure resource, and would send an unmapped Azure model name where a deployment belongs. + */ + const targetUnavailable = + (selectsAzureDeployment && azureOverrides == null) || + (selectsOpenAIForAzureAgent && openAIOverrides == null); + const { provider, clientOverrides } = + openAIOverrides ?? + azureOverrides ?? + (isSameEndpointAsAgent + ? { provider: fallbackProvider, clientOverrides: undefined } + : resolveSummarizationProvider(rawProvider, appConfig, headerContext, { + model, + parameters: userParameters, + agentProvider: fallbackProvider, + })); const trigger = config?.trigger?.type && typeof config?.trigger?.value === 'number' ? { type: config.trigger.type, value: config.trigger.value } @@ -913,9 +1386,8 @@ function shapeSummarizationConfig( /** * Custom-endpoint overrides are merged into `parameters` so the SDK's * `buildSummarizationClientConfig` spreads them onto the summarization - * client options. Only applied when summarization targets a *different* - * custom endpoint than the main agent; the same-endpoint case leaves - * `parameters` untouched so `agentContext.clientOptions` wins. + * client options. Azure self-summaries with explicit transport overrides also resolve here, + * using the initialized agent's identity and credentials instead of a new configuration lookup. * * Order matters: `clientOverrides` supplies endpoint defaults (baseURL, * apiKey, headers, transforms), then explicit user `summarization.parameters` @@ -926,18 +1398,56 @@ function shapeSummarizationConfig( */ const mergedParameters = clientOverrides != null - ? mergeParameters(clientOverrides, config?.parameters) - : config?.parameters; + ? mergeParameters( + clientOverrides, + userParameters, + azureOverrides != null || openAIOverrides != null, + ) + : userParameters; + /** Placed first so an explicit user `modelKwargs` still replaces the agent's wholesale. */ + const modelKwargs = + provider === fallbackProvider + ? summarizationModelKwargs(agentModelKwargs(agent?.model_parameters), fallbackModel, model) + : undefined; /** * A scalar `reasoning_effort` โ€” the only reasoning shape the yaml schema * accepts โ€” is inert as a client option and leaves the summarizer running at * whatever effort the main agent resolved. Translate it the way the main * flow's `getOpenAIConfig` would for the summarization target. */ - const parameters = resolveReasoningParams({ provider, model, parameters: mergedParameters }); + let parameters = resolveReasoningParams({ + provider, + model, + parameters: modelKwargs != null ? { modelKwargs, ...mergedParameters } : mergedParameters, + }); + + /** The SDK sets maxTokens for this cap, but LangChain spreads modelKwargs after it. */ + const parameterTokenCap = userParameters?.maxSummaryTokens; + const summaryTokenCap = + typeof parameterTokenCap === 'number' && parameterTokenCap > 0 + ? parameterTokenCap + : config?.maxSummaryTokens; + const inheritedKwargs = + provider === fallbackProvider ? agentModelKwargs(agent?.model_parameters) : undefined; + const effectiveKwargs = isPlainObject(parameters?.modelKwargs) + ? parameters.modelKwargs + : inheritedKwargs; + if (typeof summaryTokenCap === 'number' && effectiveKwargs != null) { + const { + max_tokens: _maxTokens, + max_completion_tokens: _maxCompletionTokens, + max_output_tokens: _maxOutputTokens, + ...kwargs + } = effectiveKwargs; + parameters = { ...parameters, modelKwargs: kwargs }; + } return { - enabled: config?.enabled !== false && isNonEmptyString(provider) && isNonEmptyString(model), + enabled: + !targetUnavailable && + config?.enabled !== false && + isNonEmptyString(provider) && + isNonEmptyString(model), config: { trigger, provider, @@ -1820,6 +2330,21 @@ export async function createRun({ ] as unknown as Providers) ?? agent.provider; const selfModel = agent.model_parameters?.model ?? (agent.model as string | undefined); + /** + * Resolve request-based headers across provider-specific header locations + * (OpenAI `configuration.defaultHeaders`, Anthropic `clientOptions.defaultHeaders`, + * Google `customHeaders`). Done at this step because the request body may + * contain dynamic values (e.g. conversationId) that are only known after + * agent initialization. Resolve before a self-summary snapshots this configuration. + */ + resolveConfigHeaders({ + llmConfig: agent.model_parameters as Partial, + user: createSafeUser(user), + tenantId, + body: requestBody, + }); + + const modelParameters = normalizeAgentModelParameters(agent.model_parameters); const shapedSummarization = shapeSummarizationConfig( agent.summarization ?? summarizationConfig, provider as string, @@ -1827,6 +2352,7 @@ export async function createRun({ appConfig, agent.endpoint ?? undefined, { user, tenantId, requestBody }, + agent, ); const summarization = modelCallbacks?.length ? { @@ -1841,7 +2367,6 @@ export async function createRun({ } : shapedSummarization; - const modelParameters = normalizeAgentModelParameters(agent.model_parameters); const hasExplicitStreamUsage = Object.prototype.hasOwnProperty.call( modelParameters ?? {}, 'streamUsage', @@ -1873,20 +2398,6 @@ export async function createRun({ .join('\n') .trim(); - /** - * Resolve request-based headers across provider-specific header locations - * (OpenAI `configuration.defaultHeaders`, Anthropic `clientOptions.defaultHeaders`, - * Google `customHeaders`). Done at this step because the request body may - * contain dynamic values (e.g. conversationId) that are only known after - * agent initialization. - */ - resolveConfigHeaders({ - llmConfig, - user: createSafeUser(user), - tenantId, - body: requestBody, - }); - /** Resolves issues with new OpenAI usage field */ if ( customProviders.has(agent.provider) || diff --git a/packages/api/src/endpoints/openai/config.backward-compat.spec.ts b/packages/api/src/endpoints/openai/config.backward-compat.spec.ts index f89482f534..3f905ba21a 100644 --- a/packages/api/src/endpoints/openai/config.backward-compat.spec.ts +++ b/packages/api/src/endpoints/openai/config.backward-compat.spec.ts @@ -165,6 +165,7 @@ describe('getOpenAIConfig - Backward Compatibility', () => { streaming: true, model: 'gpt-5', useResponsesApi: true, + firstPartyEndpoint: true, user: 'some_user_id', apiKey: 'some_azure_key', reasoning: { diff --git a/packages/api/src/endpoints/openai/config.spec.ts b/packages/api/src/endpoints/openai/config.spec.ts index 15f81ef034..980efafb0b 100644 --- a/packages/api/src/endpoints/openai/config.spec.ts +++ b/packages/api/src/endpoints/openai/config.spec.ts @@ -909,9 +909,9 @@ describe('getOpenAIConfig', () => { reverseProxyUrl: 'https://${INSTANCE_NAME}.openai.azure.com/openai/v1', }); - // The constructAzureURL should replace placeholders with actual values + // AzureChatOpenAI appends the deployment to its base path, including for a v1 resource URL. expect((result.llmConfig as Record).azureOpenAIBasePath).toBe( - 'https://test-instance.openai.azure.com/openai/v1', + 'https://test-instance.openai.azure.com/openai/deployments', ); }); diff --git a/packages/api/src/endpoints/openai/config.ts b/packages/api/src/endpoints/openai/config.ts index ae2310de45..74aeb1118d 100644 --- a/packages/api/src/endpoints/openai/config.ts +++ b/packages/api/src/endpoints/openai/config.ts @@ -7,9 +7,9 @@ import { getGoogleConfig, stripGeminiFlashBlockedParams } from '~/endpoints/goog import { getLLMConfig as getAnthropicLLMConfig } from '~/endpoints/anthropic/llm'; import { createSSRFSafeAgents, createSSRFSafeUndiciConnect } from '~/auth'; import { getOpenAILLMConfig, extractDefaultParams } from './llm'; +import { constructAzureResponsesURL } from '~/utils/azure'; import { transformToOpenAIConfig } from './transform'; import { getProxyDispatcher } from '~/utils/proxy'; -import { constructAzureURL } from '~/utils/azure'; import { createFetch } from '~/utils/generators'; import { mergeHeaders } from '~/utils/headers'; @@ -270,20 +270,21 @@ export function getOpenAIConfig( return; } - const updatedUrl = configOptions.baseURL?.replace(/\/deployments(?:\/.*)?$/, '/v1'); - - configOptions.baseURL = constructAzureURL({ - baseURL: updatedUrl || 'https://${INSTANCE_NAME}.openai.azure.com/openai/v1', - azureOptions: azure, - }); + const responsesURL = constructAzureResponsesURL(configOptions.baseURL, azure); + const urlQuery = Object.fromEntries(responsesURL.searchParams); + responsesURL.search = ''; + responsesURL.hash = ''; + configOptions.baseURL = responsesURL.toString(); configOptions.defaultHeaders = { ...configOptions.defaultHeaders, 'api-key': apiKey, }; configOptions.defaultQuery = { + ...urlQuery, ...configOptions.defaultQuery, - 'api-version': configOptions.defaultQuery?.['api-version'] ?? 'preview', + 'api-version': + configOptions.defaultQuery?.['api-version'] ?? urlQuery['api-version'] ?? 'preview', }; }; diff --git a/packages/api/src/endpoints/openai/initialize.ts b/packages/api/src/endpoints/openai/initialize.ts index 7df6e1b996..16d412ed34 100644 --- a/packages/api/src/endpoints/openai/initialize.ts +++ b/packages/api/src/endpoints/openai/initialize.ts @@ -201,7 +201,10 @@ export async function initializeOpenAI( modelOptions, }; - const options = getOpenAIConfig(apiKey, finalClientOptions, endpoint); + const options: InitializeResultBase = getOpenAIConfig(apiKey, finalClientOptions, endpoint); + if (clientOptions.azure) { + options.azureOptions = { ...clientOptions.azure }; + } /** Set useLegacyContent for Azure serverless deployments */ if (isServerless) { diff --git a/packages/api/src/endpoints/openai/llm.spec.ts b/packages/api/src/endpoints/openai/llm.spec.ts index faa5edc25c..c422ce8683 100644 --- a/packages/api/src/endpoints/openai/llm.spec.ts +++ b/packages/api/src/endpoints/openai/llm.spec.ts @@ -801,12 +801,7 @@ describe('getOpenAILLMConfig', () => { ); }); - /** - * Astra is not documented as available on Azure OpenAI, and Azure's - * first-party hosts do not satisfy the OpenAI-host check, so declaring it - * there would claim a surface this cannot verify. - */ - it('does not declare it for Azure OpenAI', () => { + it('does not declare it for Azure serverless without Azure OpenAI configuration', () => { expect(configFor({ endpoint: EModelEndpoint.azureOpenAI }).llmConfig).not.toHaveProperty( 'firstPartyEndpoint', ); @@ -825,6 +820,90 @@ describe('getOpenAILLMConfig', () => { }); }); + describe('Azure Astra routing', () => { + const azure = { + azureOpenAIApiInstanceName: 'test-instance', + azureOpenAIApiDeploymentName: 'production-deployment', + azureOpenAIApiVersion: '2025-04-01-preview', + azureOpenAIApiKey: 'test-api-key', + }; + const azureConfig = (overrides: Partial[0]> = {}) => + getOpenAILLMConfig({ + azure, + apiKey: 'test-api-key', + streaming: true, + endpoint: EModelEndpoint.azureOpenAI, + modelOptions: { model: 'gpt-6-astra', max_tokens: 2048 }, + ...overrides, + }); + + it.each([ + undefined, + 'https://${INSTANCE_NAME}.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}', + 'https://test-instance.openai.azure.com/openai/v1', + 'https://test-instance.services.ai.azure.com/openai/v1', + 'https://test-instance.cognitiveservices.azure.com/openai/v1', + 'https://test-instance.openai.azure.us/openai/v1', + 'https://test-instance.openai.azure.cn/openai/v1', + ])('routes Azure Astra to Responses at %s and retains its model identity', (baseURL) => { + expect(azureConfig({ baseURL }).llmConfig).toMatchObject({ + model: 'gpt-6-astra', + useResponsesApi: true, + firstPartyEndpoint: true, + modelKwargs: { model: 'production-deployment', max_output_tokens: 2048 }, + }); + }); + + it.each([ + { modelOptions: { model: 'gpt-6-astra-2026-09-03', max_tokens: 2048 } }, + { modelOptions: { model: 'gpt-4.1', max_tokens: 2048 }, addParams: { model: 'gpt-6-astra' } }, + { dropParams: ['reasoning_effort'] }, + { reasoningFormat: ReasoningParameterFormat.disabled }, + ])('routes using the effective model independently of reasoning: %j', (overrides) => { + expect(azureConfig(overrides).llmConfig.useResponsesApi).toBe(true); + }); + + it.each([ + { modelOptions: { model: 'gpt-6-astra', max_tokens: 2048, useResponsesApi: false } }, + { addParams: { useResponsesApi: false } }, + { dropParams: ['useResponsesApi'] }, + ])('keeps Astra constraints on an explicit Chat Completions opt-out: %j', (overrides) => { + const { llmConfig } = azureConfig(overrides); + expect(llmConfig.useResponsesApi).not.toBe(true); + expect(llmConfig).toMatchObject({ + model: 'gpt-6-astra', + firstPartyEndpoint: true, + azureOpenAIApiDeploymentName: 'production-deployment', + modelKwargs: { model: 'production-deployment', max_completion_tokens: 2048 }, + }); + }); + + it.each([ + { modelOptions: { model: 'gpt-5.4-mini' } }, + { addParams: { model: 'gpt-5.4-mini' } }, + ])('leaves other Azure models on the deployment name: %j', (overrides) => { + const { llmConfig } = azureConfig(overrides); + expect(llmConfig.useResponsesApi).not.toBe(true); + expect(llmConfig).toMatchObject({ + model: 'production-deployment', + firstPartyEndpoint: true, + }); + expect(llmConfig.modelKwargs?.model).toBeUndefined(); + }); + + it.each([ + 'https://gateway.internal/openai/v1', + 'https://test-instance.openai.azure.com.example.org/openai/v1', + 'https://openai.azure.com/openai/v1', + 'not-a-url', + ])('leaves gateway routing and request constraints unchanged: %s', (baseURL) => { + const { llmConfig } = azureConfig({ baseURL }); + expect(llmConfig.useResponsesApi).not.toBe(true); + expect(llmConfig.firstPartyEndpoint).toBeUndefined(); + expect(llmConfig.model).toBe('production-deployment'); + }); + }); + describe('GPT-5.6 Responses API Requirement', () => { it.each(['gpt-5.6-terra', 'gpt-5.6-luna', 'gpt-5.6-sol', 'gpt-5.6'])( 'should default to Responses API for %s when reasoning_effort is set', diff --git a/packages/api/src/endpoints/openai/llm.ts b/packages/api/src/endpoints/openai/llm.ts index 04f02e1810..9aa23258dd 100644 --- a/packages/api/src/endpoints/openai/llm.ts +++ b/packages/api/src/endpoints/openai/llm.ts @@ -11,7 +11,14 @@ import type { AzureOpenAIInput } from '@librechat/agents/langchain/openai'; import type { SettingDefinition } from 'librechat-data-provider'; import type { OpenAI } from 'openai'; import type * as t from '~/types'; -import { sanitizeModelName, constructAzureURL } from '~/utils/azure'; +import { + sanitizeModelName, + constructAzureURL, + isCanonicalAzureURL, + getAzureDeploymentName, + constructAzureChatBasePath, + constructAzureInstanceBasePath, +} from '~/utils/azure'; import { isEnabled } from '~/utils/common'; type OpenAILLMConfig = Omit, 'verbosity'> & @@ -138,8 +145,8 @@ const responsesApiRequiredPattern = /\bgpt-5\.6\b/; * Decided here rather than in the agents SDK at invocation time: the max-tokens * field below is shaped from `useResponsesApi`, so a later switch would send * `max_completion_tokens` to an endpoint expecting `max_output_tokens`. Config - * time is also the only place that knows the model before Azure replaces it - * with a deployment name. + * time is also where Azure decides whether the model keeps its identity, with + * the deployment as the wire model, or becomes the deployment name outright. * @see https://developers.openai.com/api/docs/guides/latest-model */ const responsesApiPreferredPattern = /^gpt-6-astra(?:-|$)/i; @@ -148,6 +155,16 @@ function prefersResponsesApi(model?: string): boolean { return typeof model === 'string' && responsesApiPreferredPattern.test(model); } +function isCanonicalAzureBaseURL(baseURL?: string | null, azure?: false | t.AzureOptions): boolean { + if (!azure) { + return false; + } + if (!baseURL) { + return true; + } + return isCanonicalAzureURL(constructAzureURL({ baseURL, azureOptions: azure })); +} + function requiresResponsesApiForReasoning({ model, reasoningEffort, @@ -874,10 +891,17 @@ export function getOpenAILLMConfig({ */ const responsesApiExplicitlyOptedOut = dropParams != null && dropParams.includes('useResponsesApi'); - if ( + const firstPartyOpenAI = + !useOpenRouter && endpoint === EModelEndpoint.openAI && isCanonicalOpenAIBaseURL(baseURL); + const firstPartyAzure = !useOpenRouter && - endpoint === EModelEndpoint.openAI && - isCanonicalOpenAIBaseURL(baseURL) && + endpoint === EModelEndpoint.azureOpenAI && + isCanonicalAzureBaseURL(baseURL, azure); + const firstPartyEndpoint = firstPartyOpenAI || firstPartyAzure; + /** Astra keeps its model identity on Azure, where the deployment becomes the wire model. */ + const firstPartyAstra = firstPartyEndpoint && prefersResponsesApi(llmConfig.model); + if ( + firstPartyOpenAI && reasoningFormat !== ReasoningParameterFormat.disabled && llmConfig.useResponsesApi == null && !responsesApiOptedOut && @@ -889,16 +913,9 @@ export function getOpenAILLMConfig({ /** * Route GPT-6 Astra to the Responses API for every turn. Unlike the GPT-5.6 * rule above this does not depend on reasoning params: Astra serves tool calls - * only from Responses, and OpenAI recommends it generally. + * only from Responses on both OpenAI and Azure OpenAI. */ - if ( - !useOpenRouter && - endpoint === EModelEndpoint.openAI && - isCanonicalOpenAIBaseURL(baseURL) && - llmConfig.useResponsesApi == null && - !responsesApiExplicitlyOptedOut && - prefersResponsesApi(llmConfig.model) - ) { + if (firstPartyAstra && llmConfig.useResponsesApi == null && !responsesApiExplicitlyOptedOut) { llmConfig.useResponsesApi = true; } @@ -908,13 +925,8 @@ export function getOpenAILLMConfig({ * above uses, so the decision lives in one place: OpenRouter and custom * gateways route through endpoints whose contract is not OpenAI's, and only * this layer can tell them apart. - * - * Scoped to the canonical OpenAI endpoint. Astra is not documented as - * available on Azure OpenAI, and Azure's first-party hosts do not satisfy the - * OpenAI-host check, so declaring it there would claim a surface this cannot - * verify. */ - if (!useOpenRouter && endpoint === EModelEndpoint.openAI && isCanonicalOpenAIBaseURL(baseURL)) { + if (firstPartyEndpoint) { llmConfig.firstPartyEndpoint = true; } @@ -1020,26 +1032,27 @@ export function getOpenAILLMConfig({ } const useModelName = isEnabled(process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME); + const model = llmConfig.model; const updatedAzure = { ...azure }; updatedAzure.azureOpenAIApiDeploymentName = useModelName ? sanitizeModelName(llmConfig.model || '') - : azure.azureOpenAIApiDeploymentName; + : azure.azureOpenAIApiDeploymentName || + getAzureDeploymentName(baseURL, azure) || + (firstPartyAstra || llmConfig.useResponsesApi ? model : undefined); if (process.env.AZURE_OPENAI_DEFAULT_MODEL) { llmConfig.model = process.env.AZURE_OPENAI_DEFAULT_MODEL; } const constructAzureOpenAIBasePath = () => { - if (!baseURL) { + if (baseURL) { + updatedAzure.azureOpenAIBasePath = constructAzureChatBasePath(baseURL, updatedAzure); return; } - const azureURL = constructAzureURL({ - baseURL, - azureOptions: updatedAzure, - }); - updatedAzure.azureOpenAIBasePath = azureURL.split( - `/${updatedAzure.azureOpenAIApiDeploymentName}`, - )[0]; + const instanceBasePath = constructAzureInstanceBasePath(updatedAzure); + if (instanceBasePath != null && updatedAzure.azureOpenAIBasePath == null) { + updatedAzure.azureOpenAIBasePath = instanceBasePath; + } }; constructAzureOpenAIBasePath(); @@ -1060,6 +1073,15 @@ export function getOpenAILLMConfig({ constructAzureResponsesApi(); - llmConfig.model = updatedAzure.azureOpenAIApiDeploymentName; + /** Keep Astra's identity for SDK constraints; only the wire model is a deployment alias. */ + if (firstPartyAstra) { + llmConfig.model = model; + llmConfig.modelKwargs = { + ...llmConfig.modelKwargs, + model: updatedAzure.azureOpenAIApiDeploymentName, + }; + } else { + llmConfig.model = updatedAzure.azureOpenAIApiDeploymentName; + } return { llmConfig, tools, azure: updatedAzure }; } diff --git a/packages/api/src/endpoints/openai/requests.spec.ts b/packages/api/src/endpoints/openai/requests.spec.ts new file mode 100644 index 0000000000..57cfe7bc59 --- /dev/null +++ b/packages/api/src/endpoints/openai/requests.spec.ts @@ -0,0 +1,227 @@ +import { Providers, initializeModel } from '@librechat/agents'; +import { EModelEndpoint, ReasoningEffort } from 'librechat-data-provider'; +import type { OpenAI } from 'openai'; +import type { OpenAIConfiguration } from '~/types'; +import { getOpenAIConfig } from './config'; + +describe('Azure Astra requests', () => { + it.each([ + { deploymentName: 'gpt-6-astra', wireModel: 'gpt-6-astra', baseURL: undefined }, + { + deploymentName: 'production-deployment', + wireModel: 'production-deployment', + baseURL: undefined, + }, + ...[ + 'https://test-instance.openai.azure.com', + 'https://test-instance.openai.azure.com/', + 'https://test-instance.openai.azure.com/openai/', + 'https://test-instance.openai.azure.com/openai/v1/', + 'https://${INSTANCE_NAME}.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}', + ].map((baseURL) => ({ + deploymentName: 'production-deployment', + wireModel: 'production-deployment', + baseURL, + })), + { + deploymentName: undefined, + wireModel: 'url-deployment', + baseURL: + 'https://test-instance.openai.azure.com/openai/deployments/url-deployment?api-version=2025-04-01-preview', + }, + { + deploymentName: undefined, + wireModel: 'gpt-6-astra', + baseURL: 'https://test-instance.openai.azure.com/openai/v1', + }, + ])( + 'sends tool requests to $wireModel from $baseURL with Astra constraints', + async ({ deploymentName, wireModel, baseURL }) => { + const requests: { + url: URL; + headers: Headers; + body: OpenAI.Responses.ResponseCreateParams; + }[] = []; + const fetch: NonNullable['fetch']> = async (url, init) => { + requests.push({ + url: new URL(String(url)), + headers: new Headers(init?.headers), + body: JSON.parse(String(init?.body)), + }); + return Response.json({ + id: 'resp_test', + object: 'response', + status: 'completed', + model: wireModel, + output: [ + { + type: 'function_call', + id: 'fc_test', + call_id: 'call_test', + name: 'calculator', + arguments: '{"input":"2 + 2"}', + status: 'completed', + }, + ], + usage: { input_tokens: 10, output_tokens: 5, total_tokens: 15 }, + }); + }; + const { llmConfig, configOptions } = getOpenAIConfig( + 'test-azure-key', + { + streaming: false, + reverseProxyUrl: baseURL, + azure: { + azureOpenAIApiInstanceName: 'test-instance', + azureOpenAIApiDeploymentName: deploymentName, + azureOpenAIApiVersion: '2025-04-01-preview', + azureOpenAIApiKey: 'test-azure-key', + }, + modelOptions: { + model: 'gpt-6-astra', + reasoning_effort: ReasoningEffort.none, + max_tokens: 2048, + temperature: 0.7, + top_p: 0.9, + }, + addParams: { logprobs: true, topLogprobs: 5, store: false }, + }, + EModelEndpoint.azureOpenAI, + ); + expect(llmConfig).not.toHaveProperty('azureOpenAIApiInstanceName'); + const model = initializeModel({ + provider: Providers.OPENAI, + clientOptions: { + ...llmConfig, + verbosity: undefined, + configuration: { ...configOptions, fetch }, + }, + tools: [ + { + type: 'function', + function: { + name: 'calculator', + description: 'Compute arithmetic', + parameters: { + type: 'object', + properties: { input: { type: 'string' } }, + required: ['input'], + }, + }, + }, + ], + }); + + const result = await model.invoke('Use the calculator to compute 2 + 2.'); + + expect(result.tool_calls).toEqual([ + expect.objectContaining({ name: 'calculator', args: { input: '2 + 2' } }), + ]); + expect(requests).toHaveLength(1); + const { url, headers, body } = requests[0]; + expect(url.origin + url.pathname).toBe( + 'https://test-instance.openai.azure.com/openai/v1/responses', + ); + expect(headers.get('api-key')).toBe('test-azure-key'); + if (baseURL?.includes('?api-version=')) { + expect(url.searchParams.get('api-version')).toBe('2025-04-01-preview'); + } + expect(body).toMatchObject({ + model: wireModel, + max_output_tokens: 2048, + reasoning: { effort: 'low' }, + tools: [expect.objectContaining({ type: 'function', name: 'calculator' })], + include: expect.arrayContaining(['reasoning.encrypted_content']), + }); + for (const key of [ + 'max_tokens', + 'max_completion_tokens', + 'temperature', + 'top_p', + 'logprobs', + 'top_logprobs', + ]) { + expect(body).not.toHaveProperty(key); + } + expect(body.include).not.toContain('message.output_text.logprobs'); + }, + ); +}); + +describe('Azure full-hostname instances', () => { + it.each([ + { + baseURL: undefined, + model: 'gpt-6-astra', + provider: Providers.OPENAI, + path: '/openai/v1/responses', + }, + { + baseURL: undefined, + model: 'gpt-4.1', + provider: Providers.AZURE, + path: '/openai/deployments/production-deployment/chat/completions', + }, + { + baseURL: 'https://test-instance.cognitiveservices.azure.com/openai/v1', + model: 'gpt-4.1', + provider: Providers.AZURE, + path: '/openai/deployments/production-deployment/chat/completions', + }, + ])('sends $model requests to the instance host (base URL: $baseURL)', async (target) => { + const urls: URL[] = []; + const fetch: NonNullable['fetch']> = async (url) => { + urls.push(new URL(String(url))); + const text = 'Four.'; + return Response.json({ + id: 'resp_test', + object: 'response', + status: 'completed', + model: 'production-deployment', + choices: [ + { index: 0, message: { role: 'assistant', content: text }, finish_reason: 'stop' }, + ], + output: [ + { + type: 'message', + id: 'msg_test', + role: 'assistant', + status: 'completed', + content: [{ type: 'output_text', text, annotations: [] }], + }, + ], + usage: { prompt_tokens: 5, completion_tokens: 1, input_tokens: 5, output_tokens: 1 }, + }); + }; + const { llmConfig, configOptions } = getOpenAIConfig( + 'test-azure-key', + { + streaming: false, + reverseProxyUrl: target.baseURL, + azure: { + azureOpenAIApiInstanceName: 'test-instance.cognitiveservices.azure.com', + azureOpenAIApiDeploymentName: 'production-deployment', + azureOpenAIApiVersion: '2024-10-21', + azureOpenAIApiKey: 'test-azure-key', + }, + modelOptions: { model: target.model }, + }, + EModelEndpoint.azureOpenAI, + ); + const model = initializeModel({ + provider: target.provider, + clientOptions: { + ...llmConfig, + verbosity: undefined, + configuration: { ...configOptions, fetch }, + }, + }); + + await model.invoke('What is 2 + 2?'); + + expect(urls).toHaveLength(1); + expect(urls[0].origin + urls[0].pathname).toBe( + `https://test-instance.cognitiveservices.azure.com${target.path}`, + ); + }); +}); diff --git a/packages/api/src/types/endpoints.ts b/packages/api/src/types/endpoints.ts index 39f20eabb9..88e54791c9 100644 --- a/packages/api/src/types/endpoints.ts +++ b/packages/api/src/types/endpoints.ts @@ -3,6 +3,7 @@ import type { AppConfig, IUser } from '@librechat/data-schemas'; import type { TConfig } from 'librechat-data-provider'; import type { RequestBody, ServerRequest } from './http'; import type { EndpointTokenConfig } from './tokens'; +import type { AzureOptions } from './azure'; export type TCustomEndpointsConfig = Partial<{ [key: string]: Omit }>; @@ -87,6 +88,8 @@ export function resolveEndpointRuntime(params: ProviderInitializeParams): Endpoi * Using a more permissive type to accommodate different provider-specific results */ export interface InitializeResultBase { + /** Request-resolved Azure identity, retained when Responses removes the Azure client fields. */ + azureOptions?: AzureOptions; llmConfig: ClientOptions; configOptions?: OpenAIClientOptions['configuration']; endpointTokenConfig?: EndpointTokenConfig; diff --git a/packages/api/src/types/openai.ts b/packages/api/src/types/openai.ts index 8e05b0711b..0706a1ccb0 100644 --- a/packages/api/src/types/openai.ts +++ b/packages/api/src/types/openai.ts @@ -37,7 +37,7 @@ export type OAIClientOptions = Omit & { promptCache?: boolean; promptCacheTtl?: '5m' | '1h'; /** - * Declares that this client talks to the first-party OpenAI surface, which is + * Declares that this client talks to a first-party OpenAI or Azure surface, which is * what gates the agents SDK's model-specific request constraints (GPT-6 * Astra: Responses-only tool calls, rejected sampling parameters, * unsupported reasoning efforts). The SDK defaults them off and takes this as diff --git a/packages/api/src/utils/azure.ts b/packages/api/src/utils/azure.ts index 1bbd0e29b2..f5a9011a22 100644 --- a/packages/api/src/utils/azure.ts +++ b/packages/api/src/utils/azure.ts @@ -1,5 +1,5 @@ -import { isEnabled } from './common'; import type { AzureOptions, GenericClient } from '~/types'; +import { isEnabled } from './common'; /** * Sanitizes the model name to be used in the URL by removing or replacing disallowed characters. @@ -124,3 +124,76 @@ export function constructAzureURL({ return finalURL; } + +export function isCanonicalAzureURL(baseURL: string): boolean { + try { + return /\.(?:openai|cognitiveservices|services\.ai)\.azure\.(?:com|us|cn)$/i.test( + new URL(baseURL).hostname, + ); + } catch { + return false; + } +} + +function isAzureRootPath(pathname: string): boolean { + return /^\/(?:openai(?:\/v1)?\/?|v1\/?)?$/.test(pathname); +} + +/** An instance name that is already a full Azure hostname, which `genAzureEndpoint` also accepts. */ +function getAzureInstanceHost(instanceName?: string): string | undefined { + return instanceName && isCanonicalAzureURL(`https://${instanceName}`) ? instanceName : undefined; +} + +/** + * The deployments path for a full-hostname instance without a base URL. A resource name needs + * none: the client derives `.openai.azure.com` itself, which a full hostname would double. + */ +export function constructAzureInstanceBasePath(azureOptions: AzureOptions): string | undefined { + const host = getAzureInstanceHost(azureOptions.azureOpenAIApiInstanceName); + return host ? `https://${host}/openai/deployments` : undefined; +} + +export function constructAzureChatBasePath(baseURL: string, azureOptions: AzureOptions): string { + const resolvedURL = constructAzureURL({ baseURL, azureOptions }); + if (isCanonicalAzureURL(resolvedURL)) { + const url = new URL(resolvedURL); + if (isAzureRootPath(url.pathname)) { + return `${url.origin}/openai/deployments`; + } + } + return resolvedURL.split(`/${azureOptions.azureOpenAIApiDeploymentName}`)[0]; +} + +/** Azure permits the deployment to be supplied entirely by the configured base URL. */ +export function getAzureDeploymentName( + baseURL?: string | null, + azureOptions?: AzureOptions, +): string | undefined { + if (!baseURL) { + return undefined; + } + try { + const url = new URL(constructAzureURL({ baseURL, azureOptions })); + const deployment = url.pathname.match(/\/deployments\/([^/]+)/)?.[1]; + return deployment ? decodeURIComponent(deployment) : undefined; + } catch { + return undefined; + } +} + +export function constructAzureResponsesURL( + baseURL: string | null | undefined, + azureOptions: AzureOptions, +): URL { + const instanceName = azureOptions.azureOpenAIApiInstanceName ?? ''; + const instanceHost = getAzureInstanceHost(instanceName) ?? `${instanceName}.openai.azure.com`; + const resolvedURL = baseURL + ? constructAzureURL({ baseURL, azureOptions }) + : `https://${instanceHost}/openai/v1`; + const url = new URL(resolvedURL); + url.pathname = url.pathname.replace(/\/deployments(?:\/.*)?$/, '/v1'); + if (isCanonicalAzureURL(resolvedURL) && isAzureRootPath(url.pathname)) { + url.pathname = '/openai/v1'; + } + return url; +} diff --git a/packages/data-provider/src/config.ts b/packages/data-provider/src/config.ts index 8cfcbd03a3..ca70c35e70 100644 --- a/packages/data-provider/src/config.ts +++ b/packages/data-provider/src/config.ts @@ -3211,10 +3211,9 @@ const fitlerAssistantModels = (str: string) => { const openAIModels = defaultModels[EModelEndpoint.openAI]; /** - * The OpenAI catalog without the models only the first-party OpenAI endpoint - * can run. Azure OpenAI shares this list, but Astra is neither routed to the - * Responses API nor given its request constraints there, and listing it first - * would let it become the default selection. + * Preserve Azure's fallback default selection when the OpenAI catalog gains + * Responses-preferred models. Configured Azure deployments supply their own + * model list, including Astra when deployed. */ const nonResponsesOnlyOpenAIModels = openAIModels.filter( (model) => !responsesOnlyOpenAIModels.includes(model),