📫 fix: Route Azure Astra Through Responses API (#15845)
Some checks are pending
Backend Unit Tests / Build packages (push) Waiting to run
Backend Unit Tests / Codegraph select (push) Waiting to run
Backend Unit Tests / TypeScript type checks (push) Blocked by required conditions
Backend Unit Tests / Circular dependency checks (push) Waiting to run
Backend Unit Tests / Tests: api (shard 1/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 2/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: api (shard 3/3) (push) Blocked by required conditions
Backend Unit Tests / Tests: data-provider (push) Blocked by required conditions
Backend Unit Tests / Tests: data-schemas (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 1/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 2/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 3/4) (push) Blocked by required conditions
Backend Unit Tests / Tests: @librechat/api (shard 4/4) (push) Blocked by required conditions
Codegraph E2E Votes / vote (full suite) (push) Waiting to run
Docker Dev Branch Images Build / publish (push) Waiting to run
Frontend Unit Tests / Client build recovery regression (push) Waiting to run
Frontend Unit Tests / Codegraph select (push) Waiting to run
Frontend Unit Tests / Build packages (push) Waiting to run
Frontend Unit Tests / TypeScript type checks (client) (push) Blocked by required conditions
Frontend Unit Tests / Tests: @librechat/client (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 1/2) (push) Blocked by required conditions
Frontend Unit Tests / Tests: Ubuntu (shard 2/2) (push) Blocked by required conditions
Frontend Unit Tests / Vite build verification (push) Blocked by required conditions
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions

* fix: route Azure Astra through Responses API

* fix: keep Azure Astra deployment alias out of summarizer overrides

The Azure deployment alias in `modelKwargs.model` is spread after `model`
by langchain, so the SDK's same-provider summarizer, which copies the
agent's client options and overrides only `model`, still reached the Astra
deployment when configured with a different model. Hand the summarizer the
agent's kwargs without the alias whenever its model differs.

Also derive the Astra routing decision once, reuse the first-party OpenAI
predicate in the GPT-5.6 rule, pin the opt-out shape in the Azure spec, and
expect `firstPartyEndpoint` for canonical Azure hosts in the backward-compat
spec.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix: resolve Azure request URLs and summarizer deployments

* fix: honor explicit OpenAI summarization for Azure agents

* 🧭 fix: Degrade Unresolvable Azure Summarizers Without Failing the Run

Summarization target resolution for Azure agents threw inside createRun, so a
summarization setting that could not be resolved failed every agent run before
any model call, not just compaction. The documented `summarization.provider:
openAI` block with the `.env.example` default `OPENAI_API_KEY=user_provided`
broke all Azure agents, and a summary model missing from the Azure
configuration did the same.

- An explicit OpenAI summarizer without server credentials now disables
  summarization for that Azure agent with a one-time warning instead of
  throwing; the agent's own client is not a fallback because Azure Responses
  shares the `openAI` provider.
- A summary model the Azure configuration cannot resolve warns once and keeps
  the agent's client, as before this branch.
- A summary group without a base URL no longer inherits the agent group's
  `azureOpenAIBasePath`, which sent the summary key and deployment to the
  agent's resource.
- Serverless summary groups without a version send no empty `api-version`.

* 🧪 test: Assert No Summary Request Leaves a Disabled OpenAI Summarizer

* 🧯 fix: Disable Summaries Whose Azure Deployment Cannot Be Resolved

When the Azure configuration could not resolve a summary model, the summarizer
fell back to the agent's own client. On an Azure Responses agent that stripped
the deployment alias and sent the logical model name where Azure expects a
deployment, so every compaction failed; on a chat agent the URL-pinned
deployment silently served a different model than the one configured.

An unresolvable Azure summary target now disables summarization for that agent
with a one-time warning, matching the unreachable OpenAI target: the run
proceeds on context pruning and no summary request is sent.

* 🔑 fix: Treat an Empty Azure Summary Key as Unavailable

`azureBaseSchema` accepts `apiKey: ''`, and `mapModelToAzureConfig` passes it
through, so a summary group with an empty key kept summarization enabled and
built a client with no credential. The agent flow already rejects an empty
Azure key during initialization; the summary resolver now disables
summarization for that target with the same one-time warning as a
user-provided key or base URL.

* 🧭 fix: Resolve Azure Summaries From Every Supported Azure Configuration

Three Azure configuration shapes the summary resolver and URL builders did not
cover:

- Legacy `AZURE_*` environment credentials: without an `azureOpenAI` block the
  resolver skipped Azure, so an Astra agent's summarizer reused the agent's
  Responses client and sent the logical summary model where a deployment
  belongs. The target now resolves from the environment the way
  `initializeOpenAI` does (including `AZURE_USE_MODEL_AS_DEPLOYMENT_NAME`) and
  is disabled when those credentials are user-provided or missing.
- `summarization.parameters` `baseURL` and `apiKey` overrides now apply to Azure
  summary targets, as they already did for explicit OpenAI targets, through one
  shared helper.
- A full Azure hostname used as the instance name (as `genAzureEndpoint`
  accepts) is used as the host for both the default Responses URL and the chat
  deployments path, instead of being suffixed into
  `<host>.openai.azure.com`.

* 🔐 fix: Expand Summarizer Credential Placeholders and Bound Warnings

Admin-authored `summarization.parameters` `apiKey` and `baseURL` values were
forwarded verbatim, so `${SUMMARY_KEY}` reached the client as a literal key
while Azure groups and custom endpoints expand the same syntax. They are now
expanded once through `extractEnvVariable` (keeping its sensitive-variable
denylist) before resolution and before the parameters are layered over the
client, and a credential or base URL that still holds an unresolved
placeholder disables summarization like a missing one. Azure base URL
templates (`${INSTANCE_NAME}`, `${DEPLOYMENT_NAME}`) are not treated as
unresolved.

The one-time warning set is now keyed per tenant, so one tenant's report no
longer hides another's, and resets after 256 distinct entries so changing
configurations cannot grow it without bound.

* 🧩 fix: Keep Azure URL Templates Out of Summarizer Env Expansion

Expanding `summarization.parameters.baseURL` through `extractEnvVariable`
also replaced Azure's reserved `${INSTANCE_NAME}` and `${DEPLOYMENT_NAME}`
templates whenever the host environment happened to define those names, so a
template override could route compaction to the host's instance or deployment
instead of the summary group's. Expansion now skips those two templates and
leaves them for `constructAzureURL`, and the unresolved-placeholder check reads
the same segments.

The warning deduplication set now evicts the least recently seen entry at its
bound instead of clearing, so warnings for problems that are still recurring
are not all logged again at once.

* 🔀 fix: Reset Inherited API Mode for Custom-Endpoint Summarizers

A custom OpenAI-compatible summarization endpoint resolves to the same `openAI`
provider as an Azure Responses agent, so the SDK layered its overrides over the
agent's client options and kept the agent's `useResponsesApi`,
`firstPartyEndpoint`, reasoning and request kwargs. An Azure Astra agent
therefore sent its compaction to the gateway's `/responses` with Astra shaping.
When the resolved custom provider matches the agent's, those fields are now
replaced from the target's own configuration, and explicit summarization
parameters can still opt back in.

Azure's reserved `${INSTANCE_NAME}`/`${DEPLOYMENT_NAME}` URL templates are now
preserved only for Azure summarization targets, so an OpenAI or custom base URL
can reference environment variables with those names.

* fix: Resolve Azure Summary Identity, Transport, and Token Limits 🧭

* fix: Resolve Headers Before Building Summary Clients 📨

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Danny Avila <danny@librechat.ai>
This commit is contained in:
Joseph Licata 2026-09-14 15:06:35 -04:00 • committed by GitHub
parent f077e8c967
commit 27acf97ad9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
15 changed files with 2090 additions and 90 deletions

View file

@ -286,6 +286,46 @@ describe('initializeAgent — execution context', () => {
jest.clearAllMocks();
});
it('carries request-resolved Azure identity to the run without changing persisted agent fields', async () => {
const { agent, req, res, loadTools, db } = createMocks({
provider: EModelEndpoint.azureOpenAI,
model: 'gpt-6-astra',
});
const azureOptions = {
azureOpenAIApiKey: 'request-key',
azureOpenAIApiInstanceName: 'request-instance',
azureOpenAIApiDeploymentName: 'deployment',
azureOpenAIApiVersion: '2024-10-21',
};
mockGetProviderConfig.mockReturnValue({
overrideProvider: Providers.AZURE,
getOptions: jest.fn().mockResolvedValue({
azureOptions,
llmConfig: {
model: 'gpt-6-astra',
useResponsesApi: true,
apiKey: 'request-key',
modelKwargs: { model: 'deployment' },
},
}),
});
const result = await initializeAgent(
{
req,
res,
agent,
loadTools,
endpointOption: { endpoint: EModelEndpoint.agents },
allowedProviders: new Set([EModelEndpoint.azureOpenAI]),
isInitialAgent: true,
},
db,
);
expect(result.azureOptions).toEqual(azureOptions);
expect(result.model_parameters).not.toHaveProperty('azureOpenAIApiInstanceName');
expect(agent).not.toHaveProperty('azureOptions');
});
it('initializes without Express request or response objects', async () => {
const { agent, loadTools, db } = createMocks();
const previousReqDescriptor = Object.getOwnPropertyDescriptor(globalThis, 'req');

File diff suppressed because it is too large Load diff

View file

@ -605,6 +605,8 @@ function resolveProviderToolConflicts({
* Extended agent type with additional fields needed after initialization
*/
export type InitializedAgent = Agent & {
/** Request-resolved Azure identity for self-summarization; never persisted on the agent. */
azureOptions?: InitializeResultBase['azureOptions'];
tools: GenericTool[];
/** @deprecated use requestAttachments or agentContextAttachments based on sharing semantics. */
attachments: IMongoFile[];
@ -2363,6 +2365,7 @@ export async function initializeAgent(
const initializedAgent: InitializedAgent = {
...agent,
azureOptions: options.azureOptions,
resendFiles,
imageDetail,
toolRegistry,

View file

@ -11,6 +11,7 @@ import {
extractEnvVariable,
providerEndpointMap,
normalizeEndpointName,
mapModelToAzureConfig,
resolveUseResponsesApi,
} from 'librechat-data-provider';
import type {
@ -37,6 +38,7 @@ import type {
import type {
Agent,
ImageDetail,
TAzureConfig,
CodeApprovalMode,
TAgentsEndpoint,
AgentModelParameters,
@ -106,11 +108,12 @@ import {
} from '~/agents/config';
import { applyCustomHandoffPromptKeyCompatibility } from '~/agents/handoffPromptKeyCompatibility';
import { stripIntentFromToolRegistry, stripIntentFromToolDefinitions } from '~/agents/intent';
import { resolveConfigHeaders, resolveModelHeaders, mergeHeaders } from '~/utils/headers';
import { extractDefaultParams, resolveReasoningParams } from '~/endpoints/openai/llm';
import { getLLMConfig as getAnthropicLLMConfig } from '~/endpoints/anthropic/llm';
import { CREATE_FILE_TOOL_NAME, EDIT_FILE_TOOL_NAME } from '~/agents/tools';
import { resolveConfigHeaders, resolveModelHeaders } from '~/utils/headers';
import { buildAgentInitialToolSessions } from '~/agents/codeFilesSession';
import { getAzureCredentials, constructAzureURL } from '~/utils/azure';
import { getBuiltInBaseURL } from '~/endpoints/openai/initialize';
import { getProviderConfig } from '~/endpoints/config/providers';
import { buildToolApprovalHooks } from '~/agents/hitl/hooks';
@ -413,6 +416,7 @@ export function shouldReplayReasoningContent(
}
type RunAgent = Omit<Agent, 'tools'> & {
azureOptions?: t.AzureOptions;
tools?: GenericTool[];
maxContextTokens?: number;
/** Pre-ratio context budget from initializeAgent. */
@ -577,20 +581,68 @@ function normalizeAgentModelParameters(
* Merges user-supplied summarization parameters on top of endpoint-resolved
* overrides. User params win for top-level keys; `configuration` is
* deep-merged so user additions (e.g. `defaultQuery`) don't wipe out the
* resolved `baseURL`/`defaultHeaders`/`fetchOptions`.
* resolved `baseURL`/`defaultHeaders`/`fetchOptions`. When transport resolution already
* consumed the user's URL, retain its normalized form instead of restoring the raw template.
*/
function mergeParameters(
overrides: SummarizationClientOverrides,
userParams: SummarizationConfig['parameters'],
resolvedTransport = false,
): Record<string, unknown> {
const merged: Record<string, unknown> = { ...overrides, ...(userParams ?? {}) };
const userConfiguration = (userParams as Record<string, unknown> | undefined)?.configuration;
if (isPlainObject(overrides.configuration) && isPlainObject(userConfiguration)) {
merged.configuration = { ...overrides.configuration, ...userConfiguration };
merged.configuration = {
...overrides.configuration,
...userConfiguration,
...(resolvedTransport ? { baseURL: overrides.configuration.baseURL } : {}),
defaultHeaders: mergeHeaders(
overrides.configuration.defaultHeaders as Record<string, string> | undefined,
userConfiguration.defaultHeaders as Record<string, string> | undefined,
),
defaultQuery: {
...(isPlainObject(overrides.configuration.defaultQuery)
? overrides.configuration.defaultQuery
: {}),
...(isPlainObject(userConfiguration.defaultQuery) ? userConfiguration.defaultQuery : {}),
},
};
}
return merged;
}
/** `model_parameters` is the agent's resolved `llmConfig`, whose kwargs the schema type omits. */
function agentModelKwargs(
modelParameters: AgentModelParameters | undefined,
): Record<string, unknown> | undefined {
if (modelParameters == null || !('modelKwargs' in modelParameters)) {
return undefined;
}
return isPlainObject(modelParameters.modelKwargs) ? modelParameters.modelKwargs : undefined;
}
/**
* `getOpenAILLMConfig` carries an Azure Astra deployment alias in
* `modelKwargs.model`, which langchain spreads after `model`. The SDK's
* same-provider summarizer copies the agent's client options and overrides only
* `model`, so a summarizer on another model would still reach the agent's
* deployment. Hand it the agent's kwargs without the alias.
*/
function summarizationModelKwargs(
agentKwargs: Record<string, unknown> | undefined,
agentModel: string | undefined,
summarizationModel: string | undefined,
): Record<string, unknown> | undefined {
if (agentKwargs == null || !('model' in agentKwargs)) {
return undefined;
}
if (!isNonEmptyString(summarizationModel) || summarizationModel === agentModel) {
return undefined;
}
const { model: _alias, ...kwargs } = agentKwargs;
return kwargs;
}
/**
* Mirrors `getOpenAIConfig`'s `llmConfig` shape (plus its `configOptions`
* assigned to `configuration`). Index signature covers fields that the
@ -701,6 +753,370 @@ function resolveBuiltInClientOverrides(
return Object.keys(shaping).length > 0 ? shaping : undefined;
}
/**
* Memory bound for the warning deduplication below, not an operator setting: it only decides when a
* warning for one of more than this many distinct, concurrently recurring problems is logged again.
*/
const MAX_UNRESOLVED_SUMMARIZATION_WARNINGS = 256;
/** Reported summarization misconfigurations per tenant, least recently seen first. */
const unresolvedSummarizationWarnings = new Set<string>();
function warnUnresolvedSummarization(message: string, tenantId?: string): void {
const key = `${tenantId ?? ''}\n${message}`;
if (unresolvedSummarizationWarnings.delete(key)) {
unresolvedSummarizationWarnings.add(key);
return;
}
if (unresolvedSummarizationWarnings.size >= MAX_UNRESOLVED_SUMMARIZATION_WARNINGS) {
const [leastRecentlySeen] = unresolvedSummarizationWarnings;
unresolvedSummarizationWarnings.delete(leastRecentlySeen);
}
unresolvedSummarizationWarnings.add(key);
if (tenantId == null) {
logger.warn(`[createRun] ${message}`);
return;
}
logger.warn(`[createRun] ${message}`, { tenantId });
}
/** Azure base URL templates the client fills from its own options rather than the environment. */
const AZURE_URL_TEMPLATE = /(\$\{(?:INSTANCE_NAME|DEPLOYMENT_NAME)\})/;
/** The URL's segments around Azure's reserved templates, which sit at the odd indexes. */
function splitAzureURLTemplates(url: string): string[] {
return url.split(AZURE_URL_TEMPLATE);
}
/** Expands environment references in a URL, leaving Azure's reserved templates for an Azure client. */
function expandTransportURL(url: string, targetsAzure: boolean): string {
if (!targetsAzure) {
return extractEnvVariable(url);
}
return splitAzureURLTemplates(url)
.map((segment, index) => (index % 2 === 1 ? segment : extractEnvVariable(segment)))
.join('');
}
/**
* Admin-authored transport values may reference environment variables, as endpoint credentials
* do. Expanded once, before resolution and before the parameters are layered over the client.
*/
function expandSummarizationTransport(
parameters: SummarizationConfig['parameters'],
targetsAzure: boolean,
): SummarizationConfig['parameters'] {
if (!isPlainObject(parameters)) {
return parameters;
}
const params = parameters as Record<string, unknown>;
const expanded: Record<string, unknown> = { ...params };
if (typeof params.apiKey === 'string') {
expanded.apiKey = extractEnvVariable(params.apiKey);
}
if (typeof params.baseURL === 'string') {
expanded.baseURL = expandTransportURL(params.baseURL, targetsAzure);
}
if (isPlainObject(params.configuration) && typeof params.configuration.baseURL === 'string') {
expanded.configuration = {
...params.configuration,
baseURL: expandTransportURL(params.configuration.baseURL, targetsAzure),
};
}
return expanded as SummarizationConfig['parameters'];
}
/** The base URL and API key a summarization target's own parameters set, which replace the resolved ones. */
function summarizationTransportOverrides(parameters: SummarizationConfig['parameters']): {
baseURL?: string;
apiKey?: string;
} {
const configuration = (parameters as Record<string, unknown> | undefined)?.configuration;
const configurationBaseURL =
isPlainObject(configuration) && typeof configuration.baseURL === 'string'
? configuration.baseURL
: undefined;
return {
baseURL:
configurationBaseURL ??
(typeof parameters?.baseURL === 'string' ? parameters.baseURL : undefined),
apiKey: typeof parameters?.apiKey === 'string' ? parameters.apiKey : undefined,
};
}
/** Reject unusable transports while shaping the run, before a compaction client can fail it. */
function isSummarizationURLAvailable(baseURL: string | undefined, azure?: t.AzureOptions): boolean {
if (baseURL == null) {
return azure == null || isNonEmptyString(azure.azureOpenAIApiInstanceName);
}
try {
const url = new URL(constructAzureURL({ baseURL, azureOptions: azure }));
return (
(url.protocol === 'https:' || url.protocol === 'http:') && !hasUnresolvedPlaceholder(url.href)
);
} catch {
return false;
}
}
/**
* Azure Responses shares OpenAI's SDK provider, so switching endpoints must replace its transport.
* Returns `undefined` when the OpenAI credentials cannot be resolved without a per-user lookup.
*/
function resolveOpenAISummarization(
model: string,
appConfig: AppConfig | undefined,
parameters: SummarizationConfig['parameters'],
headerContext: { user?: IUser; tenantId?: string; requestBody?: t.RequestBody },
): { provider: string; clientOverrides: SummarizationClientOverrides } | undefined {
const overrides = summarizationTransportOverrides(parameters);
const baseURL = overrides.baseURL ?? getBuiltInBaseURL(EModelEndpoint.openAI);
const apiKey = overrides.apiKey ?? process.env.OPENAI_API_KEY;
if (
!apiKey ||
!isSummarizationURLAvailable(baseURL) ||
isUserProvided(baseURL) ||
isUserProvided(apiKey) ||
hasUnresolvedPlaceholder(apiKey) ||
(baseURL != null && hasUnresolvedPlaceholder(baseURL))
) {
warnUnresolvedSummarization(
`Summarization with OpenAI model "${model}" is disabled for Azure OpenAI agents: it needs a server-configured OpenAI API key and base URL.`,
headerContext.tenantId,
);
return undefined;
}
const headers = mergeHeaders(
appConfig?.endpoints?.all?.headers,
appConfig?.endpoints?.openAI?.headers,
);
const { llmConfig, configOptions } = getOpenAIConfig(
apiKey,
{
modelOptions: {
model,
reasoning_effort: summarizationReasoningEffort(parameters),
useResponsesApi:
typeof parameters?.useResponsesApi === 'boolean' ? parameters.useResponsesApi : undefined,
},
reverseProxyUrl: baseURL,
proxy: process.env.PROXY ?? undefined,
headers: resolveModelHeaders({
headers: headers ?? {},
user: createSafeUser(headerContext.user),
tenantId: headerContext.tenantId,
body: headerContext.requestBody,
}),
},
EModelEndpoint.openAI,
);
return {
provider: Providers.OPENAI,
clientOverrides: {
...llmConfig,
apiKey,
useResponsesApi: llmConfig.useResponsesApi ?? false,
firstPartyEndpoint: llmConfig.firstPartyEndpoint ?? false,
reasoning: llmConfig.reasoning,
modelKwargs: llmConfig.modelKwargs ?? {},
configuration: configOptions,
},
};
}
type AzureSummarizationTarget = Omit<ReturnType<typeof mapModelToAzureConfig>, 'azureOptions'> & {
azureOptions: t.AzureOptions;
group?: TAzureConfig['groupMap'][string];
};
/**
* The summary model's Azure credentials, resolved the way `initializeOpenAI` resolves an agent's:
* the `azureOpenAI` configuration when present, otherwise the legacy environment credentials.
* Returns `undefined` (reported once) when the configuration does not define the model.
*/
function resolveAzureSummarizationTarget(
model: string,
azureConfig: TAzureConfig | undefined,
tenantId: string | undefined,
): AzureSummarizationTarget | undefined {
if (!azureConfig) {
return { azureOptions: getAzureCredentials() };
}
try {
const groupName = azureConfig.modelGroupMap[model]?.group;
return {
...mapModelToAzureConfig({
modelName: model,
modelGroupMap: azureConfig.modelGroupMap,
groupMap: azureConfig.groupMap,
}),
group: groupName ? azureConfig.groupMap[groupName] : undefined,
};
} catch (error) {
warnUnresolvedSummarization(
`Summarization with Azure OpenAI model "${model}" is disabled: ${(error as Error).message}`,
tenantId,
);
return undefined;
}
}
/** Reuse request-resolved credentials for self-summaries, including user-provided Azure keys. */
function azureSummarizationSource(agent: RunAgent): AzureSummarizationTarget {
const options = agent.model_parameters as Partial<t.OAIClientOptions> & t.AzureOptions;
const configuration = options.configuration;
const baseURL = configuration?.baseURL ?? options.azureOpenAIBasePath;
const deployment = agentModelKwargs(agent.model_parameters)?.model;
return {
serverless: agent.useLegacyContent === true,
baseURL: baseURL ?? undefined,
azureOptions: {
azureOpenAIApiKey:
options.azureOpenAIApiKey ??
(typeof options.apiKey === 'string' ? options.apiKey : undefined),
azureOpenAIApiInstanceName:
options.azureOpenAIApiInstanceName ?? agent.azureOptions?.azureOpenAIApiInstanceName,
azureOpenAIApiDeploymentName:
options.azureOpenAIApiDeploymentName ??
(typeof deployment === 'string' ? deployment : options.model),
azureOpenAIApiVersion:
options.azureOpenAIApiVersion ??
agent.azureOptions?.azureOpenAIApiVersion ??
configuration?.defaultQuery?.['api-version'] ??
undefined,
},
};
}
/**
* Resolve the summary model's deployment and transport before the SDK inherits agent options.
* A same-provider summarizer layers these over the agent's client options, so a summary group
* without a base path clears the agent's instead of sending its deployment to that resource.
* Returns `undefined` when the model cannot be resolved without a per-user lookup.
*/
function resolveAzureSummarization(
model: string,
appConfig: AppConfig | undefined,
parameters: SummarizationConfig['parameters'],
headerContext: { user?: IUser; tenantId?: string; requestBody?: t.RequestBody },
source?: RunAgent,
): { provider: string; clientOverrides: SummarizationClientOverrides } | undefined {
const sourceOptions = source?.model_parameters as Partial<t.OAIClientOptions> | undefined;
const target = source
? azureSummarizationSource(source)
: resolveAzureSummarizationTarget(
model,
appConfig?.endpoints?.[EModelEndpoint.azureOpenAI],
headerContext.tenantId,
);
if (!target) {
return undefined;
}
const { baseURL, headers, serverless, group } = target;
const overrides = summarizationTransportOverrides(parameters);
const azureOptions: t.AzureOptions = {
...target.azureOptions,
azureOpenAIApiKey: overrides.apiKey ?? target.azureOptions.azureOpenAIApiKey,
};
const resolvedBaseURL =
overrides.baseURL ?? baseURL ?? getBuiltInBaseURL(EModelEndpoint.azureOpenAI);
if (
!azureOptions.azureOpenAIApiKey ||
!isSummarizationURLAvailable(resolvedBaseURL, serverless ? undefined : azureOptions) ||
Object.values(azureOptions).some(
(value) =>
typeof value === 'string' && (isUserProvided(value) || hasUnresolvedPlaceholder(value)),
) ||
isUserProvided(resolvedBaseURL) ||
(resolvedBaseURL != null &&
splitAzureURLTemplates(resolvedBaseURL).some(
(segment, index) => index % 2 === 0 && hasUnresolvedPlaceholder(segment),
))
) {
warnUnresolvedSummarization(
`Summarization with Azure OpenAI model "${model}" is disabled: it needs a server-configured Azure OpenAI API key and base URL.`,
headerContext.tenantId,
);
return undefined;
}
const resolvedHeaders = sourceOptions
? (sourceOptions.configuration?.defaultHeaders as Record<string, string> | undefined)
: resolveModelHeaders({
headers: mergeHeaders(appConfig?.endpoints?.all?.headers, headers) ?? {},
user: createSafeUser(headerContext.user),
tenantId: headerContext.tenantId,
body: headerContext.requestBody,
});
const { llmConfig, configOptions } = getOpenAIConfig(
azureOptions.azureOpenAIApiKey,
{
azure: serverless ? undefined : azureOptions,
reverseProxyUrl: resolvedBaseURL,
proxy: process.env.PROXY ?? undefined,
headers:
serverless || sourceOptions != null
? { ...resolvedHeaders, 'api-key': azureOptions.azureOpenAIApiKey }
: resolvedHeaders,
defaultQuery:
serverless && azureOptions.azureOpenAIApiVersion
? { 'api-version': azureOptions.azureOpenAIApiVersion }
: undefined,
modelOptions: {
model: source?.model ?? model,
reasoning_effort: summarizationReasoningEffort(parameters),
useResponsesApi:
typeof parameters?.useResponsesApi === 'boolean'
? parameters.useResponsesApi
: sourceOptions?.useResponsesApi,
},
addParams: {
...group?.addParams,
...(typeof parameters?.useResponsesApi === 'boolean'
? { useResponsesApi: parameters.useResponsesApi }
: {}),
},
dropParams: group?.dropParams?.filter(
(key) => key !== 'useResponsesApi' || typeof parameters?.useResponsesApi !== 'boolean',
),
},
EModelEndpoint.azureOpenAI,
);
const sourceKwargs = agentModelKwargs(source?.model_parameters);
const preservesApiMode = !!sourceOptions?.useResponsesApi === !!llmConfig.useResponsesApi;
const sourceTokenLimit = sourceKwargs?.max_output_tokens ?? sourceKwargs?.max_completion_tokens;
const inheritedTokenLimit =
typeof sourceTokenLimit === 'number'
? {
[llmConfig.useResponsesApi ? 'max_output_tokens' : 'max_completion_tokens']:
sourceTokenLimit,
}
: {};
return {
provider: !serverless && !llmConfig.useResponsesApi ? Providers.AZURE : Providers.OPENAI,
clientOverrides: {
...sourceOptions,
azureOpenAIBasePath: undefined,
azureOpenAIApiKey: undefined,
azureOpenAIApiInstanceName: undefined,
azureOpenAIApiDeploymentName: undefined,
azureOpenAIApiVersion: undefined,
...llmConfig,
apiKey: azureOptions.azureOpenAIApiKey,
useResponsesApi: llmConfig.useResponsesApi ?? false,
firstPartyEndpoint: llmConfig.firstPartyEndpoint ?? false,
reasoning: llmConfig.reasoning ?? (preservesApiMode ? sourceOptions?.reasoning : undefined),
modelKwargs: {
...(preservesApiMode ? sourceKwargs : undefined),
...inheritedTokenLimit,
...llmConfig.modelKwargs,
model: llmConfig.modelKwargs?.model ?? llmConfig.model,
},
configuration: sourceOptions
? { ...sourceOptions.configuration, ...configOptions }
: configOptions,
},
};
}
/**
* Resolves a summarization provider string (which may be a custom-endpoint name
* like "Ollama") into the SDK-recognized provider and any client-option
@ -861,10 +1277,19 @@ function resolveSummarizationProvider(
* agent flow builds for that endpoint (`initializeAgent` applies the same
* precedence).
*/
return {
provider: detectedProvider ?? overrideProvider,
clientOverrides,
};
const provider = detectedProvider ?? overrideProvider;
/**
* On the agent's provider the SDK layers these over the agent's own client options, so this
* different endpoint replaces the agent's API mode, first-party declaration, reasoning and
* request kwargs (an Azure Astra agent's Responses routing, for one) instead of inheriting them.
*/
if (provider === target.agentProvider) {
clientOverrides.useResponsesApi ??= false;
clientOverrides.firstPartyEndpoint ??= false;
clientOverrides.modelKwargs ??= {};
clientOverrides.reasoning ??= undefined;
}
return { provider, clientOverrides };
} catch (error) {
logger.warn(
`[resolveSummarizationProvider] failed to resolve "${rawProvider}"; falling back to raw provider`,
@ -882,6 +1307,7 @@ function shapeSummarizationConfig(
appConfig: AppConfig | undefined,
agentEndpoint: string | undefined,
headerContext: { user?: IUser; tenantId?: string; requestBody?: t.RequestBody },
agent?: RunAgent,
) {
const rawProvider = config?.provider ?? fallbackProvider;
/**
@ -897,14 +1323,61 @@ function shapeSummarizationConfig(
normalizeEndpointName(rawProvider) === normalizeEndpointName(agentEndpoint);
const model = config?.model ?? fallbackModel;
const targetsAzure =
rawProvider === EModelEndpoint.azureOpenAI ||
(agentEndpoint === EModelEndpoint.azureOpenAI && config?.provider == null);
const userParameters = expandSummarizationTransport(config?.parameters, targetsAzure);
const { provider, clientOverrides } = isSameEndpointAsAgent
? { provider: fallbackProvider, clientOverrides: undefined }
: resolveSummarizationProvider(rawProvider, appConfig, headerContext, {
const selfAzureModel =
agentEndpoint === EModelEndpoint.azureOpenAI &&
(model === fallbackModel || model === agent?.model);
const transportOverrides = summarizationTransportOverrides(userParameters);
const overridesAzureTransport =
transportOverrides.baseURL != null ||
transportOverrides.apiKey != null ||
typeof userParameters?.useResponsesApi === 'boolean';
const selectsAzureDeployment =
targetsAzure &&
isNonEmptyString(model) &&
config?.enabled !== false &&
(agentEndpoint !== EModelEndpoint.azureOpenAI ||
model !== fallbackModel ||
overridesAzureTransport);
const azureOverrides = selectsAzureDeployment
? resolveAzureSummarization(
model,
parameters: config?.parameters,
agentProvider: fallbackProvider,
});
appConfig,
userParameters,
headerContext,
selfAzureModel ? agent : undefined,
)
: undefined;
const selectsOpenAIForAzureAgent =
agentEndpoint === EModelEndpoint.azureOpenAI &&
config?.provider === EModelEndpoint.openAI &&
config.enabled !== false &&
isNonEmptyString(model);
const openAIOverrides = selectsOpenAIForAzureAgent
? resolveOpenAISummarization(model, appConfig, userParameters, headerContext)
: undefined;
/**
* A target resolved here is not handed to another client when resolution fails. Azure Responses
* shares the `openAI` provider, so the SDK would summarize an OpenAI target through the agent's
* Azure resource, and would send an unmapped Azure model name where a deployment belongs.
*/
const targetUnavailable =
(selectsAzureDeployment && azureOverrides == null) ||
(selectsOpenAIForAzureAgent && openAIOverrides == null);
const { provider, clientOverrides } =
openAIOverrides ??
azureOverrides ??
(isSameEndpointAsAgent
? { provider: fallbackProvider, clientOverrides: undefined }
: resolveSummarizationProvider(rawProvider, appConfig, headerContext, {
model,
parameters: userParameters,
agentProvider: fallbackProvider,
}));
const trigger =
config?.trigger?.type && typeof config?.trigger?.value === 'number'
? { type: config.trigger.type, value: config.trigger.value }
@ -913,9 +1386,8 @@ function shapeSummarizationConfig(
/**
* Custom-endpoint overrides are merged into `parameters` so the SDK's
* `buildSummarizationClientConfig` spreads them onto the summarization
* client options. Only applied when summarization targets a *different*
* custom endpoint than the main agent; the same-endpoint case leaves
* `parameters` untouched so `agentContext.clientOptions` wins.
* client options. Azure self-summaries with explicit transport overrides also resolve here,
* using the initialized agent's identity and credentials instead of a new configuration lookup.
*
* Order matters: `clientOverrides` supplies endpoint defaults (baseURL,
* apiKey, headers, transforms), then explicit user `summarization.parameters`
@ -926,18 +1398,56 @@ function shapeSummarizationConfig(
*/
const mergedParameters =
clientOverrides != null
? mergeParameters(clientOverrides, config?.parameters)
: config?.parameters;
? mergeParameters(
clientOverrides,
userParameters,
azureOverrides != null || openAIOverrides != null,
)
: userParameters;
/** Placed first so an explicit user `modelKwargs` still replaces the agent's wholesale. */
const modelKwargs =
provider === fallbackProvider
? summarizationModelKwargs(agentModelKwargs(agent?.model_parameters), fallbackModel, model)
: undefined;
/**
* A scalar `reasoning_effort` — the only reasoning shape the yaml schema
* accepts — is inert as a client option and leaves the summarizer running at
* whatever effort the main agent resolved. Translate it the way the main
* flow's `getOpenAIConfig` would for the summarization target.
*/
const parameters = resolveReasoningParams({ provider, model, parameters: mergedParameters });
let parameters = resolveReasoningParams({
provider,
model,
parameters: modelKwargs != null ? { modelKwargs, ...mergedParameters } : mergedParameters,
});
/** The SDK sets maxTokens for this cap, but LangChain spreads modelKwargs after it. */
const parameterTokenCap = userParameters?.maxSummaryTokens;
const summaryTokenCap =
typeof parameterTokenCap === 'number' && parameterTokenCap > 0
? parameterTokenCap
: config?.maxSummaryTokens;
const inheritedKwargs =
provider === fallbackProvider ? agentModelKwargs(agent?.model_parameters) : undefined;
const effectiveKwargs = isPlainObject(parameters?.modelKwargs)
? parameters.modelKwargs
: inheritedKwargs;
if (typeof summaryTokenCap === 'number' && effectiveKwargs != null) {
const {
max_tokens: _maxTokens,
max_completion_tokens: _maxCompletionTokens,
max_output_tokens: _maxOutputTokens,
...kwargs
} = effectiveKwargs;
parameters = { ...parameters, modelKwargs: kwargs };
}
return {
enabled: config?.enabled !== false && isNonEmptyString(provider) && isNonEmptyString(model),
enabled:
!targetUnavailable &&
config?.enabled !== false &&
isNonEmptyString(provider) &&
isNonEmptyString(model),
config: {
trigger,
provider,
@ -1820,6 +2330,21 @@ export async function createRun({
] as unknown as Providers) ?? agent.provider;
const selfModel = agent.model_parameters?.model ?? (agent.model as string | undefined);
/**
* Resolve request-based headers across provider-specific header locations
* (OpenAI `configuration.defaultHeaders`, Anthropic `clientOptions.defaultHeaders`,
* Google `customHeaders`). Done at this step because the request body may
* contain dynamic values (e.g. conversationId) that are only known after
* agent initialization. Resolve before a self-summary snapshots this configuration.
*/
resolveConfigHeaders({
llmConfig: agent.model_parameters as Partial<t.RunLLMConfig>,
user: createSafeUser(user),
tenantId,
body: requestBody,
});
const modelParameters = normalizeAgentModelParameters(agent.model_parameters);
const shapedSummarization = shapeSummarizationConfig(
agent.summarization ?? summarizationConfig,
provider as string,
@ -1827,6 +2352,7 @@ export async function createRun({
appConfig,
agent.endpoint ?? undefined,
{ user, tenantId, requestBody },
agent,
);
const summarization = modelCallbacks?.length
? {
@ -1841,7 +2367,6 @@ export async function createRun({
}
: shapedSummarization;
const modelParameters = normalizeAgentModelParameters(agent.model_parameters);
const hasExplicitStreamUsage = Object.prototype.hasOwnProperty.call(
modelParameters ?? {},
'streamUsage',
@ -1873,20 +2398,6 @@ export async function createRun({
.join('\n')
.trim();
/**
* Resolve request-based headers across provider-specific header locations
* (OpenAI `configuration.defaultHeaders`, Anthropic `clientOptions.defaultHeaders`,
* Google `customHeaders`). Done at this step because the request body may
* contain dynamic values (e.g. conversationId) that are only known after
* agent initialization.
*/
resolveConfigHeaders({
llmConfig,
user: createSafeUser(user),
tenantId,
body: requestBody,
});
/** Resolves issues with new OpenAI usage field */
if (
customProviders.has(agent.provider) ||

View file

@ -165,6 +165,7 @@ describe('getOpenAIConfig - Backward Compatibility', () => {
streaming: true,
model: 'gpt-5',
useResponsesApi: true,
firstPartyEndpoint: true,
user: 'some_user_id',
apiKey: 'some_azure_key',
reasoning: {

View file

@ -909,9 +909,9 @@ describe('getOpenAIConfig', () => {
reverseProxyUrl: 'https://${INSTANCE_NAME}.openai.azure.com/openai/v1',
});
// The constructAzureURL should replace placeholders with actual values
// AzureChatOpenAI appends the deployment to its base path, including for a v1 resource URL.
expect((result.llmConfig as Record<string, unknown>).azureOpenAIBasePath).toBe(
'https://test-instance.openai.azure.com/openai/v1',
'https://test-instance.openai.azure.com/openai/deployments',
);
});

View file

@ -7,9 +7,9 @@ import { getGoogleConfig, stripGeminiFlashBlockedParams } from '~/endpoints/goog
import { getLLMConfig as getAnthropicLLMConfig } from '~/endpoints/anthropic/llm';
import { createSSRFSafeAgents, createSSRFSafeUndiciConnect } from '~/auth';
import { getOpenAILLMConfig, extractDefaultParams } from './llm';
import { constructAzureResponsesURL } from '~/utils/azure';
import { transformToOpenAIConfig } from './transform';
import { getProxyDispatcher } from '~/utils/proxy';
import { constructAzureURL } from '~/utils/azure';
import { createFetch } from '~/utils/generators';
import { mergeHeaders } from '~/utils/headers';
@ -270,20 +270,21 @@ export function getOpenAIConfig(
return;
}
const updatedUrl = configOptions.baseURL?.replace(/\/deployments(?:\/.*)?$/, '/v1');
configOptions.baseURL = constructAzureURL({
baseURL: updatedUrl || 'https://${INSTANCE_NAME}.openai.azure.com/openai/v1',
azureOptions: azure,
});
const responsesURL = constructAzureResponsesURL(configOptions.baseURL, azure);
const urlQuery = Object.fromEntries(responsesURL.searchParams);
responsesURL.search = '';
responsesURL.hash = '';
configOptions.baseURL = responsesURL.toString();
configOptions.defaultHeaders = {
...configOptions.defaultHeaders,
'api-key': apiKey,
};
configOptions.defaultQuery = {
...urlQuery,
...configOptions.defaultQuery,
'api-version': configOptions.defaultQuery?.['api-version'] ?? 'preview',
'api-version':
configOptions.defaultQuery?.['api-version'] ?? urlQuery['api-version'] ?? 'preview',
};
};

View file

@ -201,7 +201,10 @@ export async function initializeOpenAI(
modelOptions,
};
const options = getOpenAIConfig(apiKey, finalClientOptions, endpoint);
const options: InitializeResultBase = getOpenAIConfig(apiKey, finalClientOptions, endpoint);
if (clientOptions.azure) {
options.azureOptions = { ...clientOptions.azure };
}
/** Set useLegacyContent for Azure serverless deployments */
if (isServerless) {

View file

@ -801,12 +801,7 @@ describe('getOpenAILLMConfig', () => {
);
});
/**
* Astra is not documented as available on Azure OpenAI, and Azure's
* first-party hosts do not satisfy the OpenAI-host check, so declaring it
* there would claim a surface this cannot verify.
*/
it('does not declare it for Azure OpenAI', () => {
it('does not declare it for Azure serverless without Azure OpenAI configuration', () => {
expect(configFor({ endpoint: EModelEndpoint.azureOpenAI }).llmConfig).not.toHaveProperty(
'firstPartyEndpoint',
);
@ -825,6 +820,90 @@ describe('getOpenAILLMConfig', () => {
});
});
describe('Azure Astra routing', () => {
const azure = {
azureOpenAIApiInstanceName: 'test-instance',
azureOpenAIApiDeploymentName: 'production-deployment',
azureOpenAIApiVersion: '2025-04-01-preview',
azureOpenAIApiKey: 'test-api-key',
};
const azureConfig = (overrides: Partial<Parameters<typeof getOpenAILLMConfig>[0]> = {}) =>
getOpenAILLMConfig({
azure,
apiKey: 'test-api-key',
streaming: true,
endpoint: EModelEndpoint.azureOpenAI,
modelOptions: { model: 'gpt-6-astra', max_tokens: 2048 },
...overrides,
});
it.each([
undefined,
'https://${INSTANCE_NAME}.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}',
'https://test-instance.openai.azure.com/openai/v1',
'https://test-instance.services.ai.azure.com/openai/v1',
'https://test-instance.cognitiveservices.azure.com/openai/v1',
'https://test-instance.openai.azure.us/openai/v1',
'https://test-instance.openai.azure.cn/openai/v1',
])('routes Azure Astra to Responses at %s and retains its model identity', (baseURL) => {
expect(azureConfig({ baseURL }).llmConfig).toMatchObject({
model: 'gpt-6-astra',
useResponsesApi: true,
firstPartyEndpoint: true,
modelKwargs: { model: 'production-deployment', max_output_tokens: 2048 },
});
});
it.each([
{ modelOptions: { model: 'gpt-6-astra-2026-09-03', max_tokens: 2048 } },
{ modelOptions: { model: 'gpt-4.1', max_tokens: 2048 }, addParams: { model: 'gpt-6-astra' } },
{ dropParams: ['reasoning_effort'] },
{ reasoningFormat: ReasoningParameterFormat.disabled },
])('routes using the effective model independently of reasoning: %j', (overrides) => {
expect(azureConfig(overrides).llmConfig.useResponsesApi).toBe(true);
});
it.each([
{ modelOptions: { model: 'gpt-6-astra', max_tokens: 2048, useResponsesApi: false } },
{ addParams: { useResponsesApi: false } },
{ dropParams: ['useResponsesApi'] },
])('keeps Astra constraints on an explicit Chat Completions opt-out: %j', (overrides) => {
const { llmConfig } = azureConfig(overrides);
expect(llmConfig.useResponsesApi).not.toBe(true);
expect(llmConfig).toMatchObject({
model: 'gpt-6-astra',
firstPartyEndpoint: true,
azureOpenAIApiDeploymentName: 'production-deployment',
modelKwargs: { model: 'production-deployment', max_completion_tokens: 2048 },
});
});
it.each([
{ modelOptions: { model: 'gpt-5.4-mini' } },
{ addParams: { model: 'gpt-5.4-mini' } },
])('leaves other Azure models on the deployment name: %j', (overrides) => {
const { llmConfig } = azureConfig(overrides);
expect(llmConfig.useResponsesApi).not.toBe(true);
expect(llmConfig).toMatchObject({
model: 'production-deployment',
firstPartyEndpoint: true,
});
expect(llmConfig.modelKwargs?.model).toBeUndefined();
});
it.each([
'https://gateway.internal/openai/v1',
'https://test-instance.openai.azure.com.example.org/openai/v1',
'https://openai.azure.com/openai/v1',
'not-a-url',
])('leaves gateway routing and request constraints unchanged: %s', (baseURL) => {
const { llmConfig } = azureConfig({ baseURL });
expect(llmConfig.useResponsesApi).not.toBe(true);
expect(llmConfig.firstPartyEndpoint).toBeUndefined();
expect(llmConfig.model).toBe('production-deployment');
});
});
describe('GPT-5.6 Responses API Requirement', () => {
it.each(['gpt-5.6-terra', 'gpt-5.6-luna', 'gpt-5.6-sol', 'gpt-5.6'])(
'should default to Responses API for %s when reasoning_effort is set',

View file

@ -11,7 +11,14 @@ import type { AzureOpenAIInput } from '@librechat/agents/langchain/openai';
import type { SettingDefinition } from 'librechat-data-provider';
import type { OpenAI } from 'openai';
import type * as t from '~/types';
import { sanitizeModelName, constructAzureURL } from '~/utils/azure';
import {
sanitizeModelName,
constructAzureURL,
isCanonicalAzureURL,
getAzureDeploymentName,
constructAzureChatBasePath,
constructAzureInstanceBasePath,
} from '~/utils/azure';
import { isEnabled } from '~/utils/common';
type OpenAILLMConfig = Omit<Partial<t.OAIClientOptions>, 'verbosity'> &
@ -138,8 +145,8 @@ const responsesApiRequiredPattern = /\bgpt-5\.6\b/;
* Decided here rather than in the agents SDK at invocation time: the max-tokens
* field below is shaped from `useResponsesApi`, so a later switch would send
* `max_completion_tokens` to an endpoint expecting `max_output_tokens`. Config
* time is also the only place that knows the model before Azure replaces it
* with a deployment name.
* time is also where Azure decides whether the model keeps its identity, with
* the deployment as the wire model, or becomes the deployment name outright.
* @see https://developers.openai.com/api/docs/guides/latest-model
*/
const responsesApiPreferredPattern = /^gpt-6-astra(?:-|$)/i;
@ -148,6 +155,16 @@ function prefersResponsesApi(model?: string): boolean {
return typeof model === 'string' && responsesApiPreferredPattern.test(model);
}
function isCanonicalAzureBaseURL(baseURL?: string | null, azure?: false | t.AzureOptions): boolean {
if (!azure) {
return false;
}
if (!baseURL) {
return true;
}
return isCanonicalAzureURL(constructAzureURL({ baseURL, azureOptions: azure }));
}
function requiresResponsesApiForReasoning({
model,
reasoningEffort,
@ -874,10 +891,17 @@ export function getOpenAILLMConfig({
*/
const responsesApiExplicitlyOptedOut =
dropParams != null && dropParams.includes('useResponsesApi');
if (
const firstPartyOpenAI =
!useOpenRouter && endpoint === EModelEndpoint.openAI && isCanonicalOpenAIBaseURL(baseURL);
const firstPartyAzure =
!useOpenRouter &&
endpoint === EModelEndpoint.openAI &&
isCanonicalOpenAIBaseURL(baseURL) &&
endpoint === EModelEndpoint.azureOpenAI &&
isCanonicalAzureBaseURL(baseURL, azure);
const firstPartyEndpoint = firstPartyOpenAI || firstPartyAzure;
/** Astra keeps its model identity on Azure, where the deployment becomes the wire model. */
const firstPartyAstra = firstPartyEndpoint && prefersResponsesApi(llmConfig.model);
if (
firstPartyOpenAI &&
reasoningFormat !== ReasoningParameterFormat.disabled &&
llmConfig.useResponsesApi == null &&
!responsesApiOptedOut &&
@ -889,16 +913,9 @@ export function getOpenAILLMConfig({
/**
* Route GPT-6 Astra to the Responses API for every turn. Unlike the GPT-5.6
* rule above this does not depend on reasoning params: Astra serves tool calls
* only from Responses, and OpenAI recommends it generally.
* only from Responses on both OpenAI and Azure OpenAI.
*/
if (
!useOpenRouter &&
endpoint === EModelEndpoint.openAI &&
isCanonicalOpenAIBaseURL(baseURL) &&
llmConfig.useResponsesApi == null &&
!responsesApiExplicitlyOptedOut &&
prefersResponsesApi(llmConfig.model)
) {
if (firstPartyAstra && llmConfig.useResponsesApi == null && !responsesApiExplicitlyOptedOut) {
llmConfig.useResponsesApi = true;
}
@ -908,13 +925,8 @@ export function getOpenAILLMConfig({
* above uses, so the decision lives in one place: OpenRouter and custom
* gateways route through endpoints whose contract is not OpenAI's, and only
* this layer can tell them apart.
*
* Scoped to the canonical OpenAI endpoint. Astra is not documented as
* available on Azure OpenAI, and Azure's first-party hosts do not satisfy the
* OpenAI-host check, so declaring it there would claim a surface this cannot
* verify.
*/
if (!useOpenRouter && endpoint === EModelEndpoint.openAI && isCanonicalOpenAIBaseURL(baseURL)) {
if (firstPartyEndpoint) {
llmConfig.firstPartyEndpoint = true;
}
@ -1020,26 +1032,27 @@ export function getOpenAILLMConfig({
}
const useModelName = isEnabled(process.env.AZURE_USE_MODEL_AS_DEPLOYMENT_NAME);
const model = llmConfig.model;
const updatedAzure = { ...azure };
updatedAzure.azureOpenAIApiDeploymentName = useModelName
? sanitizeModelName(llmConfig.model || '')
: azure.azureOpenAIApiDeploymentName;
: azure.azureOpenAIApiDeploymentName ||
getAzureDeploymentName(baseURL, azure) ||
(firstPartyAstra || llmConfig.useResponsesApi ? model : undefined);
if (process.env.AZURE_OPENAI_DEFAULT_MODEL) {
llmConfig.model = process.env.AZURE_OPENAI_DEFAULT_MODEL;
}
const constructAzureOpenAIBasePath = () => {
if (!baseURL) {
if (baseURL) {
updatedAzure.azureOpenAIBasePath = constructAzureChatBasePath(baseURL, updatedAzure);
return;
}
const azureURL = constructAzureURL({
baseURL,
azureOptions: updatedAzure,
});
updatedAzure.azureOpenAIBasePath = azureURL.split(
`/${updatedAzure.azureOpenAIApiDeploymentName}`,
)[0];
const instanceBasePath = constructAzureInstanceBasePath(updatedAzure);
if (instanceBasePath != null && updatedAzure.azureOpenAIBasePath == null) {
updatedAzure.azureOpenAIBasePath = instanceBasePath;
}
};
constructAzureOpenAIBasePath();
@ -1060,6 +1073,15 @@ export function getOpenAILLMConfig({
constructAzureResponsesApi();
llmConfig.model = updatedAzure.azureOpenAIApiDeploymentName;
/** Keep Astra's identity for SDK constraints; only the wire model is a deployment alias. */
if (firstPartyAstra) {
llmConfig.model = model;
llmConfig.modelKwargs = {
...llmConfig.modelKwargs,
model: updatedAzure.azureOpenAIApiDeploymentName,
};
} else {
llmConfig.model = updatedAzure.azureOpenAIApiDeploymentName;
}
return { llmConfig, tools, azure: updatedAzure };
}

View file

@ -0,0 +1,227 @@
import { Providers, initializeModel } from '@librechat/agents';
import { EModelEndpoint, ReasoningEffort } from 'librechat-data-provider';
import type { OpenAI } from 'openai';
import type { OpenAIConfiguration } from '~/types';
import { getOpenAIConfig } from './config';
describe('Azure Astra requests', () => {
it.each([
{ deploymentName: 'gpt-6-astra', wireModel: 'gpt-6-astra', baseURL: undefined },
{
deploymentName: 'production-deployment',
wireModel: 'production-deployment',
baseURL: undefined,
},
...[
'https://test-instance.openai.azure.com',
'https://test-instance.openai.azure.com/',
'https://test-instance.openai.azure.com/openai/',
'https://test-instance.openai.azure.com/openai/v1/',
'https://${INSTANCE_NAME}.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}',
].map((baseURL) => ({
deploymentName: 'production-deployment',
wireModel: 'production-deployment',
baseURL,
})),
{
deploymentName: undefined,
wireModel: 'url-deployment',
baseURL:
'https://test-instance.openai.azure.com/openai/deployments/url-deployment?api-version=2025-04-01-preview',
},
{
deploymentName: undefined,
wireModel: 'gpt-6-astra',
baseURL: 'https://test-instance.openai.azure.com/openai/v1',
},
])(
'sends tool requests to $wireModel from $baseURL with Astra constraints',
async ({ deploymentName, wireModel, baseURL }) => {
const requests: {
url: URL;
headers: Headers;
body: OpenAI.Responses.ResponseCreateParams;
}[] = [];
const fetch: NonNullable<NonNullable<OpenAIConfiguration>['fetch']> = async (url, init) => {
requests.push({
url: new URL(String(url)),
headers: new Headers(init?.headers),
body: JSON.parse(String(init?.body)),
});
return Response.json({
id: 'resp_test',
object: 'response',
status: 'completed',
model: wireModel,
output: [
{
type: 'function_call',
id: 'fc_test',
call_id: 'call_test',
name: 'calculator',
arguments: '{"input":"2 + 2"}',
status: 'completed',
},
],
usage: { input_tokens: 10, output_tokens: 5, total_tokens: 15 },
});
};
const { llmConfig, configOptions } = getOpenAIConfig(
'test-azure-key',
{
streaming: false,
reverseProxyUrl: baseURL,
azure: {
azureOpenAIApiInstanceName: 'test-instance',
azureOpenAIApiDeploymentName: deploymentName,
azureOpenAIApiVersion: '2025-04-01-preview',
azureOpenAIApiKey: 'test-azure-key',
},
modelOptions: {
model: 'gpt-6-astra',
reasoning_effort: ReasoningEffort.none,
max_tokens: 2048,
temperature: 0.7,
top_p: 0.9,
},
addParams: { logprobs: true, topLogprobs: 5, store: false },
},
EModelEndpoint.azureOpenAI,
);
expect(llmConfig).not.toHaveProperty('azureOpenAIApiInstanceName');
const model = initializeModel({
provider: Providers.OPENAI,
clientOptions: {
...llmConfig,
verbosity: undefined,
configuration: { ...configOptions, fetch },
},
tools: [
{
type: 'function',
function: {
name: 'calculator',
description: 'Compute arithmetic',
parameters: {
type: 'object',
properties: { input: { type: 'string' } },
required: ['input'],
},
},
},
],
});
const result = await model.invoke('Use the calculator to compute 2 + 2.');
expect(result.tool_calls).toEqual([
expect.objectContaining({ name: 'calculator', args: { input: '2 + 2' } }),
]);
expect(requests).toHaveLength(1);
const { url, headers, body } = requests[0];
expect(url.origin + url.pathname).toBe(
'https://test-instance.openai.azure.com/openai/v1/responses',
);
expect(headers.get('api-key')).toBe('test-azure-key');
if (baseURL?.includes('?api-version=')) {
expect(url.searchParams.get('api-version')).toBe('2025-04-01-preview');
}
expect(body).toMatchObject({
model: wireModel,
max_output_tokens: 2048,
reasoning: { effort: 'low' },
tools: [expect.objectContaining({ type: 'function', name: 'calculator' })],
include: expect.arrayContaining(['reasoning.encrypted_content']),
});
for (const key of [
'max_tokens',
'max_completion_tokens',
'temperature',
'top_p',
'logprobs',
'top_logprobs',
]) {
expect(body).not.toHaveProperty(key);
}
expect(body.include).not.toContain('message.output_text.logprobs');
},
);
});
describe('Azure full-hostname instances', () => {
it.each([
{
baseURL: undefined,
model: 'gpt-6-astra',
provider: Providers.OPENAI,
path: '/openai/v1/responses',
},
{
baseURL: undefined,
model: 'gpt-4.1',
provider: Providers.AZURE,
path: '/openai/deployments/production-deployment/chat/completions',
},
{
baseURL: 'https://test-instance.cognitiveservices.azure.com/openai/v1',
model: 'gpt-4.1',
provider: Providers.AZURE,
path: '/openai/deployments/production-deployment/chat/completions',
},
])('sends $model requests to the instance host (base URL: $baseURL)', async (target) => {
const urls: URL[] = [];
const fetch: NonNullable<NonNullable<OpenAIConfiguration>['fetch']> = async (url) => {
urls.push(new URL(String(url)));
const text = 'Four.';
return Response.json({
id: 'resp_test',
object: 'response',
status: 'completed',
model: 'production-deployment',
choices: [
{ index: 0, message: { role: 'assistant', content: text }, finish_reason: 'stop' },
],
output: [
{
type: 'message',
id: 'msg_test',
role: 'assistant',
status: 'completed',
content: [{ type: 'output_text', text, annotations: [] }],
},
],
usage: { prompt_tokens: 5, completion_tokens: 1, input_tokens: 5, output_tokens: 1 },
});
};
const { llmConfig, configOptions } = getOpenAIConfig(
'test-azure-key',
{
streaming: false,
reverseProxyUrl: target.baseURL,
azure: {
azureOpenAIApiInstanceName: 'test-instance.cognitiveservices.azure.com',
azureOpenAIApiDeploymentName: 'production-deployment',
azureOpenAIApiVersion: '2024-10-21',
azureOpenAIApiKey: 'test-azure-key',
},
modelOptions: { model: target.model },
},
EModelEndpoint.azureOpenAI,
);
const model = initializeModel({
provider: target.provider,
clientOptions: {
...llmConfig,
verbosity: undefined,
configuration: { ...configOptions, fetch },
},
});
await model.invoke('What is 2 + 2?');
expect(urls).toHaveLength(1);
expect(urls[0].origin + urls[0].pathname).toBe(
`https://test-instance.cognitiveservices.azure.com${target.path}`,
);
});
});

View file

@ -3,6 +3,7 @@ import type { AppConfig, IUser } from '@librechat/data-schemas';
import type { TConfig } from 'librechat-data-provider';
import type { RequestBody, ServerRequest } from './http';
import type { EndpointTokenConfig } from './tokens';
import type { AzureOptions } from './azure';
export type TCustomEndpointsConfig = Partial<{ [key: string]: Omit<TConfig, 'order'> }>;
@ -87,6 +88,8 @@ export function resolveEndpointRuntime(params: ProviderInitializeParams): Endpoi
* Using a more permissive type to accommodate different provider-specific results
*/
export interface InitializeResultBase {
/** Request-resolved Azure identity, retained when Responses removes the Azure client fields. */
azureOptions?: AzureOptions;
llmConfig: ClientOptions;
configOptions?: OpenAIClientOptions['configuration'];
endpointTokenConfig?: EndpointTokenConfig;

View file

@ -37,7 +37,7 @@ export type OAIClientOptions = Omit<OpenAIClientOptions, 'verbosity'> & {
promptCache?: boolean;
promptCacheTtl?: '5m' | '1h';
/**
* Declares that this client talks to the first-party OpenAI surface, which is
* Declares that this client talks to a first-party OpenAI or Azure surface, which is
* what gates the agents SDK's model-specific request constraints (GPT-6
* Astra: Responses-only tool calls, rejected sampling parameters,
* unsupported reasoning efforts). The SDK defaults them off and takes this as

View file

@ -1,5 +1,5 @@
import { isEnabled } from './common';
import type { AzureOptions, GenericClient } from '~/types';
import { isEnabled } from './common';
/**
* Sanitizes the model name to be used in the URL by removing or replacing disallowed characters.
@ -124,3 +124,76 @@ export function constructAzureURL({
return finalURL;
}
export function isCanonicalAzureURL(baseURL: string): boolean {
try {
return /\.(?:openai|cognitiveservices|services\.ai)\.azure\.(?:com|us|cn)$/i.test(
new URL(baseURL).hostname,
);
} catch {
return false;
}
}
function isAzureRootPath(pathname: string): boolean {
return /^\/(?:openai(?:\/v1)?\/?|v1\/?)?$/.test(pathname);
}
/** An instance name that is already a full Azure hostname, which `genAzureEndpoint` also accepts. */
function getAzureInstanceHost(instanceName?: string): string | undefined {
return instanceName && isCanonicalAzureURL(`https://${instanceName}`) ? instanceName : undefined;
}
/**
* The deployments path for a full-hostname instance without a base URL. A resource name needs
* none: the client derives `<name>.openai.azure.com` itself, which a full hostname would double.
*/
export function constructAzureInstanceBasePath(azureOptions: AzureOptions): string | undefined {
const host = getAzureInstanceHost(azureOptions.azureOpenAIApiInstanceName);
return host ? `https://${host}/openai/deployments` : undefined;
}
export function constructAzureChatBasePath(baseURL: string, azureOptions: AzureOptions): string {
const resolvedURL = constructAzureURL({ baseURL, azureOptions });
if (isCanonicalAzureURL(resolvedURL)) {
const url = new URL(resolvedURL);
if (isAzureRootPath(url.pathname)) {
return `${url.origin}/openai/deployments`;
}
}
return resolvedURL.split(`/${azureOptions.azureOpenAIApiDeploymentName}`)[0];
}
/** Azure permits the deployment to be supplied entirely by the configured base URL. */
export function getAzureDeploymentName(
baseURL?: string | null,
azureOptions?: AzureOptions,
): string | undefined {
if (!baseURL) {
return undefined;
}
try {
const url = new URL(constructAzureURL({ baseURL, azureOptions }));
const deployment = url.pathname.match(/\/deployments\/([^/]+)/)?.[1];
return deployment ? decodeURIComponent(deployment) : undefined;
} catch {
return undefined;
}
}
export function constructAzureResponsesURL(
baseURL: string | null | undefined,
azureOptions: AzureOptions,
): URL {
const instanceName = azureOptions.azureOpenAIApiInstanceName ?? '';
const instanceHost = getAzureInstanceHost(instanceName) ?? `${instanceName}.openai.azure.com`;
const resolvedURL = baseURL
? constructAzureURL({ baseURL, azureOptions })
: `https://${instanceHost}/openai/v1`;
const url = new URL(resolvedURL);
url.pathname = url.pathname.replace(/\/deployments(?:\/.*)?$/, '/v1');
if (isCanonicalAzureURL(resolvedURL) && isAzureRootPath(url.pathname)) {
url.pathname = '/openai/v1';
}
return url;
}

View file

@ -3211,10 +3211,9 @@ const fitlerAssistantModels = (str: string) => {
const openAIModels = defaultModels[EModelEndpoint.openAI];
/**
* The OpenAI catalog without the models only the first-party OpenAI endpoint
* can run. Azure OpenAI shares this list, but Astra is neither routed to the
* Responses API nor given its request constraints there, and listing it first
* would let it become the default selection.
* Preserve Azure's fallback default selection when the OpenAI catalog gains
* Responses-preferred models. Configured Azure deployments supply their own
* model list, including Astra when deployed.
*/
const nonResponsesOnlyOpenAIModels = openAIModels.filter(
(model) => !responsesOnlyOpenAIModels.includes(model),