mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-10-23 18:43:21 +00:00
🌌 feat: Add GPT-6 Astra Support (#15567)
* 🌌 feat: Add GPT-6 Astra Support Registers OpenAI's GPT-6 Astra (`gpt-6-astra`) as a first-class model on the canonical OpenAI endpoint. Every model-specific behavior Astra requires — Responses routing for tool calls, stripping the sampling and logprob parameters it rejects, and substituting the `none`/`minimal` reasoning efforts it does not accept — lands in `@librechat/agents` (danny-avila/agents#506) rather than here. None of it is expressible at configuration time: tools bind after `getOpenAIConfig` runs, and LangChain's request builders emit `temperature`/`top_p` from instance fields regardless of what this layer sets. Keeping it in the SDK also means the routing gate can be the documented rule (tools present) instead of the over-approximation the config-time GPT-5.6 path is forced into. What is left here is registration: - 1,050,000-token context window and 128K max output - Model dropdown (config.ts) and the `OPENAI_MODELS` example - Standard pricing: $10/M input, $1/M cached input, $12.50/M cache writes, $50/M output - The >272K long-context tier, which LibreChat already supports through `premiumTokenValues`/`premiumCacheTokenValues`: 2x input and cache rates and 1.5x output for the full request Batch/Flex (50%) and Fast mode (2x) have no representation in the pricing model today and are not introduced here; they apply per-request rather than per-model, so they need a request-shape signal this layer does not yet carry. Tests pin the pricing against the documented multipliers rather than restating the literals — cache writes as 1.25x input, and the premium tier as 2x/1.5x of standard — plus threshold behavior either side of 272K, and that `gpt-6-astra` resolves to its own key for snapshots and provider-prefixed ids instead of collapsing onto a shorter `gpt-6` match. Resolves #15560 Ref: https://developers.openai.com/api/docs/models/gpt-6-astra Ref: https://developers.openai.com/api/docs/guides/latest-model * 🧪 test: Guard the GPT-6 Astra Max Output Entry [Copilot] The Astra tests asserted the context window but not the 128K max output added alongside it in `anthropicMaxOutputs`' OpenAI counterpart, so that entry could regress silently. The gpt-5.6 tier test above covers both, and the delete-the-fix check showed the gap: removing the source entries failed only the context assertions. Assert `getModelMaxOutputTokens` alongside `getModelMaxTokens`, and fold the snapshot/prefix cases into one loop so both dimensions are covered for every id shape rather than only the bare one. Verified by deleting just the max-output entry: both Astra tests now fail where previously neither did. * 🪧 feat: Declare the First-Party OpenAI Endpoint to the Agents SDK The agents SDK gates GPT-6 Astra's request constraints — Responses-only tool calls, rejected sampling parameters, unsupported reasoning efforts — on knowing it is talking to the first-party OpenAI surface, and defaults them off. It takes that as a declaration rather than inferring it, because a base URL cannot answer the question: only this layer knows whether a URL is a faithful first-party route, a gateway, or a proxy with its own semantics. That decision already exists here, in `isCanonicalOpenAIBaseURL`, used by the GPT-5.6 Responses default. Declaring it from the same checks keeps it in one place instead of duplicating it downstream where the two could drift — and if the endpoint logic ever moves, it moves once. `firstPartyEndpoint` is declared on `OAIClientOptions` alongside the other agents SDK fields this layer already sets (`promptCache`, `_lc_stream_delay`, `includeReasoningContent`), so it typechecks against the current published SDK and is consumed once the SDK release lands. Two backward-compatibility expectations gain the field. That is the intended signal: they assert the full emitted config precisely so a new field cannot slip through unnoticed. * 🗂 fix: Keep GPT-6 Astra Out of the Assistants Catalogs [Codex P2] `sharedOpenAIModels` feeds `defaultModels[assistants]` and `[azureAssistants]` as well as the OpenAI and agents catalogs. Astra serves tool calls only from the Responses API, and the Assistants endpoints do not route through `getOpenAILLMConfig`, so listing it there offered a configuration the provider rejects. Astra now sits in its own `responsesOnlyOpenAIModels` list, spread into the OpenAI and agents catalogs only. Agents keeps it deliberately: that path does go through `getOpenAILLMConfig`. * 🎚 fix: Route GPT-6 Astra to the Responses API at Config Time [Codex P1] `max_tokens` was shaped into `max_completion_tokens` here, from `llmConfig.useResponsesApi`, while the agents SDK switched Astra to the Responses API later at invocation time — so the request reached an endpoint expecting `max_output_tokens` and was rejected. Deciding the API here fixes that, and generalizes: any field whose shape depends on the endpoint has the same problem, so the choice has to precede the shaping. It also lands before Azure replaces `llmConfig.model` with a deployment name, which is the only point that still knows the real model id. Unlike the GPT-5.6 rule above, this does not depend on reasoning parameters. Astra serves tool calls only from Responses, and OpenAI recommends Responses for it generally, so a static decision loses nothing — which was the reason the SDK was deciding it dynamically. The existing guards carry over: OpenRouter, custom gateways, and an explicit `dropParams` opt-out keep their configured path. The agents SDK is still required, for the request shaping Astra needs on either API (danny-avila/agents#506); it no longer decides routing. * 🏷 fix: Record the Served Model Before Azure Overwrites It [Codex P1] `llmConfig.model` is unconditionally replaced with the Azure deployment name, so the agents SDK received an alias and could not tell which model it was talking to — a deployment not literally named `gpt-6-astra` got none of the request shaping Astra requires. Records the real id in `servedModel` immediately before the overwrite, which is the last point that still knows it. The SDK prefers it over `model` when deciding which model-specific constraints apply (danny-avila/agents#506). Set for every first-party Azure request rather than only Astra: "what this deployment serves" is a fact this layer knows and the SDK decides what to do with, and keeping model-specific knowledge out of here is the point of the split. The backward-compatibility expectation gains the field, which is that test doing its job. * 🎛 fix: Do Not Let a reasoning_effort Drop Disable Astra Routing [Codex P1] The Astra routing reused the GPT-5.6 opt-out wholesale, which treats `dropParams: ['reasoning_effort']` as opting out of the Responses API. That holds for the 5.6 default, which routes *because* of reasoning — removing the effort removes its reason to route. Astra's routing is not reasoning-driven, so a rule clearing an unsupported stored effort silently put it back on Chat Completions, where its tool calls fail. Astra now honors only an explicit `useResponsesApi` drop. Reusing the guard without re-deriving whether each clause applied was the error. * ✂️ fix: Scope GPT-6 Astra to the First-Party OpenAI Endpoint [Codex P1 + P2] Two findings, both Azure, resolved by removing the Azure support rather than extending it. Azure's first-party hosts do not satisfy `isCanonicalOpenAIBaseURL`, so a configured `*.openai.azure.com` was neither routed to Responses nor declared first-party; and moving Astra out of `sharedOpenAIModels` also removed it from the Azure chat fallback, since `getOpenAIModels({ azure: true })` resolves through `defaultModels[azureAssistants]`. Both are moot once the scope is right: OpenAI does not document Astra as available on Azure OpenAI. Its model page lists the API and OpenAI's own subscription plans, with no third-party cloud surface. The Azure handling was support for a deployment the model is not documented to reach, and it produced most of the review findings on this branch and its agents SDK counterpart. Routing and the first-party declaration are now scoped to `EModelEndpoint.openAI`, and the Azure-only `servedModel` recording is gone along with its SDK counterpart. Astra staying out of the Azure catalogs is now the intended outcome rather than a regression. * 🚫 fix: Keep Astra Out of the Initial Azure Catalog [Codex P1] `initialModelsConfig` feeds the OpenAI catalog to Azure OpenAI as well, so scoping Astra to the first-party OpenAI endpoint left it advertised on Azure anyway — and being first in the list, it could become the default selection where no prior choice exists. The server's Azure fallback already excludes it, so the client would offer a model the server then rejects. The Azure initial catalog now uses the OpenAI list minus the Responses-only models, following the same scoping decision the routing guard makes. The initial Assistants catalog was already safe: `fitlerAssistantModels` requires `gpt-4` or `gpt-3.5`, which Astra does not match. Pinned that in a test rather than leaving it to a regex written for another purpose. * ⬆️ chore: Bump `@librechat/agents` to 3.7.20 for GPT-6 Astra Constraints
This commit is contained in:
Danny Avila
GitHub
parent
9a64e85b1e
commit
c00acb36a0
14 changed files with 337 additions and 10 deletions
|
|
@ -587,7 +587,7 @@ GOOGLE_KEY=user_provided
|
|||
#============#
|
||||
|
||||
OPENAI_API_KEY=user_provided
|
||||
# OPENAI_MODELS=gpt-5.6,gpt-5.6-terra,gpt-5.6-luna,gpt-5.5,gpt-5.5-pro,chat-latest,gpt-5.4,gpt-5.4-pro,gpt-5.4-mini,gpt-5.4-nano,gpt-5.3-codex,gpt-5.2,gpt-5,gpt-5-codex,gpt-5-mini,gpt-5-nano,o3-pro,o3,o4-mini,gpt-4.1,gpt-4.1-mini,gpt-4.1-nano,o3-mini,o1-pro,o1,gpt-4o,gpt-4o-mini
|
||||
# OPENAI_MODELS=gpt-6-astra,gpt-5.6,gpt-5.6-terra,gpt-5.6-luna,gpt-5.5,gpt-5.5-pro,chat-latest,gpt-5.4,gpt-5.4-pro,gpt-5.4-mini,gpt-5.4-nano,gpt-5.3-codex,gpt-5.2,gpt-5,gpt-5-codex,gpt-5-mini,gpt-5-nano,o3-pro,o3,o4-mini,gpt-4.1,gpt-4.1-mini,gpt-4.1-nano,o3-mini,o1-pro,o1,gpt-4o,gpt-4o-mini
|
||||
|
||||
DEBUG_OPENAI=false
|
||||
|
||||
|
|
|
|||
|
|
@ -46,7 +46,7 @@
|
|||
"@azure/storage-blob": "^12.30.0",
|
||||
"@google/genai": "^2.8.0",
|
||||
"@keyv/redis": "5.1.6",
|
||||
"@librechat/agents": "^3.7.19",
|
||||
"@librechat/agents": "^3.7.20",
|
||||
"@librechat/api": "*",
|
||||
"@librechat/data-schemas": "*",
|
||||
"@microsoft/microsoft-graph-client": "^3.0.7",
|
||||
|
|
|
|||
10
package-lock.json
generated
10
package-lock.json
generated
|
|
@ -63,7 +63,7 @@
|
|||
"@azure/storage-blob": "^12.30.0",
|
||||
"@google/genai": "^2.8.0",
|
||||
"@keyv/redis": "5.1.6",
|
||||
"@librechat/agents": "^3.7.19",
|
||||
"@librechat/agents": "^3.7.20",
|
||||
"@librechat/api": "*",
|
||||
"@librechat/data-schemas": "*",
|
||||
"@microsoft/microsoft-graph-client": "^3.0.7",
|
||||
|
|
@ -10627,9 +10627,9 @@
|
|||
}
|
||||
},
|
||||
"node_modules/@librechat/agents": {
|
||||
"version": "3.7.19",
|
||||
"resolved": "https://registry.npmjs.org/@librechat/agents/-/agents-3.7.19.tgz",
|
||||
"integrity": "sha512-/dHHKBvLoXjX1hFkDJBGP5CjkMp1vS+e3OyLqffa4c7JCenFhOHThBDUdZmCDvoEylRhima0h3fYl/JqCR8+hQ==",
|
||||
"version": "3.7.20",
|
||||
"resolved": "https://registry.npmjs.org/@librechat/agents/-/agents-3.7.20.tgz",
|
||||
"integrity": "sha512-YnfBpC5MK7QDgP+6VpmOYBTA2mXbaA+fDggfi7n/bsB56FSqenCzizSpC+Nn83lhwJz7GNntJpYNX53nRhFGgw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@anthropic-ai/sdk": "^0.115.0",
|
||||
|
|
@ -42681,7 +42681,7 @@
|
|||
"@azure/storage-blob": "^12.30.0",
|
||||
"@google/genai": "^2.8.0",
|
||||
"@keyv/redis": "5.1.6",
|
||||
"@librechat/agents": "^3.7.19",
|
||||
"@librechat/agents": "^3.7.20",
|
||||
"@librechat/data-schemas": "*",
|
||||
"@modelcontextprotocol/sdk": "^1.30.0",
|
||||
"@opentelemetry/api": "^1.9.0",
|
||||
|
|
|
|||
|
|
@ -113,7 +113,7 @@
|
|||
"@azure/storage-blob": "^12.30.0",
|
||||
"@google/genai": "^2.8.0",
|
||||
"@keyv/redis": "5.1.6",
|
||||
"@librechat/agents": "^3.7.19",
|
||||
"@librechat/agents": "^3.7.20",
|
||||
"@librechat/data-schemas": "*",
|
||||
"@modelcontextprotocol/sdk": "^1.30.0",
|
||||
"@opentelemetry/api": "^1.9.0",
|
||||
|
|
|
|||
|
|
@ -33,6 +33,7 @@ describe('getOpenAIConfig - Backward Compatibility', () => {
|
|||
streaming: true,
|
||||
model: 'gpt-5-nano',
|
||||
useResponsesApi: true,
|
||||
firstPartyEndpoint: true,
|
||||
user: 'some-user',
|
||||
apiKey: 'sk-proj-somekey',
|
||||
reasoning: {
|
||||
|
|
|
|||
|
|
@ -709,6 +709,122 @@ describe('getOpenAILLMConfig', () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe('GPT-6 Astra Responses API routing', () => {
|
||||
const astraConfig = (overrides: Record<string, unknown> = {}) =>
|
||||
getOpenAILLMConfig({
|
||||
apiKey: 'test-api-key',
|
||||
streaming: true,
|
||||
endpoint: EModelEndpoint.openAI,
|
||||
modelOptions: { model: 'gpt-6-astra' },
|
||||
...overrides,
|
||||
});
|
||||
|
||||
it('routes every Astra turn to the Responses API, not only reasoning ones', () => {
|
||||
expect(astraConfig().llmConfig).toHaveProperty('useResponsesApi', true);
|
||||
});
|
||||
|
||||
/**
|
||||
* The reason routing is decided here rather than at invocation time: the
|
||||
* max-tokens field is shaped from `useResponsesApi`, so switching APIs later
|
||||
* would send `max_completion_tokens` to an endpoint expecting
|
||||
* `max_output_tokens`.
|
||||
*/
|
||||
it('shapes max tokens for the API it actually uses', () => {
|
||||
const result = astraConfig({
|
||||
modelOptions: { model: 'gpt-6-astra', max_tokens: 4096 },
|
||||
});
|
||||
const kwargs = (result.llmConfig.modelKwargs ?? {}) as Record<string, unknown>;
|
||||
expect(kwargs).not.toHaveProperty('max_completion_tokens');
|
||||
expect(kwargs.max_output_tokens ?? result.llmConfig.maxTokens).toBeDefined();
|
||||
});
|
||||
|
||||
it('keeps routing when a drop rule only clears reasoning_effort', () => {
|
||||
/**
|
||||
* Unlike the GPT-5.6 default, Astra's routing is not reasoning-driven, so
|
||||
* a rule clearing an unsupported stored effort must not disable it.
|
||||
*/
|
||||
expect(astraConfig({ dropParams: ['reasoning_effort'] }).llmConfig).toHaveProperty(
|
||||
'useResponsesApi',
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it('respects an explicit opt-out', () => {
|
||||
expect(astraConfig({ dropParams: ['useResponsesApi'] }).llmConfig).not.toHaveProperty(
|
||||
'useResponsesApi',
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it('leaves a custom gateway on its configured path', () => {
|
||||
expect(astraConfig({ baseURL: 'https://gateway.internal/v1' }).llmConfig).not.toHaveProperty(
|
||||
'useResponsesApi',
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it('does not change routing for other models', () => {
|
||||
const result = getOpenAILLMConfig({
|
||||
apiKey: 'test-api-key',
|
||||
streaming: true,
|
||||
endpoint: EModelEndpoint.openAI,
|
||||
modelOptions: { model: 'gpt-5.5' },
|
||||
});
|
||||
expect(result.llmConfig).not.toHaveProperty('useResponsesApi', true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('First-party endpoint declaration', () => {
|
||||
/**
|
||||
* The agents SDK gates its model-specific request constraints on this flag
|
||||
* and defaults them off, rather than inferring the endpoint from a base
|
||||
* URL. Only this layer can tell a faithful first-party route from a
|
||||
* gateway, so the decision is made here and declared downstream.
|
||||
*/
|
||||
const configFor = (overrides: Record<string, unknown> = {}) =>
|
||||
getOpenAILLMConfig({
|
||||
apiKey: 'test-api-key',
|
||||
streaming: true,
|
||||
endpoint: EModelEndpoint.openAI,
|
||||
modelOptions: { model: 'gpt-6-astra' },
|
||||
...overrides,
|
||||
});
|
||||
|
||||
it('declares the first-party endpoint for canonical OpenAI', () => {
|
||||
expect(configFor().llmConfig).toHaveProperty('firstPartyEndpoint', true);
|
||||
});
|
||||
|
||||
it('declares it for an explicit api.openai.com base URL', () => {
|
||||
expect(configFor({ baseURL: 'https://api.openai.com/v1' }).llmConfig).toHaveProperty(
|
||||
'firstPartyEndpoint',
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
/**
|
||||
* Astra is not documented as available on Azure OpenAI, and Azure's
|
||||
* first-party hosts do not satisfy the OpenAI-host check, so declaring it
|
||||
* there would claim a surface this cannot verify.
|
||||
*/
|
||||
it('does not declare it for Azure OpenAI', () => {
|
||||
expect(configFor({ endpoint: EModelEndpoint.azureOpenAI }).llmConfig).not.toHaveProperty(
|
||||
'firstPartyEndpoint',
|
||||
);
|
||||
});
|
||||
|
||||
it('does not declare it for a custom gateway base URL', () => {
|
||||
expect(configFor({ baseURL: 'https://gateway.internal/v1' }).llmConfig).not.toHaveProperty(
|
||||
'firstPartyEndpoint',
|
||||
);
|
||||
});
|
||||
|
||||
it('does not declare it for a non-OpenAI endpoint', () => {
|
||||
expect(configFor({ endpoint: EModelEndpoint.custom }).llmConfig).not.toHaveProperty(
|
||||
'firstPartyEndpoint',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('GPT-5.6 Responses API Requirement', () => {
|
||||
it.each(['gpt-5.6-terra', 'gpt-5.6-luna', 'gpt-5.6-sol', 'gpt-5.6'])(
|
||||
'should default to Responses API for %s when reasoning_effort is set',
|
||||
|
|
|
|||
|
|
@ -130,6 +130,24 @@ function isOpenAIEndpoint(endpoint?: EModelEndpoint | string | null): boolean {
|
|||
*/
|
||||
const responsesApiRequiredPattern = /\bgpt-5\.6\b/;
|
||||
|
||||
/**
|
||||
* Models that take the Responses API for every turn, not only reasoning ones.
|
||||
* OpenAI's guidance for GPT-6 Astra is to use Responses, and tool calls require
|
||||
* it outright.
|
||||
*
|
||||
* Decided here rather than in the agents SDK at invocation time: the max-tokens
|
||||
* field below is shaped from `useResponsesApi`, so a later switch would send
|
||||
* `max_completion_tokens` to an endpoint expecting `max_output_tokens`. Config
|
||||
* time is also the only place that knows the model before Azure replaces it
|
||||
* with a deployment name.
|
||||
* @see https://developers.openai.com/api/docs/guides/latest-model
|
||||
*/
|
||||
const responsesApiPreferredPattern = /^gpt-6-astra(?:-|$)/i;
|
||||
|
||||
function prefersResponsesApi(model?: string): boolean {
|
||||
return typeof model === 'string' && responsesApiPreferredPattern.test(model);
|
||||
}
|
||||
|
||||
function requiresResponsesApiForReasoning({
|
||||
model,
|
||||
reasoningEffort,
|
||||
|
|
@ -848,6 +866,14 @@ export function getOpenAILLMConfig({
|
|||
const responsesApiOptedOut =
|
||||
dropParams != null &&
|
||||
(dropParams.includes('reasoning_effort') || dropParams.includes('useResponsesApi'));
|
||||
/**
|
||||
* The GPT-5.6 default above is reasoning-driven, so dropping `reasoning_effort`
|
||||
* removes its reason to route. Astra's is not: it takes Responses for every
|
||||
* turn, and a drop rule clearing an unsupported stored effort must not also
|
||||
* disable its routing. Only an explicit `useResponsesApi` drop does that.
|
||||
*/
|
||||
const responsesApiExplicitlyOptedOut =
|
||||
dropParams != null && dropParams.includes('useResponsesApi');
|
||||
if (
|
||||
!useOpenRouter &&
|
||||
endpoint === EModelEndpoint.openAI &&
|
||||
|
|
@ -860,6 +886,38 @@ export function getOpenAILLMConfig({
|
|||
llmConfig.useResponsesApi = true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Route GPT-6 Astra to the Responses API for every turn. Unlike the GPT-5.6
|
||||
* rule above this does not depend on reasoning params: Astra serves tool calls
|
||||
* only from Responses, and OpenAI recommends it generally.
|
||||
*/
|
||||
if (
|
||||
!useOpenRouter &&
|
||||
endpoint === EModelEndpoint.openAI &&
|
||||
isCanonicalOpenAIBaseURL(baseURL) &&
|
||||
llmConfig.useResponsesApi == null &&
|
||||
!responsesApiExplicitlyOptedOut &&
|
||||
prefersResponsesApi(llmConfig.model)
|
||||
) {
|
||||
llmConfig.useResponsesApi = true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Declare the first-party surface for the agents SDK's model-specific request
|
||||
* constraints. Computed here, from the same checks the Responses default
|
||||
* above uses, so the decision lives in one place: OpenRouter and custom
|
||||
* gateways route through endpoints whose contract is not OpenAI's, and only
|
||||
* this layer can tell them apart.
|
||||
*
|
||||
* Scoped to the canonical OpenAI endpoint. Astra is not documented as
|
||||
* available on Azure OpenAI, and Azure's first-party hosts do not satisfy the
|
||||
* OpenAI-host check, so declaring it there would claim a surface this cannot
|
||||
* verify.
|
||||
*/
|
||||
if (!useOpenRouter && endpoint === EModelEndpoint.openAI && isCanonicalOpenAIBaseURL(baseURL)) {
|
||||
llmConfig.firstPartyEndpoint = true;
|
||||
}
|
||||
|
||||
if (!useOpenRouter) {
|
||||
hasModelKwargs =
|
||||
applyReasoningConfig({
|
||||
|
|
|
|||
|
|
@ -36,6 +36,15 @@ export type OAIClientOptions = Omit<OpenAIClientOptions, 'verbosity'> & {
|
|||
includeReasoningContent?: boolean;
|
||||
promptCache?: boolean;
|
||||
promptCacheTtl?: '5m' | '1h';
|
||||
/**
|
||||
* Declares that this client talks to the first-party OpenAI surface, which is
|
||||
* what gates the agents SDK's model-specific request constraints (GPT-6
|
||||
* Astra: Responses-only tool calls, rejected sampling parameters,
|
||||
* unsupported reasoning efforts). The SDK defaults them off and takes this as
|
||||
* a declaration rather than inferring it from a base URL, because only this
|
||||
* layer knows whether a URL is a faithful first-party route or a gateway.
|
||||
*/
|
||||
firstPartyEndpoint?: boolean;
|
||||
_lc_stream_delay?: number;
|
||||
verbosity?: string | null;
|
||||
};
|
||||
|
|
|
|||
|
|
@ -91,6 +91,24 @@ describe('Gemini 3.8 Flash', () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe('GPT-6 Astra', () => {
|
||||
it('resolves 1.05M context and 128K output', () => {
|
||||
expect(getModelMaxTokens('gpt-6-astra', EModelEndpoint.openAI)).toBe(1050000);
|
||||
expect(getModelMaxOutputTokens('gpt-6-astra', EModelEndpoint.openAI)).toBe(128000);
|
||||
});
|
||||
|
||||
it('matches its own key for snapshots and provider prefixes', () => {
|
||||
for (const model of [
|
||||
'gpt-6-astra-2026-04-30',
|
||||
'openai/gpt-6-astra',
|
||||
'gpt-6-astra-2026-04-30/openai',
|
||||
]) {
|
||||
expect(getModelMaxTokens(model, EModelEndpoint.openAI)).toBe(1050000);
|
||||
expect(getModelMaxOutputTokens(model, EModelEndpoint.openAI)).toBe(128000);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('Qwen3.5 and later generations', () => {
|
||||
it('resolves 262K for the vLLM model id that regressed to the qwen3 window', () => {
|
||||
expect(getModelMaxTokens('Qwen/Qwen3.5-397B-A17B-FP8', EModelEndpoint.openAI)).toBe(262144);
|
||||
|
|
|
|||
|
|
@ -64,6 +64,7 @@ const openAIModels = {
|
|||
'gpt-5.6': 1050000,
|
||||
'gpt-5.6-terra': 1050000,
|
||||
'gpt-5.6-luna': 1050000,
|
||||
'gpt-6-astra': 1050000, // >272K input prices at the long-context tier (2x input/cache, 1.5x output)
|
||||
'chat-latest': 400000,
|
||||
'gpt-5-mini': 400000,
|
||||
'gpt-5-nano': 400000,
|
||||
|
|
@ -508,6 +509,7 @@ export const modelMaxOutputs = {
|
|||
'gpt-5.6': 128000,
|
||||
'gpt-5.6-terra': 128000,
|
||||
'gpt-5.6-luna': 128000,
|
||||
'gpt-6-astra': 128000,
|
||||
'chat-latest': 128000,
|
||||
'gpt-5-mini': 128000,
|
||||
'gpt-5-nano': 128000,
|
||||
|
|
|
|||
45
packages/data-provider/specs/astra-catalog.spec.ts
Normal file
45
packages/data-provider/specs/astra-catalog.spec.ts
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
import { EModelEndpoint } from '../src/schemas';
|
||||
import { defaultModels, initialModelsConfig } from '../src/config';
|
||||
|
||||
/**
|
||||
* GPT-6 Astra serves tool calls only from the Responses API. The Assistants
|
||||
* endpoints do not route through `getOpenAILLMConfig`, so listing it there
|
||||
* would offer a configuration the provider rejects.
|
||||
*/
|
||||
describe('GPT-6 Astra catalog placement', () => {
|
||||
it('is offered on the endpoints that route through the OpenAI config', () => {
|
||||
expect(defaultModels[EModelEndpoint.openAI]).toContain('gpt-6-astra');
|
||||
expect(defaultModels[EModelEndpoint.agents]).toContain('gpt-6-astra');
|
||||
});
|
||||
|
||||
it('is kept out of both Assistants catalogs', () => {
|
||||
expect(defaultModels[EModelEndpoint.assistants]).not.toContain('gpt-6-astra');
|
||||
expect(defaultModels[EModelEndpoint.azureAssistants]).not.toContain('gpt-6-astra');
|
||||
});
|
||||
|
||||
it('is kept out of the initial Azure and Assistants catalogs', () => {
|
||||
/**
|
||||
* Azure shares the OpenAI list, but Astra gets neither Responses routing nor
|
||||
* its request constraints there — and being first in the list would let it
|
||||
* become the default selection.
|
||||
*/
|
||||
expect(initialModelsConfig[EModelEndpoint.azureOpenAI]).not.toContain('gpt-6-astra');
|
||||
expect(initialModelsConfig[EModelEndpoint.assistants]).not.toContain('gpt-6-astra');
|
||||
});
|
||||
|
||||
it('keeps Astra in the initial catalogs that can run it', () => {
|
||||
expect(initialModelsConfig[EModelEndpoint.openAI]).toContain('gpt-6-astra');
|
||||
expect(initialModelsConfig[EModelEndpoint.agents]).toContain('gpt-6-astra');
|
||||
});
|
||||
|
||||
it('does not disturb the rest of the shared OpenAI catalog', () => {
|
||||
for (const endpoint of [
|
||||
EModelEndpoint.openAI,
|
||||
EModelEndpoint.agents,
|
||||
EModelEndpoint.assistants,
|
||||
EModelEndpoint.azureAssistants,
|
||||
]) {
|
||||
expect(defaultModels[endpoint]).toContain('gpt-5.6');
|
||||
}
|
||||
});
|
||||
});
|
||||
|
|
@ -2695,6 +2695,15 @@ export const alternateName = {
|
|||
[KnownEndpoints.helicone]: 'Helicone',
|
||||
};
|
||||
|
||||
/**
|
||||
* Models the Assistants endpoints cannot run. GPT-6 Astra serves tool calls only
|
||||
* from the Responses API, and the Assistants surface does not route through
|
||||
* `getOpenAILLMConfig`, so listing it there would offer a configuration the
|
||||
* provider rejects. Kept out of `sharedOpenAIModels`, which both Assistants
|
||||
* catalogs consume.
|
||||
*/
|
||||
const responsesOnlyOpenAIModels = ['gpt-6-astra'];
|
||||
|
||||
const sharedOpenAIModels = [
|
||||
'gpt-5.6',
|
||||
'gpt-5.6-terra',
|
||||
|
|
@ -2797,7 +2806,8 @@ export const bedrockModels = [
|
|||
export const defaultModels = {
|
||||
[EModelEndpoint.azureAssistants]: sharedOpenAIModels,
|
||||
[EModelEndpoint.assistants]: [...sharedOpenAIModels, 'chatgpt-4o-latest'],
|
||||
[EModelEndpoint.agents]: sharedOpenAIModels, // TODO: Add agent models (agentsModels)
|
||||
// TODO: Add agent models (agentsModels)
|
||||
[EModelEndpoint.agents]: [...responsesOnlyOpenAIModels, ...sharedOpenAIModels],
|
||||
[EModelEndpoint.google]: [
|
||||
// Gemini 3.8 Models
|
||||
'gemini-3.8-flash',
|
||||
|
|
@ -2822,6 +2832,7 @@ export const defaultModels = {
|
|||
],
|
||||
[EModelEndpoint.anthropic]: sharedAnthropicModels,
|
||||
[EModelEndpoint.openAI]: [
|
||||
...responsesOnlyOpenAIModels,
|
||||
...sharedOpenAIModels,
|
||||
'chatgpt-4o-latest',
|
||||
'gpt-4-vision-preview',
|
||||
|
|
@ -2837,12 +2848,22 @@ const fitlerAssistantModels = (str: string) => {
|
|||
|
||||
const openAIModels = defaultModels[EModelEndpoint.openAI];
|
||||
|
||||
/**
|
||||
* The OpenAI catalog without the models only the first-party OpenAI endpoint
|
||||
* can run. Azure OpenAI shares this list, but Astra is neither routed to the
|
||||
* Responses API nor given its request constraints there, and listing it first
|
||||
* would let it become the default selection.
|
||||
*/
|
||||
const nonResponsesOnlyOpenAIModels = openAIModels.filter(
|
||||
(model) => !responsesOnlyOpenAIModels.includes(model),
|
||||
);
|
||||
|
||||
export const initialModelsConfig: TModelsConfig = {
|
||||
initial: [],
|
||||
[EModelEndpoint.openAI]: openAIModels,
|
||||
[EModelEndpoint.assistants]: openAIModels.filter(fitlerAssistantModels),
|
||||
[EModelEndpoint.agents]: openAIModels, // TODO: Add agent models (agentsModels)
|
||||
[EModelEndpoint.azureOpenAI]: openAIModels,
|
||||
[EModelEndpoint.azureOpenAI]: nonResponsesOnlyOpenAIModels,
|
||||
[EModelEndpoint.google]: defaultModels[EModelEndpoint.google],
|
||||
[EModelEndpoint.anthropic]: defaultModels[EModelEndpoint.anthropic],
|
||||
[EModelEndpoint.bedrock]: defaultModels[EModelEndpoint.bedrock],
|
||||
|
|
|
|||
|
|
@ -470,6 +470,59 @@ describe('getMultiplier', () => {
|
|||
}
|
||||
});
|
||||
|
||||
it('should use the documented gpt-6-astra pricing', () => {
|
||||
expect(tokenValues['gpt-6-astra']).toEqual({ prompt: 10, completion: 50 });
|
||||
expect(cacheTokenValues['gpt-6-astra']).toEqual({ write: 12.5, read: 1 });
|
||||
expect(premiumTokenValues['gpt-6-astra']).toEqual({
|
||||
threshold: 272000,
|
||||
prompt: 20,
|
||||
completion: 75,
|
||||
});
|
||||
expect(premiumCacheTokenValues['gpt-6-astra']).toEqual({
|
||||
threshold: 272000,
|
||||
write: 25,
|
||||
read: 2,
|
||||
});
|
||||
});
|
||||
|
||||
it('should bill gpt-6-astra cache writes at the documented 1.25x input surcharge', () => {
|
||||
expect(cacheTokenValues['gpt-6-astra'].write).toBeCloseTo(
|
||||
tokenValues['gpt-6-astra'].prompt * 1.25,
|
||||
);
|
||||
});
|
||||
|
||||
it('should apply the documented gpt-6-astra long-context multipliers', () => {
|
||||
const standard = tokenValues['gpt-6-astra'];
|
||||
const premium = premiumTokenValues['gpt-6-astra'];
|
||||
const standardCache = cacheTokenValues['gpt-6-astra'];
|
||||
const premiumCache = premiumCacheTokenValues['gpt-6-astra'];
|
||||
/** >272K input: 2x input and cache rates, 1.5x output, for the full request. */
|
||||
expect(premium.prompt).toBeCloseTo(standard.prompt * 2);
|
||||
expect(premium.completion).toBeCloseTo(standard.completion * 1.5);
|
||||
expect(premiumCache.write).toBeCloseTo(standardCache.write * 2);
|
||||
expect(premiumCache.read).toBeCloseTo(standardCache.read * 2);
|
||||
});
|
||||
|
||||
it('should resolve gpt-6-astra to its own key rather than a gpt-6 prefix', () => {
|
||||
for (const model of [
|
||||
'gpt-6-astra',
|
||||
'gpt-6-astra-2026-04-30',
|
||||
'openai/gpt-6-astra',
|
||||
'gpt-6-astra/openai',
|
||||
]) {
|
||||
expect(getValueKey(model)).toBe('gpt-6-astra');
|
||||
expect(getMultiplier({ model, tokenType: 'prompt' })).toBe(10);
|
||||
expect(getMultiplier({ model, tokenType: 'completion' })).toBe(50);
|
||||
}
|
||||
});
|
||||
|
||||
it('should charge gpt-6-astra premium rates only past the 272K threshold', () => {
|
||||
const model = 'gpt-6-astra';
|
||||
expect(getMultiplier({ model, tokenType: 'prompt', inputTokenCount: 272000 })).toBe(10);
|
||||
expect(getMultiplier({ model, tokenType: 'prompt', inputTokenCount: 272001 })).toBe(20);
|
||||
expect(getMultiplier({ model, tokenType: 'completion', inputTokenCount: 272001 })).toBe(75);
|
||||
});
|
||||
|
||||
it('should use the documented gpt-5.6 pricing', () => {
|
||||
const expectedPricing = {
|
||||
'gpt-5.6': {
|
||||
|
|
|
|||
|
|
@ -150,6 +150,7 @@ export const tokenValues: Record<string, { prompt: number; completion: number }>
|
|||
'gpt-5.6': { prompt: 4, completion: 20 },
|
||||
'gpt-5.6-terra': { prompt: 2, completion: 12 },
|
||||
'gpt-5.6-luna': { prompt: 0.2, completion: 1.2 },
|
||||
'gpt-6-astra': { prompt: 10, completion: 50 },
|
||||
'chat-latest': { prompt: 5, completion: 30 },
|
||||
'gpt-5-chat-latest': { prompt: 1.25, completion: 10 },
|
||||
'gpt-5.1-chat-latest': { prompt: 1.25, completion: 10 },
|
||||
|
|
@ -389,6 +390,7 @@ export const cacheTokenValues: Record<string, { write: number; read: number }> =
|
|||
'gpt-5.6': { write: 5, read: 0.4 },
|
||||
'gpt-5.6-terra': { write: 2.5, read: 0.2 },
|
||||
'gpt-5.6-luna': { write: 0.25, read: 0.02 },
|
||||
'gpt-6-astra': { write: 12.5, read: 1 },
|
||||
'chat-latest': { write: 5, read: 0.5 },
|
||||
'gpt-5-chat-latest': { write: 1.25, read: 0.125 },
|
||||
'gpt-5.1-chat-latest': { write: 1.25, read: 0.125 },
|
||||
|
|
@ -445,6 +447,7 @@ export const premiumTokenValues: Record<
|
|||
'gpt-5.6': { threshold: 272000, prompt: 8, completion: 30 },
|
||||
'gpt-5.6-terra': { threshold: 272000, prompt: 4, completion: 18 },
|
||||
'gpt-5.6-luna': { threshold: 272000, prompt: 0.4, completion: 1.8 },
|
||||
'gpt-6-astra': { threshold: 272000, prompt: 20, completion: 75 },
|
||||
'grok-4.5': { threshold: 200000, prompt: 4, completion: 12 },
|
||||
'grok-4-5': { threshold: 200000, prompt: 4, completion: 12 },
|
||||
'grok-4.6': { threshold: 200000, prompt: 4, completion: 12 },
|
||||
|
|
@ -466,6 +469,7 @@ export const premiumCacheTokenValues: Record<
|
|||
'gpt-5.6': { threshold: 272000, write: 10, read: 0.8 },
|
||||
'gpt-5.6-terra': { threshold: 272000, write: 5, read: 0.4 },
|
||||
'gpt-5.6-luna': { threshold: 272000, write: 0.5, read: 0.04 },
|
||||
'gpt-6-astra': { threshold: 272000, write: 25, read: 2 },
|
||||
};
|
||||
|
||||
export function createTxMethods(
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue