mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-08-04 14:57:42 +00:00
✳️ feat: Claude Opus 5 Support (#14422)
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
Some checks are pending
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
* ✳️ feat: Claude Opus 5 Support - Add claude-opus-5 to Anthropic/Bedrock model lists, token maps, and pricing - Extend requiresExplicitThinkingDisabled to Opus 5 so thinking-off sticks - Clamp xhigh/max effort to high when thinking is disabled (Opus 5 400) * 🪣 fix: Use Bedrock Inference Profiles and Add Vertex Opus Models Bare `anthropic.` Claude 4+ IDs are not invocable on-demand via Converse: Bedrock rejects them with "Retry your request with the ID or ARN of an inference profile that contains this model." Verified live against us-west-2 for Fable 5, Opus 5, Opus 4.8, Sonnet 5, Sonnet 4.6, Opus 4.6, Sonnet 4.5, Haiku 4.5, and Opus 4.1. Switch those defaults to the `global.` profile (no regional pricing premium); Opus 4.1 has no global profile, so it uses `us.`. Also add the modern Opus family to the Vertex defaults. `loadEndpoints` swaps the shared Anthropic list for the Vertex model names, so Opus was invisible to every Vertex deployment that did not enumerate models by hand. * 📋 chore: Cover Opus 5 Gaps From PR #14420 Picks up items from the parallel community PR by @jona7o: - Add claude-opus-5 to the librechat.example.yaml Vertex example (both the legacy array and the deploymentName map), which already lists Fable 5 and Opus 4.8 - Mention Opus 5 in the configureReasoning doc comment, and note that its early return is why the effort cap is enforced by the caller - Assert Opus 5 carries no long-context premium pricing - Cover the Sonnet 5 negative case for the effort cap, and the persisted disabled-object round-trip carrying an effort * 🌍 docs: Warn That Vertex Regional Endpoints Reject Modern Models Anthropic serves Sonnet 4.6 and earlier on specific Vertex regional endpoints; newer models (Opus 4.7+, Opus 5, Sonnet 5, Fable 5) require `global` or a multi-region location and 404 on a specific region. The `us-east5` default therefore cannot serve the Opus models added here, nor the Sonnet 5 entry that predates this branch. Documents the constraint at all three places an operator sets the region, and at the fallback itself. Leaves the default unchanged: switching it to `global` would silently alter data routing and residency for existing deployments, which is a separate call. * 🩹 fix: Restore PDF Exemption for Undated IDs and Gate Vertex Defaults Two issues raised in review: - BEDROCK_CLAUDE_4_PLUS_RE required a `-` after the major version, so it matched `claude-opus-4-8` but not undated IDs like `claude-opus-5`. Those models silently lost the Claude 4+ PDF exemption and fell back to the 4.5 MB limit. Sonnet 5 and Fable 5 were already affected before this branch; Fable/Mythos were also missing from the family alternation. - The Vertex defaults advertised models that only `global` and the multi-region locations serve, so a default `us-east5` deployment listed Opus choices that 404 on first request. Filter the built-in defaults by configured region instead of changing the region default, which would alter data routing for existing deployments. An explicit `vertex.models` list is the operator's choice and is never pruned. * 🧩 fix: Match Bare Claude IDs in the Bedrock PDF Exemption An application inference profile maps a LibreChat model ID with no `anthropic.` segment, so `claude-opus-5` failed the Claude 4+ check and fell back to the 4.5 MB PDF limit. Make the prefix optional and accept both segment orders, mirroring BEDROCK_CLAUDE_4PLUS_THINKING in librechat-data-provider, which matches on the family token for exactly this reason. Only reached for the Bedrock provider, so the looser prefix cannot leak into other endpoints. Verified Claude 3.x, Nova, Llama, Cohere, and Mistral IDs still fall through to the default limit. * 🧹 fix: Drop Retired Claude 3.5 Models From Bedrock Defaults The three Claude 3.5 entries reached end of life at AWS and return ResourceNotFoundException in every prefix form (bare, `us.`, `global.` — verified live against us-west-2), so selecting one was a hard error. Their modern equivalents are already in the list: Sonnet 5 / Sonnet 4.6 supersede the 3.5 Sonnets, and Haiku 4.5 supersedes 3.5 Haiku. Every remaining Anthropic default is now live-verified invocable. `.env.example` swaps its retired example ID for Haiku 4.5. * 🔒 refactor: Narrow Effort-Clamp Types Instead of Asserting Both clamp sites reached into loosely-typed containers with assertions: llm.ts used an `as unknown as { type?: string }` double assertion to read the thinking type, and the Bedrock parser cast `output_config` to `{ effort?: unknown }` before confirming it was an object. CLAUDE.md's type-safety rules call for narrowing over both. Adds `isThinkingDisabled` and `clampOutputConfigEffort` to librechat-data-provider, using `in`-operator narrowing and a type predicate so no assertion is needed at all. Both call sites now share one implementation rather than duplicating the clamp. Behavior is unchanged; existing clamp tests cover it.
This commit is contained in:
Danny Avila
GitHub
parent
6c97a7f467
commit
cd215150cc
18 changed files with 714 additions and 36 deletions
12
.env.example
12
.env.example
|
|
@ -282,12 +282,15 @@ PROXY=
|
|||
#============#
|
||||
|
||||
ANTHROPIC_API_KEY=user_provided
|
||||
# ANTHROPIC_MODELS=claude-fable-5,claude-opus-4-8,claude-opus-4-7,claude-sonnet-5,claude-sonnet-4-6,claude-opus-4-6,claude-opus-4-20250514,claude-3-7-sonnet-20250219,claude-3-5-sonnet-20241022,claude-3-5-haiku-20241022,claude-3-opus-20240229,claude-3-sonnet-20240229,claude-3-haiku-20240307
|
||||
# ANTHROPIC_MODELS=claude-fable-5,claude-opus-5,claude-opus-4-8,claude-opus-4-7,claude-sonnet-5,claude-sonnet-4-6,claude-opus-4-6,claude-opus-4-20250514,claude-3-7-sonnet-20250219,claude-3-5-sonnet-20241022,claude-3-5-haiku-20241022,claude-3-opus-20240229,claude-3-sonnet-20240229,claude-3-haiku-20240307
|
||||
# ANTHROPIC_REVERSE_PROXY=
|
||||
|
||||
# Set to true to use Anthropic models through Google Vertex AI instead of direct API
|
||||
# ANTHROPIC_USE_VERTEX=
|
||||
# Supports regional locations like us-east5 and multi-region locations: us, eu, global
|
||||
# IMPORTANT: specific regional endpoints (us-east5, europe-west1, ...) only serve Claude Sonnet 4.6
|
||||
# and earlier. Newer models (Opus 4.7+, Opus 5, Sonnet 5, Fable 5) require `global` or a multi-region
|
||||
# location (`us`/`eu`) and will 404 on a specific region. `global` also avoids the 10% regional premium.
|
||||
# ANTHROPIC_VERTEX_REGION=us-east5
|
||||
|
||||
#============#
|
||||
|
|
@ -348,8 +351,11 @@ ANTHROPIC_API_KEY=user_provided
|
|||
# BEDROCK_AWS_BEARER_TOKEN=yourBedrockApiKey
|
||||
|
||||
# Note: This example list is not meant to be exhaustive. If omitted, all known, supported model IDs will be included for you.
|
||||
# BEDROCK_AWS_MODELS=anthropic.claude-fable-5,anthropic.claude-opus-4-8,anthropic.claude-opus-4-7,anthropic.claude-sonnet-5,anthropic.claude-sonnet-4-6,anthropic.claude-opus-4-6-v1,anthropic.claude-3-5-sonnet-20240620-v1:0,meta.llama3-1-8b-instruct-v1:0
|
||||
# Cross-region inference model IDs: us.anthropic.claude-fable-5,us.anthropic.claude-opus-4-8,us.anthropic.claude-opus-4-7,us.anthropic.claude-sonnet-5,us.anthropic.claude-sonnet-4-6,us.anthropic.claude-opus-4-6-v1,global.anthropic.claude-opus-4-6-v1
|
||||
# Claude 4+ models cannot be invoked on-demand by their bare `anthropic.` foundation-model ID; Bedrock requires a
|
||||
# cross-region inference profile (`global.` or `us.`) for those. The `global.` profile has no regional pricing premium.
|
||||
# BEDROCK_AWS_MODELS=global.anthropic.claude-fable-5,global.anthropic.claude-opus-5,global.anthropic.claude-opus-4-8,global.anthropic.claude-opus-4-7,global.anthropic.claude-sonnet-5,global.anthropic.claude-sonnet-4-6,global.anthropic.claude-opus-4-6-v1,global.anthropic.claude-haiku-4-5-20251001-v1:0,meta.llama3-1-8b-instruct-v1:0
|
||||
# US-only routing alternative: us.anthropic.claude-fable-5,us.anthropic.claude-opus-5,us.anthropic.claude-opus-4-8,us.anthropic.claude-opus-4-7,us.anthropic.claude-sonnet-5,us.anthropic.claude-sonnet-4-6,us.anthropic.claude-opus-4-6-v1
|
||||
# List the profiles available to your account with: aws bedrock list-inference-profiles --region <your-region>
|
||||
|
||||
# See all Bedrock model IDs here: https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html#model-ids-arns
|
||||
|
||||
|
|
|
|||
|
|
@ -1544,6 +1544,37 @@ describe('Claude Model Tests', () => {
|
|||
});
|
||||
});
|
||||
|
||||
it('should return correct context length for Claude Opus 5 (1M)', () => {
|
||||
expect(getModelMaxTokens('claude-opus-5', EModelEndpoint.anthropic)).toBe(
|
||||
maxTokensMap[EModelEndpoint.anthropic]['claude-opus-5'],
|
||||
);
|
||||
expect(getModelMaxTokens('claude-opus-5')).toBe(
|
||||
maxTokensMap[EModelEndpoint.anthropic]['claude-opus-5'],
|
||||
);
|
||||
});
|
||||
|
||||
it('should return correct max output tokens for Claude Opus 5 (128K)', () => {
|
||||
const { getModelMaxOutputTokens } = require('@librechat/api');
|
||||
expect(getModelMaxOutputTokens('claude-opus-5', EModelEndpoint.anthropic)).toBe(
|
||||
maxOutputTokensMap[EModelEndpoint.anthropic]['claude-opus-5'],
|
||||
);
|
||||
});
|
||||
|
||||
it('should match model names correctly for Claude Opus 5', () => {
|
||||
const modelVariations = [
|
||||
'claude-opus-5',
|
||||
'claude-opus-5-20260701',
|
||||
'claude-opus-5-latest',
|
||||
'anthropic/claude-opus-5',
|
||||
'claude-opus-5/anthropic',
|
||||
'claude-opus-5-preview',
|
||||
];
|
||||
|
||||
modelVariations.forEach((model) => {
|
||||
expect(matchModelName(model, EModelEndpoint.anthropic)).toBe('claude-opus-5');
|
||||
});
|
||||
});
|
||||
|
||||
it('should return correct context length for Claude Fable 5 (1M)', () => {
|
||||
expect(getModelMaxTokens('claude-fable-5', EModelEndpoint.anthropic)).toBe(
|
||||
maxTokensMap[EModelEndpoint.anthropic]['claude-fable-5'],
|
||||
|
|
|
|||
|
|
@ -465,6 +465,9 @@ endpoints:
|
|||
# # Vertex AI region (optional, defaults to 'us-east5')
|
||||
# # Available regions: us-east5, us-central1, europe-west1, europe-west4, asia-southeast1
|
||||
# # Multi-region endpoints: us, eu, global
|
||||
# # IMPORTANT: specific regional endpoints only serve Claude Sonnet 4.6 and earlier. Newer
|
||||
# # models (Opus 4.7+, Opus 5, Sonnet 5, Fable 5) require "global" or a multi-region value
|
||||
# # ("us"/"eu") and will 404 on a specific region. "global" also avoids the 10% regional premium.
|
||||
# region: "us-east5"
|
||||
# # Path to Google service account key file (optional)
|
||||
# # If not specified, uses GOOGLE_SERVICE_KEY_FILE env var or default path (api/data/auth.json)
|
||||
|
|
@ -483,6 +486,7 @@ endpoints:
|
|||
# # Use this if you want the technical model IDs to show in the UI
|
||||
# # models:
|
||||
# # - "claude-fable-5"
|
||||
# # - "claude-opus-5"
|
||||
# # - "claude-opus-4-8"
|
||||
# # - "claude-sonnet-4-6"
|
||||
# # - "claude-3-7-sonnet-20250219"
|
||||
|
|
@ -496,6 +500,8 @@ endpoints:
|
|||
# models:
|
||||
# claude-fable-5:
|
||||
# deploymentName: claude-fable-5
|
||||
# claude-opus-5:
|
||||
# deploymentName: claude-opus-5
|
||||
# claude-opus-4.8:
|
||||
# deploymentName: claude-opus-4-8
|
||||
# claude-opus-4.5:
|
||||
|
|
|
|||
|
|
@ -101,9 +101,11 @@ function configureReasoning(
|
|||
const modelName = updatedOptions.model ?? '';
|
||||
|
||||
/**
|
||||
* Sonnet 5 runs adaptive thinking by default when the `thinking` field is
|
||||
* omitted, so honoring a user who turns thinking off requires sending an
|
||||
* explicit disabled config rather than leaving the field unset.
|
||||
* Sonnet 5 and Opus 5 run adaptive thinking by default when the `thinking`
|
||||
* field is omitted, so honoring a user who turns thinking off requires
|
||||
* sending an explicit disabled config rather than leaving the field unset.
|
||||
* This returns before effort is applied, which is why the Opus 5 effort cap
|
||||
* is enforced by the caller.
|
||||
*/
|
||||
if (!extendedOptions.thinking && modelName && requiresExplicitThinkingDisabled(modelName)) {
|
||||
updatedOptions.thinking = { type: 'disabled' } as AnthropicClientOptions['thinking'];
|
||||
|
|
|
|||
|
|
@ -1266,6 +1266,112 @@ describe('getLLMConfig', () => {
|
|||
expect((result.llmConfig.thinking as unknown as { type: string }).type).toBe('disabled');
|
||||
});
|
||||
|
||||
it('should send explicit disabled thinking for Opus 5 when thinking is off', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: { model: 'claude-opus-5', thinking: false },
|
||||
});
|
||||
|
||||
expect((result.llmConfig.thinking as unknown as { type: string }).type).toBe('disabled');
|
||||
});
|
||||
|
||||
it('should omit sampling parameters for Opus 5', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: {
|
||||
model: 'claude-opus-5',
|
||||
thinking: true,
|
||||
temperature: 0.7,
|
||||
topP: 0.9,
|
||||
topK: 40,
|
||||
},
|
||||
});
|
||||
|
||||
expect(result.llmConfig).not.toHaveProperty('temperature');
|
||||
expect(result.llmConfig).not.toHaveProperty('topP');
|
||||
expect(result.llmConfig).not.toHaveProperty('topK');
|
||||
});
|
||||
|
||||
it('should keep xhigh/max effort for Opus 5 while thinking is on', () => {
|
||||
(['xhigh', 'max'] as AnthropicEffort[]).forEach((effort) => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: { model: 'claude-opus-5', thinking: true, effort },
|
||||
});
|
||||
|
||||
expect(result.llmConfig.invocationKwargs?.output_config).toEqual({ effort });
|
||||
});
|
||||
});
|
||||
|
||||
it('should clamp xhigh/max effort to high for Opus 5 when thinking is disabled', () => {
|
||||
(['xhigh', 'max'] as AnthropicEffort[]).forEach((effort) => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: { model: 'claude-opus-5', thinking: false, effort },
|
||||
});
|
||||
|
||||
expect((result.llmConfig.thinking as unknown as { type: string }).type).toBe('disabled');
|
||||
expect(result.llmConfig.invocationKwargs?.output_config).toEqual({
|
||||
effort: AnthropicEffort.high,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it('should leave sub-xhigh effort untouched for Opus 5 when thinking is disabled', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: {
|
||||
model: 'claude-opus-5',
|
||||
thinking: false,
|
||||
effort: AnthropicEffort.medium,
|
||||
},
|
||||
});
|
||||
|
||||
expect(result.llmConfig.invocationKwargs?.output_config).toEqual({
|
||||
effort: AnthropicEffort.medium,
|
||||
});
|
||||
});
|
||||
|
||||
it('should clamp effort for Opus 5 when a disabled config round-trips from persistence', () => {
|
||||
/** Persisted model_parameters send the prior disabled object rather than
|
||||
* `false`, so the clamp must key off the resolved thinking config. */
|
||||
(['xhigh', 'max'] as AnthropicEffort[]).forEach((effort) => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: {
|
||||
model: 'claude-opus-5',
|
||||
thinking: { type: 'disabled' } as unknown as boolean,
|
||||
effort,
|
||||
},
|
||||
});
|
||||
|
||||
expect((result.llmConfig.thinking as unknown as { type: string }).type).toBe('disabled');
|
||||
expect(result.llmConfig.invocationKwargs?.output_config).toEqual({
|
||||
effort: AnthropicEffort.high,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it('should NOT clamp xhigh effort for Sonnet 5 when thinking is disabled', () => {
|
||||
/** Sonnet 5 also sends an explicit disabled config but has no effort cap. */
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: {
|
||||
model: 'claude-sonnet-5',
|
||||
thinking: false,
|
||||
effort: 'xhigh' as AnthropicEffort,
|
||||
},
|
||||
});
|
||||
|
||||
expect((result.llmConfig.thinking as unknown as { type: string }).type).toBe('disabled');
|
||||
expect(result.llmConfig.invocationKwargs?.output_config).toEqual({ effort: 'xhigh' });
|
||||
});
|
||||
|
||||
it('should NOT clamp xhigh effort for Opus 4.8 when thinking is disabled', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: {
|
||||
model: 'claude-opus-4-8',
|
||||
thinking: false,
|
||||
effort: 'xhigh' as AnthropicEffort,
|
||||
},
|
||||
});
|
||||
|
||||
expect(result.llmConfig.invocationKwargs?.output_config).toEqual({ effort: 'xhigh' });
|
||||
});
|
||||
|
||||
it('should omit sampling parameters for Sonnet 5', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: {
|
||||
|
|
|
|||
|
|
@ -2,8 +2,10 @@ import { Agent } from 'undici';
|
|||
import { logger } from '@librechat/data-schemas';
|
||||
import { AnthropicClientOptions } from '@librechat/agents';
|
||||
import {
|
||||
anthropicSettings,
|
||||
clampOutputConfigEffort,
|
||||
omitsSamplingParameters,
|
||||
isThinkingDisabled,
|
||||
anthropicSettings,
|
||||
removeNullishValues,
|
||||
ThinkingDisplay,
|
||||
AuthKeys,
|
||||
|
|
@ -256,6 +258,16 @@ function getLLMConfig(
|
|||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Opus 5 rejects `xhigh`/`max` effort while thinking is disabled (400).
|
||||
* `configureReasoning` returns before setting effort on the disabled path, so
|
||||
* the value applied just above is the one that would ship — clamp it to the
|
||||
* highest level the model accepts in that combination.
|
||||
*/
|
||||
if (isThinkingDisabled(requestOptions.thinking)) {
|
||||
clampOutputConfigEffort(resolvedModel, requestOptions.invocationKwargs?.output_config);
|
||||
}
|
||||
|
||||
const hasActiveThinking = requestOptions.thinking != null;
|
||||
const isThinkingModel =
|
||||
/claude-3[-.]7/.test(resolvedModel) || supportsAdaptiveThinking(resolvedModel);
|
||||
|
|
|
|||
|
|
@ -183,7 +183,15 @@ export function createAnthropicVertexClient(
|
|||
throw new Error('Google service account key is required for Vertex AI');
|
||||
}
|
||||
|
||||
// Priority: vertexOptions > env vars > service key project_id
|
||||
/**
|
||||
* Priority: vertexOptions > env vars > service key project_id.
|
||||
*
|
||||
* The `us-east5` fallback only serves Sonnet 4.6 and earlier — specific
|
||||
* regional endpoints 404 on newer models (Opus 4.7+, Opus 5, Sonnet 5,
|
||||
* Fable 5), which need `global` or a multi-region (`us`/`eu`) location.
|
||||
* Kept for backwards compatibility; deployments using modern models must
|
||||
* set the region explicitly.
|
||||
*/
|
||||
const region = vertexOptions?.region || process.env.ANTHROPIC_VERTEX_REGION || 'us-east5';
|
||||
const projectId =
|
||||
vertexOptions?.projectId || process.env.VERTEX_PROJECT_ID || serviceKey.project_id;
|
||||
|
|
|
|||
|
|
@ -239,6 +239,72 @@ describe('PDF Validation with fileConfig.endpoints.*.fileSizeLimit', () => {
|
|||
expect(result.error).toBeUndefined();
|
||||
});
|
||||
|
||||
it.each([
|
||||
'anthropic.claude-opus-5',
|
||||
'global.anthropic.claude-opus-5',
|
||||
'us.anthropic.claude-opus-5',
|
||||
'global.anthropic.claude-sonnet-5',
|
||||
'global.anthropic.claude-fable-5',
|
||||
])('should exempt undated Claude 4+ ID %s from the 4.5MB limit', async (model) => {
|
||||
/** These IDs end at the major version, so a pattern requiring a trailing
|
||||
* `-` after it silently dropped the exemption. */
|
||||
const pdfBuffer = createMockPdfBuffer(10);
|
||||
const result = await validatePdf(pdfBuffer, pdfBuffer.length, provider, undefined, model);
|
||||
|
||||
expect(result.isValid).toBe(true);
|
||||
expect(result.error).toBeUndefined();
|
||||
});
|
||||
|
||||
it.each([
|
||||
'global.anthropic.claude-opus-4-8',
|
||||
'global.anthropic.claude-opus-4-7',
|
||||
'global.anthropic.claude-sonnet-4-6',
|
||||
'global.anthropic.claude-opus-4-6-v1',
|
||||
])('should exempt global inference profile ID %s', async (model) => {
|
||||
const pdfBuffer = createMockPdfBuffer(10);
|
||||
const result = await validatePdf(pdfBuffer, pdfBuffer.length, provider, undefined, model);
|
||||
|
||||
expect(result.isValid).toBe(true);
|
||||
});
|
||||
|
||||
it.each(['claude-opus-5', 'claude-sonnet-5', 'claude-opus-4-8', 'claude-fable-5'])(
|
||||
'should exempt bare application inference profile ID %s',
|
||||
async (model) => {
|
||||
/** A LibreChat model ID mapping to an application inference profile has
|
||||
* no `anthropic.` segment. */
|
||||
const pdfBuffer = createMockPdfBuffer(10);
|
||||
const result = await validatePdf(pdfBuffer, pdfBuffer.length, provider, undefined, model);
|
||||
|
||||
expect(result.isValid).toBe(true);
|
||||
expect(result.error).toBeUndefined();
|
||||
},
|
||||
);
|
||||
|
||||
it.each(['claude-4-6-opus', 'claude-5-sonnet', 'anthropic.claude-4-6-opus'])(
|
||||
'should exempt version-first ID %s',
|
||||
async (model) => {
|
||||
const pdfBuffer = createMockPdfBuffer(10);
|
||||
const result = await validatePdf(pdfBuffer, pdfBuffer.length, provider, undefined, model);
|
||||
|
||||
expect(result.isValid).toBe(true);
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
'anthropic.claude-3-5-sonnet-20241022-v2:0',
|
||||
'anthropic.claude-3-opus-20240229-v1:0',
|
||||
'claude-3-5-sonnet',
|
||||
'claude-3-opus',
|
||||
'mistral.mistral-large-2402-v1:0',
|
||||
])('should NOT exempt pre-Claude-4 or non-Claude model %s', async (model) => {
|
||||
/** The relaxed prefix must not pull in Claude 3.x or other providers. */
|
||||
const pdfBuffer = createMockPdfBuffer(10);
|
||||
const result = await validatePdf(pdfBuffer, pdfBuffer.length, provider, undefined, model);
|
||||
|
||||
expect(result.isValid).toBe(false);
|
||||
expect(result.error).toContain('4.5MB');
|
||||
});
|
||||
|
||||
it('should still enforce 4.5MB for non-exempt models without config override', async () => {
|
||||
const pdfBuffer = createMockPdfBuffer(5);
|
||||
const model = 'anthropic.claude-3-5-sonnet-20241022-v2:0';
|
||||
|
|
|
|||
|
|
@ -130,11 +130,26 @@ async function validateAnthropicPdf(
|
|||
}
|
||||
|
||||
/**
|
||||
* Matches Bedrock Claude 4+ model identifiers, including cross-region inference profile IDs.
|
||||
* Pattern: [region.]anthropic.claude-{family}-{version≥4}-{date}-v{n}:{rev}
|
||||
* e.g. "anthropic.claude-sonnet-4-6" or "us.anthropic.claude-sonnet-4-6"
|
||||
* Matches Bedrock Claude 4+ model identifiers in every form they occur:
|
||||
* prefixed (`anthropic.claude-*`, `us.anthropic.claude-*`,
|
||||
* `global.anthropic.claude-*`), bare (`claude-*`, used when the LibreChat model
|
||||
* ID maps to an application inference profile), and either segment order
|
||||
* (`claude-opus-5`, `claude-4-6-opus`).
|
||||
*
|
||||
* Two forms were previously dropped, each defaulting the model back to the
|
||||
* 4.5 MB limit: requiring a `-` after the major version excluded undated IDs
|
||||
* like `claude-opus-5`, and requiring a literal `anthropic.` excluded bare
|
||||
* inference-profile IDs. Fable/Mythos are Claude 4+ generation and take the
|
||||
* same PDF exemption.
|
||||
*
|
||||
* Mirrors `BEDROCK_CLAUDE_4PLUS_THINKING` in `librechat-data-provider`, which
|
||||
* matches on the family token for the same reason. Only reached for the Bedrock
|
||||
* provider, so the loose prefix cannot leak into other endpoints.
|
||||
*/
|
||||
const BEDROCK_CLAUDE_4_PLUS_RE = /(?:^|\.)anthropic\.claude-(?:sonnet|opus|haiku)-[4-9]\d*-/;
|
||||
const CLAUDE_FAMILY = 'sonnet|opus|haiku|fable|mythos';
|
||||
const BEDROCK_CLAUDE_4_PLUS_RE = new RegExp(
|
||||
`(?:^|\\.)(?:anthropic\\.)?claude-(?:(?:${CLAUDE_FAMILY})-[4-9]\\d*|[4-9]\\d*(?:[-.]\\d+)?-(?:${CLAUDE_FAMILY}))(?:[-.]|$)`,
|
||||
);
|
||||
const isBedrockClaude4Plus = (model?: string): boolean =>
|
||||
model != null && BEDROCK_CLAUDE_4_PLUS_RE.test(model);
|
||||
|
||||
|
|
|
|||
|
|
@ -172,6 +172,7 @@ const anthropicModels = {
|
|||
'claude-opus-4-6': 1000000,
|
||||
'claude-opus-4-7': 1000000,
|
||||
'claude-opus-4-8': 1000000,
|
||||
'claude-opus-5': 1000000,
|
||||
'claude-fable-5': 1000000,
|
||||
'claude-mythos-5': 1000000,
|
||||
};
|
||||
|
|
@ -472,6 +473,7 @@ const anthropicMaxOutputs = {
|
|||
'claude-opus-4-6': 128000,
|
||||
'claude-opus-4-7': 128000,
|
||||
'claude-opus-4-8': 128000,
|
||||
'claude-opus-5': 128000,
|
||||
'claude-fable-5': 128000,
|
||||
'claude-mythos-5': 128000,
|
||||
'claude-3.5-sonnet': 8192,
|
||||
|
|
|
|||
|
|
@ -1,10 +1,17 @@
|
|||
import { ThinkingDisplay, isMythosClassModel, MYTHOS_CLASS_FAMILIES } from '../src/schemas';
|
||||
import {
|
||||
ThinkingDisplay,
|
||||
AnthropicEffort,
|
||||
isMythosClassModel,
|
||||
MYTHOS_CLASS_FAMILIES,
|
||||
} from '../src/schemas';
|
||||
import {
|
||||
BEDROCK_OUTPUT_128K_BETA,
|
||||
supportsAdaptiveThinking,
|
||||
omitsSamplingParameters,
|
||||
omitsThinkingByDefault,
|
||||
requiresExplicitThinkingDisabled,
|
||||
capsEffortWhenThinkingDisabled,
|
||||
clampEffortForDisabledThinking,
|
||||
resolveThinkingDisplay,
|
||||
bedrockOutputParser,
|
||||
bedrockInputParser,
|
||||
|
|
@ -400,16 +407,60 @@ describe('requiresExplicitThinkingDisabled', () => {
|
|||
expect(requiresExplicitThinkingDisabled('claude-sonnet-9')).toBe(true);
|
||||
});
|
||||
|
||||
test('returns false for pre-5 Sonnet, Opus, and Mythos-class models', () => {
|
||||
// Opus 4.7+ omit -> off; Fable/Mythos reject an explicit disabled config (400)
|
||||
test('returns true for Opus 5+ (omitted thinking runs adaptive by default)', () => {
|
||||
expect(requiresExplicitThinkingDisabled('claude-opus-5')).toBe(true);
|
||||
expect(requiresExplicitThinkingDisabled('claude-opus-5-20260701')).toBe(true);
|
||||
expect(requiresExplicitThinkingDisabled('anthropic.claude-opus-5')).toBe(true);
|
||||
expect(requiresExplicitThinkingDisabled('us.anthropic.claude-opus-5')).toBe(true);
|
||||
expect(requiresExplicitThinkingDisabled('claude-opus-9')).toBe(true);
|
||||
});
|
||||
|
||||
test('returns false for pre-5 Sonnet, pre-5 Opus, and Mythos-class models', () => {
|
||||
// Opus 4.7/4.8 omit -> off; Fable/Mythos reject an explicit disabled config (400)
|
||||
expect(requiresExplicitThinkingDisabled('claude-sonnet-4-6')).toBe(false);
|
||||
expect(requiresExplicitThinkingDisabled('claude-opus-4-8')).toBe(false);
|
||||
expect(requiresExplicitThinkingDisabled('claude-opus-4-7')).toBe(false);
|
||||
expect(requiresExplicitThinkingDisabled('claude-fable-5')).toBe(false);
|
||||
expect(requiresExplicitThinkingDisabled('claude-mythos-5')).toBe(false);
|
||||
expect(requiresExplicitThinkingDisabled('gpt-4o')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('capsEffortWhenThinkingDisabled', () => {
|
||||
test('returns true for Opus 5+', () => {
|
||||
expect(capsEffortWhenThinkingDisabled('claude-opus-5')).toBe(true);
|
||||
expect(capsEffortWhenThinkingDisabled('anthropic.claude-opus-5')).toBe(true);
|
||||
expect(capsEffortWhenThinkingDisabled('claude-opus-9')).toBe(true);
|
||||
});
|
||||
|
||||
test('returns false for models that accept every effort with thinking off', () => {
|
||||
// Live-verified: these all return 200 for thinking disabled + effort xhigh/max
|
||||
expect(capsEffortWhenThinkingDisabled('claude-opus-4-8')).toBe(false);
|
||||
expect(capsEffortWhenThinkingDisabled('claude-opus-4-7')).toBe(false);
|
||||
expect(capsEffortWhenThinkingDisabled('claude-sonnet-5')).toBe(false);
|
||||
expect(capsEffortWhenThinkingDisabled('claude-fable-5')).toBe(false);
|
||||
expect(capsEffortWhenThinkingDisabled('gpt-4o')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('clampEffortForDisabledThinking', () => {
|
||||
test('lowers xhigh/max to high on Opus 5', () => {
|
||||
expect(clampEffortForDisabledThinking('claude-opus-5', AnthropicEffort.xhigh)).toBe('high');
|
||||
expect(clampEffortForDisabledThinking('claude-opus-5', AnthropicEffort.max)).toBe('high');
|
||||
});
|
||||
|
||||
test('leaves accepted effort levels untouched on Opus 5', () => {
|
||||
expect(clampEffortForDisabledThinking('claude-opus-5', AnthropicEffort.high)).toBe('high');
|
||||
expect(clampEffortForDisabledThinking('claude-opus-5', AnthropicEffort.medium)).toBe('medium');
|
||||
expect(clampEffortForDisabledThinking('claude-opus-5', AnthropicEffort.low)).toBe('low');
|
||||
});
|
||||
|
||||
test('leaves effort untouched on models without the cap', () => {
|
||||
expect(clampEffortForDisabledThinking('claude-opus-4-8', AnthropicEffort.xhigh)).toBe('xhigh');
|
||||
expect(clampEffortForDisabledThinking('claude-sonnet-5', AnthropicEffort.max)).toBe('max');
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveThinkingDisplay', () => {
|
||||
test('returns "summarized" for Opus 4.7 when explicit is auto/null/undefined', () => {
|
||||
expect(resolveThinkingDisplay('claude-opus-4-7', ThinkingDisplay.auto)).toBe('summarized');
|
||||
|
|
@ -1228,6 +1279,66 @@ describe('bedrockInputParser', () => {
|
|||
expect(additionalFields.thinking).toEqual({ type: 'adaptive' });
|
||||
expect(additionalFields.output_config).toEqual({ effort: 'max' });
|
||||
});
|
||||
|
||||
test.each(['xhigh', 'max'])(
|
||||
'clamps %s effort to high for Opus 5 when thinking is disabled',
|
||||
(effort) => {
|
||||
const result = bedrockInputParser.parse({
|
||||
model: 'anthropic.claude-opus-5',
|
||||
thinking: false,
|
||||
effort,
|
||||
}) as Record<string, unknown>;
|
||||
const additionalFields = result.additionalModelRequestFields as Record<string, unknown>;
|
||||
expect(additionalFields.thinking).toEqual({ type: 'disabled' });
|
||||
expect(additionalFields.output_config).toEqual({ effort: 'high' });
|
||||
},
|
||||
);
|
||||
|
||||
test('clamps persisted output_config effort for Opus 5 when thinking is disabled', () => {
|
||||
const result = bedrockInputParser.parse({
|
||||
model: 'anthropic.claude-opus-5',
|
||||
thinking: false,
|
||||
additionalModelRequestFields: {
|
||||
output_config: { effort: 'xhigh' },
|
||||
},
|
||||
}) as Record<string, unknown>;
|
||||
const additionalFields = result.additionalModelRequestFields as Record<string, unknown>;
|
||||
expect(additionalFields.output_config).toEqual({ effort: 'high' });
|
||||
});
|
||||
|
||||
test('clamps persisted effort for Opus 5 when a disabled config round-trips from persistence', () => {
|
||||
const result = bedrockInputParser.parse({
|
||||
model: 'anthropic.claude-opus-5',
|
||||
additionalModelRequestFields: {
|
||||
thinking: { type: 'disabled' },
|
||||
output_config: { effort: 'max' },
|
||||
},
|
||||
}) as Record<string, unknown>;
|
||||
const additionalFields = result.additionalModelRequestFields as Record<string, unknown>;
|
||||
expect(additionalFields.thinking).toEqual({ type: 'disabled' });
|
||||
expect(additionalFields.output_config).toEqual({ effort: 'high' });
|
||||
});
|
||||
|
||||
test('keeps xhigh effort for Opus 5 while thinking is enabled', () => {
|
||||
const result = bedrockInputParser.parse({
|
||||
model: 'anthropic.claude-opus-5',
|
||||
thinking: true,
|
||||
effort: 'xhigh',
|
||||
}) as Record<string, unknown>;
|
||||
const additionalFields = result.additionalModelRequestFields as Record<string, unknown>;
|
||||
expect(additionalFields.thinking).toEqual({ type: 'adaptive', display: 'summarized' });
|
||||
expect(additionalFields.output_config).toEqual({ effort: 'xhigh' });
|
||||
});
|
||||
|
||||
test('does not clamp xhigh effort for Opus 4.8 when thinking is disabled', () => {
|
||||
const result = bedrockInputParser.parse({
|
||||
model: 'anthropic.claude-opus-4-8',
|
||||
thinking: false,
|
||||
effort: 'xhigh',
|
||||
}) as Record<string, unknown>;
|
||||
const additionalFields = result.additionalModelRequestFields as Record<string, unknown>;
|
||||
expect(additionalFields.output_config).toEqual({ effort: 'xhigh' });
|
||||
});
|
||||
});
|
||||
|
||||
describe('bedrockOutputParser with configureThinking', () => {
|
||||
|
|
|
|||
|
|
@ -179,17 +179,84 @@ export function omitsSamplingParameters(model: string): boolean {
|
|||
* Whether disabling thinking requires sending an explicit `{ type: 'disabled' }`
|
||||
* config rather than simply omitting the `thinking` field.
|
||||
*
|
||||
* Sonnet 5 treats an omitted `thinking` field as adaptive thinking ON by
|
||||
* default, so honoring a user who turns thinking off means sending the disabled
|
||||
* config explicitly. Opus 4.7+ run without thinking when the field is omitted,
|
||||
* and Fable/Mythos reject an explicit disabled config (400, thinking always
|
||||
* on), so both are excluded.
|
||||
* Sonnet 5 and Opus 5 treat an omitted `thinking` field as adaptive thinking ON
|
||||
* by default, so honoring a user who turns thinking off means sending the
|
||||
* disabled config explicitly. Opus 4.7/4.8 run without thinking when the field
|
||||
* is omitted, and Fable/Mythos reject an explicit disabled config (400,
|
||||
* thinking always on), so both are excluded.
|
||||
*
|
||||
* See https://platform.claude.com/docs/en/about-claude/models/migration-guide#migrating-to-claude-sonnet-5
|
||||
*/
|
||||
export function requiresExplicitThinkingDisabled(model: string): boolean {
|
||||
const sonnet = parseSonnetVersion(model);
|
||||
return sonnet != null && sonnet.major >= 5;
|
||||
if (sonnet != null && sonnet.major >= 5) {
|
||||
return true;
|
||||
}
|
||||
const opus = parseOpusVersion(model);
|
||||
return opus != null && opus.major >= 5;
|
||||
}
|
||||
|
||||
/** Effort levels Opus 5 rejects while thinking is explicitly disabled. */
|
||||
const EFFORTS_REJECTED_WHEN_THINKING_DISABLED = new Set<string>([
|
||||
s.AnthropicEffort.xhigh,
|
||||
s.AnthropicEffort.max,
|
||||
]);
|
||||
|
||||
/**
|
||||
* Whether the model caps `output_config.effort` while thinking is disabled.
|
||||
*
|
||||
* Opus 5 rejects `xhigh`/`max` in that combination with a 400: "output_config
|
||||
* .effort 'xhigh' is not supported when thinking is disabled on this model. Use
|
||||
* effort 'high' or below, or enable thinking." Opus 4.7/4.8, Sonnet 5, and
|
||||
* Sonnet 4.6 accept every effort level they otherwise support with thinking
|
||||
* off, so the cap is Opus 5+ only.
|
||||
*/
|
||||
export function capsEffortWhenThinkingDisabled(model: string): boolean {
|
||||
const opus = parseOpusVersion(model);
|
||||
return opus != null && opus.major >= 5;
|
||||
}
|
||||
|
||||
/**
|
||||
* Lowers an effort level the model would reject while thinking is disabled to
|
||||
* the highest accepted value (`high`, which is also the API default). Returns
|
||||
* the effort unchanged when the combination is valid.
|
||||
*/
|
||||
export function clampEffortForDisabledThinking(model: string, effort: string): string {
|
||||
if (
|
||||
capsEffortWhenThinkingDisabled(model) &&
|
||||
EFFORTS_REJECTED_WHEN_THINKING_DISABLED.has(effort)
|
||||
) {
|
||||
return s.AnthropicEffort.high;
|
||||
}
|
||||
return effort;
|
||||
}
|
||||
|
||||
/** An `output_config` container carrying a usable effort level. */
|
||||
function hasStringEffort(value: unknown): value is { effort: string } {
|
||||
if (typeof value !== 'object' || value === null || !('effort' in value)) {
|
||||
return false;
|
||||
}
|
||||
return typeof value.effort === 'string';
|
||||
}
|
||||
|
||||
/**
|
||||
* Clamps an `output_config.effort` in place when the model would reject it
|
||||
* while thinking is disabled. No-op when the container carries no string
|
||||
* effort, so callers can pass a possibly-absent config directly.
|
||||
*/
|
||||
export function clampOutputConfigEffort(model: string, outputConfig: unknown): void {
|
||||
if (!hasStringEffort(outputConfig)) {
|
||||
return;
|
||||
}
|
||||
outputConfig.effort = clampEffortForDisabledThinking(model, outputConfig.effort);
|
||||
}
|
||||
|
||||
/** Whether a resolved thinking config is an explicit `{ type: 'disabled' }`. */
|
||||
export function isThinkingDisabled(thinking: unknown): boolean {
|
||||
if (typeof thinking !== 'object' || thinking === null || !('type' in thinking)) {
|
||||
return false;
|
||||
}
|
||||
return thinking.type === 'disabled';
|
||||
}
|
||||
|
||||
/** Checks if a model has a 1M context window (Sonnet 4.6+, Opus 4.6+, Opus 5+, Fable/Mythos) */
|
||||
|
|
@ -458,6 +525,7 @@ export const bedrockInputParser = s.tConversationSchema
|
|||
const persistedAmrf = typedData.additionalModelRequestFields as
|
||||
| Record<string, unknown>
|
||||
| undefined;
|
||||
const thinkingDisabled = additionalFields.thinking === false;
|
||||
const effort = additionalFields.effort;
|
||||
if (typeof effort === 'string' && effort !== '') {
|
||||
additionalFields.output_config = { effort };
|
||||
|
|
@ -469,6 +537,18 @@ export const bedrockInputParser = s.tConversationSchema
|
|||
}
|
||||
delete additionalFields.effort;
|
||||
|
||||
/**
|
||||
* Opus 5 rejects `xhigh`/`max` effort while thinking is disabled, so
|
||||
* clamp both the effort derived above and any effort still carried in
|
||||
* persisted AMRF (agent resume sends `output_config` with no top-level
|
||||
* `effort`, so the branch above leaves it untouched).
|
||||
*/
|
||||
if (thinkingDisabled) {
|
||||
[additionalFields, persistedAmrf].forEach((target) =>
|
||||
clampOutputConfigEffort(typedData.model as string, target?.output_config),
|
||||
);
|
||||
}
|
||||
|
||||
if (additionalFields.thinking === false) {
|
||||
delete additionalFields.thinkingBudget;
|
||||
delete additionalFields.thinkingDisplay;
|
||||
|
|
|
|||
|
|
@ -1,13 +1,14 @@
|
|||
import type { TEndpointsConfig } from './types';
|
||||
import { EModelEndpoint, isDocumentSupportedProvider } from './schemas';
|
||||
import { getEndpointFileConfig, mergeFileConfig } from './file-config';
|
||||
import {
|
||||
allowedAddressesSchema,
|
||||
bedrockModels,
|
||||
configSchema,
|
||||
excludedKeys,
|
||||
resolveEndpointType,
|
||||
webSearchSchema,
|
||||
} from './config';
|
||||
import { EModelEndpoint, isDocumentSupportedProvider } from './schemas';
|
||||
import { getEndpointFileConfig, mergeFileConfig } from './file-config';
|
||||
|
||||
const endpointsConfig: TEndpointsConfig = {
|
||||
[EModelEndpoint.openAI]: { userProvide: false, order: 0 },
|
||||
|
|
@ -559,3 +560,43 @@ describe('webSearchSchema', () => {
|
|||
).toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('bedrockModels defaults', () => {
|
||||
/**
|
||||
* Bedrock rejects on-demand Converse invocation of Claude 4+ foundation-model
|
||||
* IDs ("Retry your request with the ID or ARN of an inference profile"), so
|
||||
* every Claude 4+ default must ship as a cross-region profile ID or the model
|
||||
* fails on first use.
|
||||
*/
|
||||
const claude4Plus =
|
||||
/claude-(?:[4-9](?:-\d+)?-(?:sonnet|opus|haiku)|(?:sonnet|opus|haiku|fable)-[4-9])/;
|
||||
|
||||
it('uses a cross-region inference profile for every Claude 4+ entry', () => {
|
||||
const bare = bedrockModels.filter(
|
||||
(model) => claude4Plus.test(model) && !/^(?:global|us)\./.test(model),
|
||||
);
|
||||
|
||||
expect(bare).toEqual([]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
'anthropic.claude-3-5-sonnet-20241022-v2:0',
|
||||
'anthropic.claude-3-5-sonnet-20240620-v1:0',
|
||||
'anthropic.claude-3-5-haiku-20241022-v1:0',
|
||||
])('does not offer retired model %s', (model) => {
|
||||
/** These reached end of life at AWS and return ResourceNotFoundException in
|
||||
* every prefix form, so selecting one is a hard error for the user. */
|
||||
expect(bedrockModels).not.toContain(model);
|
||||
});
|
||||
|
||||
it('offers no Claude 3.x model at all', () => {
|
||||
const claude3 = bedrockModels.filter((model) => /claude-3[-.]/.test(model));
|
||||
|
||||
expect(claude3).toEqual([]);
|
||||
});
|
||||
|
||||
it('keeps Opus 5 available as a global profile', () => {
|
||||
expect(bedrockModels).toContain('global.anthropic.claude-opus-5');
|
||||
expect(bedrockModels).not.toContain('anthropic.claude-opus-5');
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -2045,6 +2045,7 @@ const sharedOpenAIModels = [
|
|||
|
||||
const sharedAnthropicModels = [
|
||||
'claude-fable-5',
|
||||
'claude-opus-5',
|
||||
'claude-opus-4-8',
|
||||
'claude-opus-4-7',
|
||||
'claude-sonnet-5',
|
||||
|
|
@ -2069,19 +2070,25 @@ const sharedAnthropicModels = [
|
|||
'claude-3-5-sonnet-latest',
|
||||
];
|
||||
|
||||
/**
|
||||
* Claude 4+ models are not invocable on-demand by their bare foundation-model
|
||||
* ID on the Converse path — Bedrock rejects those with "Invocation of model ID
|
||||
* ... with on-demand throughput isn't supported. Retry your request with the ID
|
||||
* or ARN of an inference profile that contains this model." Default to the
|
||||
* `global.` cross-region profile (no regional pricing premium, widest
|
||||
* availability); Opus 4.1 has no global profile, so it uses `us.`.
|
||||
*/
|
||||
export const bedrockModels = [
|
||||
'anthropic.claude-fable-5',
|
||||
'anthropic.claude-opus-4-8',
|
||||
'anthropic.claude-opus-4-7',
|
||||
'anthropic.claude-sonnet-5',
|
||||
'anthropic.claude-sonnet-4-6',
|
||||
'anthropic.claude-opus-4-6-v1',
|
||||
'anthropic.claude-sonnet-4-5-20250929-v1:0',
|
||||
'anthropic.claude-haiku-4-5-20251001-v1:0',
|
||||
'anthropic.claude-opus-4-1-20250805-v1:0',
|
||||
'anthropic.claude-3-5-sonnet-20241022-v2:0',
|
||||
'anthropic.claude-3-5-sonnet-20240620-v1:0',
|
||||
'anthropic.claude-3-5-haiku-20241022-v1:0',
|
||||
'global.anthropic.claude-fable-5',
|
||||
'global.anthropic.claude-opus-5',
|
||||
'global.anthropic.claude-opus-4-8',
|
||||
'global.anthropic.claude-opus-4-7',
|
||||
'global.anthropic.claude-sonnet-5',
|
||||
'global.anthropic.claude-sonnet-4-6',
|
||||
'global.anthropic.claude-opus-4-6-v1',
|
||||
'global.anthropic.claude-sonnet-4-5-20250929-v1:0',
|
||||
'global.anthropic.claude-haiku-4-5-20251001-v1:0',
|
||||
'us.anthropic.claude-opus-4-1-20250805-v1:0',
|
||||
// 'cohere.command-text-v14', // no conversation history
|
||||
// 'cohere.command-light-text-v14', // no conversation history
|
||||
'cohere.command-r-v1:0',
|
||||
|
|
|
|||
82
packages/data-schemas/src/app/vertex.spec.ts
Normal file
82
packages/data-schemas/src/app/vertex.spec.ts
Normal file
|
|
@ -0,0 +1,82 @@
|
|||
import { vertexAISchema } from 'librechat-data-provider';
|
||||
import { defaultVertexModels, validateVertexConfig } from './vertex';
|
||||
|
||||
describe('defaultVertexModels', () => {
|
||||
/**
|
||||
* `loadEndpoints` swaps the shared Anthropic model list for the Vertex model
|
||||
* names, which fall back to these defaults when `vertex.models` is omitted.
|
||||
* A model missing here is invisible to every Vertex deployment that has not
|
||||
* enumerated models by hand.
|
||||
*/
|
||||
it('includes the modern Opus family served by Vertex', () => {
|
||||
expect(defaultVertexModels).toEqual(
|
||||
expect.arrayContaining([
|
||||
'claude-opus-5',
|
||||
'claude-opus-4-8',
|
||||
'claude-opus-4-7',
|
||||
'claude-opus-4-6',
|
||||
]),
|
||||
);
|
||||
});
|
||||
|
||||
it('keeps the modern Sonnet models', () => {
|
||||
expect(defaultVertexModels).toEqual(
|
||||
expect.arrayContaining(['claude-sonnet-5', 'claude-sonnet-4-6']),
|
||||
);
|
||||
});
|
||||
|
||||
it('uses bare IDs for the 4.6+ generation and @-dated IDs for older models', () => {
|
||||
const modern = defaultVertexModels.filter((model) =>
|
||||
/claude-(?:opus|sonnet)-(?:4-[6-9]|[5-9])$/.test(model),
|
||||
);
|
||||
expect(modern.length).toBeGreaterThan(0);
|
||||
modern.forEach((model) => expect(model).not.toContain('@'));
|
||||
|
||||
expect(defaultVertexModels).toContain('claude-3-opus@20240229');
|
||||
});
|
||||
|
||||
it('has no duplicate entries', () => {
|
||||
expect(new Set(defaultVertexModels).size).toBe(defaultVertexModels.length);
|
||||
});
|
||||
});
|
||||
|
||||
describe('validateVertexConfig region gating', () => {
|
||||
/**
|
||||
* Specific regional endpoints serve Sonnet 4.6 and earlier only. Publishing
|
||||
* the modern defaults there would advertise models that 404 on first use.
|
||||
*/
|
||||
const modern = ['claude-opus-5', 'claude-opus-4-8', 'claude-opus-4-7', 'claude-sonnet-5'];
|
||||
const legacy = ['claude-sonnet-4-6', 'claude-3-7-sonnet-20250219', 'claude-3-opus@20240229'];
|
||||
|
||||
it('drops multi-region-only defaults on a specific regional endpoint', () => {
|
||||
const config = validateVertexConfig({ region: 'us-east5' });
|
||||
|
||||
modern.forEach((model) => expect(config?.modelNames).not.toContain(model));
|
||||
legacy.forEach((model) => expect(config?.modelNames).toContain(model));
|
||||
});
|
||||
|
||||
it('drops them when the region comes from the schema default', () => {
|
||||
/** `region` is schema-defaulted, so an operator who omits it still lands on
|
||||
* a specific regional endpoint that cannot serve the modern models. */
|
||||
const config = validateVertexConfig(vertexAISchema.parse({}));
|
||||
|
||||
expect(config?.region).toBe('us-east5');
|
||||
modern.forEach((model) => expect(config?.modelNames).not.toContain(model));
|
||||
});
|
||||
|
||||
it.each(['global', 'us', 'eu', 'GLOBAL'])('keeps every default on %s', (region) => {
|
||||
const config = validateVertexConfig({ region });
|
||||
|
||||
modern.forEach((model) => expect(config?.modelNames).toContain(model));
|
||||
legacy.forEach((model) => expect(config?.modelNames).toContain(model));
|
||||
});
|
||||
|
||||
it('never prunes an explicit model list, even on a regional endpoint', () => {
|
||||
const config = validateVertexConfig({
|
||||
region: 'us-east5',
|
||||
models: ['claude-opus-5', 'claude-sonnet-4-6'],
|
||||
});
|
||||
|
||||
expect(config?.modelNames).toEqual(['claude-opus-5', 'claude-sonnet-4-6']);
|
||||
});
|
||||
});
|
||||
|
|
@ -18,6 +18,10 @@ import logger from '~/config/winston';
|
|||
* These are the standard Anthropic model names as served by Vertex AI
|
||||
*/
|
||||
export const defaultVertexModels: string[] = [
|
||||
'claude-opus-5',
|
||||
'claude-opus-4-8',
|
||||
'claude-opus-4-7',
|
||||
'claude-opus-4-6',
|
||||
'claude-sonnet-5',
|
||||
'claude-sonnet-4-6',
|
||||
'claude-3-7-sonnet-20250219',
|
||||
|
|
@ -28,23 +32,49 @@ export const defaultVertexModels: string[] = [
|
|||
'claude-3-haiku@20240307',
|
||||
];
|
||||
|
||||
/** Locations that serve every model: `global` plus the `us`/`eu` multi-region endpoints. */
|
||||
const MULTI_REGION_LOCATIONS = new Set(['global', 'us', 'eu']);
|
||||
|
||||
/**
|
||||
* Models Vertex serves only from `global` or a multi-region location. Specific
|
||||
* regional endpoints (us-east5, europe-west1, ...) carry Sonnet 4.6 and earlier
|
||||
* and return 404 for anything newer.
|
||||
*/
|
||||
const REQUIRES_MULTI_REGION =
|
||||
/^claude-(?:opus-(?:4-[7-9]|[5-9])|sonnet-[5-9]|fable-[5-9]|mythos-[5-9])/;
|
||||
|
||||
/**
|
||||
* Whether the configured location can serve models that require `global` or a
|
||||
* multi-region endpoint.
|
||||
*/
|
||||
function servesModernModels(region: string): boolean {
|
||||
return MULTI_REGION_LOCATIONS.has(region.trim().toLowerCase());
|
||||
}
|
||||
|
||||
/**
|
||||
* Processes models configuration and creates deployment name mapping
|
||||
* Similar to Azure's model mapping logic
|
||||
* @param models - The models configuration (can be array or object)
|
||||
* @param defaultDeploymentName - Optional default deployment name
|
||||
* @param region - Configured Vertex location, used to filter the built-in defaults
|
||||
* @returns Object containing modelNames array and modelDeploymentMap
|
||||
*/
|
||||
function processVertexModels(
|
||||
models: string[] | Record<string, TVertexModelConfig> | undefined,
|
||||
defaultDeploymentName?: string,
|
||||
region?: string,
|
||||
): { modelNames: string[]; modelDeploymentMap: TVertexModelMap } {
|
||||
const modelNames: string[] = [];
|
||||
const modelDeploymentMap: TVertexModelMap = {};
|
||||
|
||||
if (!models) {
|
||||
// No models specified, use defaults
|
||||
/** Only filter the built-in defaults — an explicit list is the operator's
|
||||
* choice and is never silently pruned. */
|
||||
const canServeModern = region == null || servesModernModels(region);
|
||||
for (const model of defaultVertexModels) {
|
||||
if (!canServeModern && REQUIRES_MULTI_REGION.test(model)) {
|
||||
continue;
|
||||
}
|
||||
modelNames.push(model);
|
||||
modelDeploymentMap[model] = model; // Default: model name = deployment name
|
||||
}
|
||||
|
|
@ -132,6 +162,7 @@ export function validateVertexConfig(
|
|||
const { modelNames, modelDeploymentMap } = processVertexModels(
|
||||
vertexConfig.models,
|
||||
defaultDeploymentName,
|
||||
region,
|
||||
);
|
||||
|
||||
// Note: projectId is optional - if not provided, it will be auto-detected from the service key file
|
||||
|
|
|
|||
|
|
@ -2443,6 +2443,75 @@ describe('Claude Model Tests', () => {
|
|||
);
|
||||
});
|
||||
|
||||
it('should return correct prompt and completion rates for Claude Opus 5', () => {
|
||||
expect(getMultiplier({ model: 'claude-opus-5', tokenType: 'prompt' })).toBe(
|
||||
tokenValues['claude-opus-5'].prompt,
|
||||
);
|
||||
expect(getMultiplier({ model: 'claude-opus-5', tokenType: 'completion' })).toBe(
|
||||
tokenValues['claude-opus-5'].completion,
|
||||
);
|
||||
});
|
||||
|
||||
it('should handle Claude Opus 5 model name variations', () => {
|
||||
const modelVariations = [
|
||||
'claude-opus-5',
|
||||
'claude-opus-5-20260701',
|
||||
'claude-opus-5-latest',
|
||||
'anthropic/claude-opus-5',
|
||||
'claude-opus-5/anthropic',
|
||||
'claude-opus-5-preview',
|
||||
];
|
||||
|
||||
modelVariations.forEach((model) => {
|
||||
const valueKey = getValueKey(model);
|
||||
expect(valueKey).toBe('claude-opus-5');
|
||||
expect(getMultiplier({ model, tokenType: 'prompt' })).toBe(
|
||||
tokenValues['claude-opus-5'].prompt,
|
||||
);
|
||||
expect(getMultiplier({ model, tokenType: 'completion' })).toBe(
|
||||
tokenValues['claude-opus-5'].completion,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
it('should not confuse Claude Opus 5 with Claude Opus 4.5', () => {
|
||||
expect(getValueKey('claude-opus-5')).toBe('claude-opus-5');
|
||||
expect(getValueKey('claude-opus-4-5')).toBe('claude-opus-4-5');
|
||||
});
|
||||
|
||||
it('should return correct cache rates for Claude Opus 5', () => {
|
||||
expect(getCacheMultiplier({ model: 'claude-opus-5', cacheType: 'write' })).toBe(
|
||||
cacheTokenValues['claude-opus-5'].write,
|
||||
);
|
||||
expect(getCacheMultiplier({ model: 'claude-opus-5', cacheType: 'read' })).toBe(
|
||||
cacheTokenValues['claude-opus-5'].read,
|
||||
);
|
||||
});
|
||||
|
||||
it('should price Bedrock cross-region inference profile IDs like their base model', () => {
|
||||
const profileToBase: Array<[string, string]> = [
|
||||
['global.anthropic.claude-opus-5', 'claude-opus-5'],
|
||||
['us.anthropic.claude-opus-5', 'claude-opus-5'],
|
||||
['global.anthropic.claude-opus-4-8', 'claude-opus-4-8'],
|
||||
['global.anthropic.claude-sonnet-5', 'claude-sonnet-5'],
|
||||
['global.anthropic.claude-sonnet-4-6', 'claude-sonnet-4-6'],
|
||||
['global.anthropic.claude-fable-5', 'claude-fable-5'],
|
||||
];
|
||||
|
||||
profileToBase.forEach(([profileId, base]) => {
|
||||
expect(getValueKey(profileId)).toBe(base);
|
||||
expect(getMultiplier({ model: profileId, tokenType: 'prompt' })).toBe(
|
||||
tokenValues[base].prompt,
|
||||
);
|
||||
expect(getMultiplier({ model: profileId, tokenType: 'completion' })).toBe(
|
||||
tokenValues[base].completion,
|
||||
);
|
||||
expect(getCacheMultiplier({ model: profileId, cacheType: 'write' })).toBe(
|
||||
cacheTokenValues[base].write,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
it('should return correct prompt and completion rates for Claude Fable 5', () => {
|
||||
expect(getMultiplier({ model: 'claude-fable-5', tokenType: 'prompt' })).toBe(
|
||||
tokenValues['claude-fable-5'].prompt,
|
||||
|
|
@ -2602,6 +2671,7 @@ describe('Premium Token Pricing', () => {
|
|||
'claude-opus-4-6',
|
||||
'claude-opus-4-7',
|
||||
'claude-opus-4-8',
|
||||
'claude-opus-5',
|
||||
'claude-fable-5',
|
||||
'claude-mythos-5',
|
||||
'claude-sonnet-4-6',
|
||||
|
|
|
|||
|
|
@ -170,6 +170,7 @@ export const tokenValues: Record<string, { prompt: number; completion: number }>
|
|||
'claude-opus-4-6': { prompt: 5, completion: 25 },
|
||||
'claude-opus-4-7': { prompt: 5, completion: 25 },
|
||||
'claude-opus-4-8': { prompt: 5, completion: 25 },
|
||||
'claude-opus-5': { prompt: 5, completion: 25 },
|
||||
'claude-fable-5': { prompt: 10, completion: 50 },
|
||||
'claude-mythos-5': { prompt: 10, completion: 50 },
|
||||
'claude-sonnet-4': { prompt: 3, completion: 15 },
|
||||
|
|
@ -319,6 +320,7 @@ export const cacheTokenValues: Record<string, { write: number; read: number }> =
|
|||
'claude-opus-4-6': { write: 6.25, read: 0.5 },
|
||||
'claude-opus-4-7': { write: 6.25, read: 0.5 },
|
||||
'claude-opus-4-8': { write: 6.25, read: 0.5 },
|
||||
'claude-opus-5': { write: 6.25, read: 0.5 },
|
||||
'claude-fable-5': { write: 12.5, read: 1 },
|
||||
'claude-mythos-5': { write: 12.5, read: 1 },
|
||||
'gpt-4o': { write: 2.5, read: 1.25 },
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue