mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-10-25 03:32:12 +00:00
🤖 feat: Add Claude Sonnet 5 Support (#14042)
* ✨ feat: Add Claude Sonnet 5 Support Wire up the claude-sonnet-5 model across token, pricing, and model-list config: - Context window (1M) and max output (128K) in @librechat/api token maps - Standard pricing ($3/$15 per MTok) and cache rates in data-schemas tx - 128K output-token carve-out in anthropicSettings (the family-wide 64K rule capped Sonnet 5 below its real limit); Bedrock/Vertex thinking and 1M-context detection already cover sonnet major >= 5 generically - Add to shared Anthropic, Bedrock, and Vertex default model lists, plus the .env.example examples - Tests for context/output/pricing/matching across the affected packages * ✅ test: Align Sonnet 5+ maxOutputTokens defaults with 128K spec getLLMConfig defaults flow from anthropicSettings.maxOutputTokens.reset(), which now returns 128K for Sonnet 5+. Update the future-proofing assertions in llm.spec.ts (Sonnet 5.x and 6-9.x) that still expected the old family-wide 64K cap. Haiku stays 64K; Opus stays 128K. * 🎚️ fix: Gate Sonnet 5 capability behaviors (sampling, thinking) Adding claude-sonnet-5 to the default list exposed it without the Anthropic capability gates, all confirmed against the live API: - omitsSamplingParameters: Sonnet 5 returns 400 on non-default temperature/ top_p/top_k ('deprecated for this model'); now dropped so selecting the model with saved sampling settings no longer fails. - requiresExplicitThinkingDisabled: omitting 'thinking' runs adaptive ON by default on Sonnet 5, so disabling thinking now sends { type: 'disabled' } (verified: 200, no thinking block) instead of omitting the field. - omitsThinkingByDefault: thinking.display defaults to omitted (empty thinking blocks); the display resolver now returns 'summarized' for Sonnet 5+ so the Thoughts UI keeps working (verified: 757-char summary returned). Gates apply to both the direct Anthropic and Bedrock paths. Tests added in bedrock.spec and llm.spec. * 🩹 fix: Sonnet 5 Bedrock availability + thinking-off persistence Round-2 Codex review (all verified against the live API / Anthropic docs): - Sonnet 5 is NOT available on the legacy Bedrock InvokeModel/Converse surface (Anthropic docs: 'use Claude in Amazon Bedrock or Claude Platform on AWS'), which is what LibreChat's ChatBedrockConverse uses. Removed it from the default Bedrock model lists (config + .env.example). Opus 4.8/4.7/Fable 5 stay — those ARE reachable via InvokeModel. Sonnet 5 remains on the direct Anthropic API and Vertex, where it works. - Reverted the Bedrock-side explicit-disabled thinking handling added last round: with Sonnet 5 off Bedrock, no Bedrock model needs { type: 'disabled' }, so that path (and its round-trip concern) no longer applies. - Direct Anthropic path: a persisted { type: 'disabled' } thinking object now normalizes to a boolean flag in getLLMConfig, so a user's Sonnet 5 'thinking off' setting stays off across the model_parameters round trip instead of flipping back to adaptive (a truthy object skipped the disabled branch). * ↩️ fix: Restore Sonnet 5 on Bedrock (Converse) — verified live Reverses the round-2 removal: Sonnet 5 IS available on AWS Bedrock. Tested live via the Converse API: - global.anthropic.claude-sonnet-5 returns a normal response - bare anthropic.claude-sonnet-5 needs an inference profile — but that's identical to the already-shipping Opus 4.8 / Fable 5 / Sonnet 4.6 entries, which all fail bare on-demand the same way - temperature=0.5 -> 400 'deprecated for this model'; thinking {type:disabled} suppresses reasoning — same as the direct API The 'legacy' Bedrock docs page that claimed Sonnet 5 wasn't on the surface is stale. Restored: - anthropic.claude-sonnet-5 in bedrockModels + .env.example - the Bedrock explicit-disabled thinking handling (requiresExplicitThinkingDisabled -> { type: 'disabled' }) - the Finding 4 round-trip fix in bedrockInputSchema (coerce a persisted disabled AMRF.thinking to thinking=false instead of !!thinking -> true), with an end-to-end schema->parser test proving 'thinking off' stays sticky. Direct-path round-trip fix (getLLMConfig thinkingFlag) is unchanged. * 💵 fix: Sonnet 5 intro pricing + sticky disabled thinking on Bedrock reload Round-4 Codex review (both verified): - Pricing: Anthropic lists Sonnet 5 at introductory $2/$10 per MTok (cache $2.50/$0.20) through 2026-08-31, reverting to $3/$15 ($3.75/$0.30) on Sep 1 (confirmed on platform.claude.com/pricing). The static tx multiplier table is used for real balance transactions, so the post-intro rates were overcharging ~50% during the launch window. Switched to the intro rates with a revert comment on both the token and cache entries. - Bedrock disabled-thinking persistence: initializeBedrock feeds persisted model_parameters straight through bedrockInputParser (NOT bedrockInputSchema), where additionalModelRequestFields is a known key — so a prior thinking:{type:'disabled'} was ignored and rebuilt as adaptive on reload. bedrockInputParser now surfaces a persisted disabled AMRF.thinking as thinking=false so it re-emits {type:'disabled'}. Verified end-to-end against the real initializeBedrock call path.
This commit is contained in:
Danny Avila
GitHub
parent
6523a5add6
commit
9f8b6d92c0
14 changed files with 377 additions and 21 deletions
|
|
@ -279,7 +279,7 @@ PROXY=
|
|||
#============#
|
||||
|
||||
ANTHROPIC_API_KEY=user_provided
|
||||
# ANTHROPIC_MODELS=claude-fable-5,claude-opus-4-8,claude-opus-4-7,claude-sonnet-4-6,claude-opus-4-6,claude-opus-4-20250514,claude-3-7-sonnet-20250219,claude-3-5-sonnet-20241022,claude-3-5-haiku-20241022,claude-3-opus-20240229,claude-3-sonnet-20240229,claude-3-haiku-20240307
|
||||
# ANTHROPIC_MODELS=claude-fable-5,claude-opus-4-8,claude-opus-4-7,claude-sonnet-5,claude-sonnet-4-6,claude-opus-4-6,claude-opus-4-20250514,claude-3-7-sonnet-20250219,claude-3-5-sonnet-20241022,claude-3-5-haiku-20241022,claude-3-opus-20240229,claude-3-sonnet-20240229,claude-3-haiku-20240307
|
||||
# ANTHROPIC_REVERSE_PROXY=
|
||||
|
||||
# Set to true to use Anthropic models through Google Vertex AI instead of direct API
|
||||
|
|
@ -345,8 +345,8 @@ ANTHROPIC_API_KEY=user_provided
|
|||
# BEDROCK_AWS_BEARER_TOKEN=yourBedrockApiKey
|
||||
|
||||
# Note: This example list is not meant to be exhaustive. If omitted, all known, supported model IDs will be included for you.
|
||||
# BEDROCK_AWS_MODELS=anthropic.claude-fable-5,anthropic.claude-opus-4-8,anthropic.claude-opus-4-7,anthropic.claude-sonnet-4-6,anthropic.claude-opus-4-6-v1,anthropic.claude-3-5-sonnet-20240620-v1:0,meta.llama3-1-8b-instruct-v1:0
|
||||
# Cross-region inference model IDs: us.anthropic.claude-fable-5,us.anthropic.claude-opus-4-8,us.anthropic.claude-opus-4-7,us.anthropic.claude-sonnet-4-6,us.anthropic.claude-opus-4-6-v1,global.anthropic.claude-opus-4-6-v1
|
||||
# BEDROCK_AWS_MODELS=anthropic.claude-fable-5,anthropic.claude-opus-4-8,anthropic.claude-opus-4-7,anthropic.claude-sonnet-5,anthropic.claude-sonnet-4-6,anthropic.claude-opus-4-6-v1,anthropic.claude-3-5-sonnet-20240620-v1:0,meta.llama3-1-8b-instruct-v1:0
|
||||
# Cross-region inference model IDs: us.anthropic.claude-fable-5,us.anthropic.claude-opus-4-8,us.anthropic.claude-opus-4-7,us.anthropic.claude-sonnet-5,us.anthropic.claude-sonnet-4-6,us.anthropic.claude-opus-4-6-v1,global.anthropic.claude-opus-4-6-v1
|
||||
|
||||
# See all Bedrock model IDs here: https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html#model-ids-arns
|
||||
|
||||
|
|
|
|||
|
|
@ -1635,6 +1635,59 @@ describe('Claude Model Tests', () => {
|
|||
expect(matchModelName(model, EModelEndpoint.anthropic)).toBe('claude-sonnet-4-6');
|
||||
});
|
||||
});
|
||||
|
||||
it('should return correct context length for Claude Sonnet 5 (1M)', () => {
|
||||
expect(getModelMaxTokens('claude-sonnet-5', EModelEndpoint.anthropic)).toBe(
|
||||
maxTokensMap[EModelEndpoint.anthropic]['claude-sonnet-5'],
|
||||
);
|
||||
expect(getModelMaxTokens('claude-sonnet-5')).toBe(
|
||||
maxTokensMap[EModelEndpoint.anthropic]['claude-sonnet-5'],
|
||||
);
|
||||
});
|
||||
|
||||
it('should return correct max output tokens for Claude Sonnet 5 (128K)', () => {
|
||||
const { getModelMaxOutputTokens } = require('@librechat/api');
|
||||
expect(getModelMaxOutputTokens('claude-sonnet-5', EModelEndpoint.anthropic)).toBe(
|
||||
maxOutputTokensMap[EModelEndpoint.anthropic]['claude-sonnet-5'],
|
||||
);
|
||||
expect(getModelMaxOutputTokens('claude-sonnet-5', EModelEndpoint.anthropic)).toBeGreaterThan(
|
||||
getModelMaxOutputTokens('claude-sonnet-4-6', EModelEndpoint.anthropic),
|
||||
);
|
||||
});
|
||||
|
||||
it('should not match Claude Sonnet 5 against Claude Sonnet 4 keys', () => {
|
||||
const modelVariations = [
|
||||
'claude-sonnet-5',
|
||||
'claude-sonnet-5-20260101',
|
||||
'claude-sonnet-5-latest',
|
||||
'anthropic/claude-sonnet-5',
|
||||
'claude-sonnet-5/anthropic',
|
||||
'anthropic.claude-sonnet-5',
|
||||
];
|
||||
|
||||
modelVariations.forEach((model) => {
|
||||
const modelKey = findMatchingPattern(model, maxTokensMap[EModelEndpoint.anthropic]);
|
||||
expect(modelKey).toBe('claude-sonnet-5');
|
||||
expect(getModelMaxTokens(model, EModelEndpoint.anthropic)).toBe(
|
||||
maxTokensMap[EModelEndpoint.anthropic]['claude-sonnet-5'],
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
it('should match model names correctly for Claude Sonnet 5', () => {
|
||||
const modelVariations = [
|
||||
'claude-sonnet-5',
|
||||
'claude-sonnet-5-20260101',
|
||||
'claude-sonnet-5-latest',
|
||||
'anthropic/claude-sonnet-5',
|
||||
'claude-sonnet-5/anthropic',
|
||||
'claude-sonnet-5-preview',
|
||||
];
|
||||
|
||||
modelVariations.forEach((model) => {
|
||||
expect(matchModelName(model, EModelEndpoint.anthropic)).toBe('claude-sonnet-5');
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('Moonshot/Kimi Model Tests', () => {
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ import {
|
|||
isMythosClassModel,
|
||||
resolveThinkingDisplay,
|
||||
supportsAdaptiveThinking,
|
||||
requiresExplicitThinkingDisabled,
|
||||
} from 'librechat-data-provider';
|
||||
import { matchModelName } from '~/utils/tokens';
|
||||
|
||||
|
|
@ -99,6 +100,16 @@ function configureReasoning(
|
|||
const currentMaxTokens = updatedOptions.max_tokens ?? updatedOptions.maxTokens;
|
||||
const modelName = updatedOptions.model ?? '';
|
||||
|
||||
/**
|
||||
* Sonnet 5 runs adaptive thinking by default when the `thinking` field is
|
||||
* omitted, so honoring a user who turns thinking off requires sending an
|
||||
* explicit disabled config rather than leaving the field unset.
|
||||
*/
|
||||
if (!extendedOptions.thinking && modelName && requiresExplicitThinkingDisabled(modelName)) {
|
||||
updatedOptions.thinking = { type: 'disabled' } as AnthropicClientOptions['thinking'];
|
||||
return updatedOptions;
|
||||
}
|
||||
|
||||
if (extendedOptions.thinking && modelName && supportsAdaptiveThinking(modelName)) {
|
||||
/**
|
||||
* For Opus 4.7+, Anthropic omits thinking content from responses by
|
||||
|
|
|
|||
|
|
@ -1215,6 +1215,64 @@ describe('getLLMConfig', () => {
|
|||
expect(result.llmConfig).not.toHaveProperty('topK');
|
||||
});
|
||||
|
||||
it('should request summarized thinking display for Sonnet 5 (opt back in)', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: { model: 'claude-sonnet-5', thinking: true },
|
||||
});
|
||||
|
||||
const thinking = result.llmConfig.thinking as unknown as {
|
||||
type: string;
|
||||
display?: string;
|
||||
};
|
||||
expect(thinking.type).toBe('adaptive');
|
||||
expect(thinking.display).toBe('summarized');
|
||||
});
|
||||
|
||||
it('should send explicit disabled thinking for Sonnet 5 when thinking is off', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: { model: 'claude-sonnet-5', thinking: false },
|
||||
});
|
||||
|
||||
expect((result.llmConfig.thinking as unknown as { type: string }).type).toBe('disabled');
|
||||
});
|
||||
|
||||
it('should keep Sonnet 5 thinking off when a disabled config round-trips from persistence', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: {
|
||||
model: 'claude-sonnet-5',
|
||||
// Persisted model_parameters round-trip the prior disabled object,
|
||||
// not a boolean — it must stay disabled, not flip back to adaptive.
|
||||
thinking: { type: 'disabled' } as unknown as boolean,
|
||||
},
|
||||
});
|
||||
|
||||
expect((result.llmConfig.thinking as unknown as { type: string }).type).toBe('disabled');
|
||||
});
|
||||
|
||||
it('should omit sampling parameters for Sonnet 5', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: {
|
||||
model: 'claude-sonnet-5',
|
||||
thinking: true,
|
||||
temperature: 0.7,
|
||||
topP: 0.9,
|
||||
topK: 40,
|
||||
},
|
||||
});
|
||||
|
||||
expect(result.llmConfig).not.toHaveProperty('temperature');
|
||||
expect(result.llmConfig).not.toHaveProperty('topP');
|
||||
expect(result.llmConfig).not.toHaveProperty('topK');
|
||||
});
|
||||
|
||||
it('should NOT send explicit disabled thinking for pre-5 Sonnet (omission is off)', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: { model: 'claude-sonnet-4-6', thinking: false },
|
||||
});
|
||||
|
||||
expect(result.llmConfig.thinking).toBeUndefined();
|
||||
});
|
||||
|
||||
it('should NOT set thinking.display for pre-Opus-4.7 adaptive models', () => {
|
||||
const pre47Models = ['claude-opus-4-6', 'claude-sonnet-4-6'];
|
||||
|
||||
|
|
@ -1408,7 +1466,7 @@ describe('getLLMConfig', () => {
|
|||
});
|
||||
});
|
||||
|
||||
it('should future-proof Claude 5.x Sonnet models with 64K default', () => {
|
||||
it('should default Claude 5.x Sonnet models to 128K (matches Anthropic spec)', () => {
|
||||
const testCases = [
|
||||
'claude-sonnet-5',
|
||||
'claude-sonnet-5-0',
|
||||
|
|
@ -1420,7 +1478,7 @@ describe('getLLMConfig', () => {
|
|||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: { model },
|
||||
});
|
||||
expect(result.llmConfig.maxTokens).toBe(64000);
|
||||
expect(result.llmConfig.maxTokens).toBe(128000);
|
||||
});
|
||||
});
|
||||
|
||||
|
|
@ -1458,20 +1516,20 @@ describe('getLLMConfig', () => {
|
|||
|
||||
it('should future-proof Claude 6-9.x models with correct defaults', () => {
|
||||
const testCases = [
|
||||
// Claude 6.x - Sonnet/Haiku get 64K, Opus gets 128K
|
||||
{ model: 'claude-sonnet-6', expected: 64000 },
|
||||
// Claude 6.x - Sonnet/Opus get 128K, Haiku gets 64K
|
||||
{ model: 'claude-sonnet-6', expected: 128000 },
|
||||
{ model: 'claude-haiku-6-0', expected: 64000 },
|
||||
{ model: 'claude-opus-6-1', expected: 128000 },
|
||||
// Claude 7.x
|
||||
{ model: 'claude-sonnet-7-20270101', expected: 64000 },
|
||||
{ model: 'claude-sonnet-7-20270101', expected: 128000 },
|
||||
{ model: 'claude-haiku-7.5', expected: 64000 },
|
||||
{ model: 'claude-opus-7', expected: 128000 },
|
||||
// Claude 8.x
|
||||
{ model: 'claude-sonnet-8', expected: 64000 },
|
||||
{ model: 'claude-sonnet-8', expected: 128000 },
|
||||
{ model: 'claude-haiku-8-2', expected: 64000 },
|
||||
{ model: 'claude-opus-8-latest', expected: 128000 },
|
||||
// Claude 9.x
|
||||
{ model: 'claude-sonnet-9', expected: 64000 },
|
||||
{ model: 'claude-sonnet-9', expected: 128000 },
|
||||
{ model: 'claude-haiku-9', expected: 64000 },
|
||||
{ model: 'claude-opus-9', expected: 128000 },
|
||||
];
|
||||
|
|
|
|||
|
|
@ -145,8 +145,19 @@ function getLLMConfig(
|
|||
? ((persistedThinking as { display: string }).display as ThinkingDisplay | string)
|
||||
: undefined;
|
||||
|
||||
/**
|
||||
* `thinking` may round-trip as the full Anthropic object rather than a
|
||||
* boolean. Normalize to a flag so a persisted `{ type: 'disabled' }` (e.g. a
|
||||
* Sonnet 5 "thinking off" config stored back into `model_parameters`) is
|
||||
* treated as off — a truthy object would otherwise flip thinking back on.
|
||||
*/
|
||||
const thinkingFlag =
|
||||
typeof persistedThinking === 'object' && persistedThinking != null
|
||||
? (persistedThinking as { type?: string }).type !== 'disabled'
|
||||
: (persistedThinking ?? anthropicSettings.thinking.default);
|
||||
|
||||
const systemOptions = {
|
||||
thinking: options.modelOptions?.thinking ?? anthropicSettings.thinking.default,
|
||||
thinking: thinkingFlag,
|
||||
promptCache: options.modelOptions?.promptCache ?? anthropicSettings.promptCache.default,
|
||||
promptCacheTtl:
|
||||
options.modelOptions?.promptCacheTtl ?? anthropicSettings.promptCacheTtl.default,
|
||||
|
|
|
|||
|
|
@ -156,6 +156,7 @@ const anthropicModels = {
|
|||
'claude-sonnet-4': 200000,
|
||||
'claude-sonnet-4-5': 200000,
|
||||
'claude-sonnet-4-6': 1000000,
|
||||
'claude-sonnet-5': 1000000,
|
||||
'claude-opus-4-6': 1000000,
|
||||
'claude-opus-4-7': 1000000,
|
||||
'claude-opus-4-8': 1000000,
|
||||
|
|
@ -408,6 +409,7 @@ const anthropicMaxOutputs = {
|
|||
'claude-haiku-4-5': 64000,
|
||||
'claude-sonnet-4': 64000,
|
||||
'claude-sonnet-4-6': 64000,
|
||||
'claude-sonnet-5': 128000,
|
||||
'claude-opus-4': 32000,
|
||||
'claude-opus-4-5': 64000,
|
||||
'claude-opus-4-6': 128000,
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ import {
|
|||
supportsAdaptiveThinking,
|
||||
omitsSamplingParameters,
|
||||
omitsThinkingByDefault,
|
||||
requiresExplicitThinkingDisabled,
|
||||
resolveThinkingDisplay,
|
||||
bedrockOutputParser,
|
||||
bedrockInputParser,
|
||||
|
|
@ -288,10 +289,17 @@ describe('omitsThinkingByDefault', () => {
|
|||
expect(omitsThinkingByDefault('claude-sonnet-4-6')).toBe(false);
|
||||
});
|
||||
|
||||
test('returns false for claude-sonnet-4-7 (Sonnet is not affected by the Opus 4.7 default)', () => {
|
||||
test('returns false for claude-sonnet-4-7 (pre-5 Sonnet keeps summarized default)', () => {
|
||||
expect(omitsThinkingByDefault('claude-sonnet-4-7')).toBe(false);
|
||||
});
|
||||
|
||||
test('returns true for Sonnet 5+ (omits thinking display by default)', () => {
|
||||
expect(omitsThinkingByDefault('claude-sonnet-5')).toBe(true);
|
||||
expect(omitsThinkingByDefault('claude-sonnet-5-20260101')).toBe(true);
|
||||
expect(omitsThinkingByDefault('anthropic.claude-sonnet-5')).toBe(true);
|
||||
expect(omitsThinkingByDefault('claude-sonnet-9')).toBe(true);
|
||||
});
|
||||
|
||||
test('returns false for claude-haiku-4-5', () => {
|
||||
expect(omitsThinkingByDefault('claude-haiku-4-5')).toBe(false);
|
||||
});
|
||||
|
|
@ -334,12 +342,27 @@ describe('omitsSamplingParameters', () => {
|
|||
});
|
||||
});
|
||||
|
||||
test('returns false for older Opus and non-Opus models', () => {
|
||||
test('returns true for Sonnet 5+ models (non-default sampling params rejected)', () => {
|
||||
const models = [
|
||||
'claude-sonnet-5',
|
||||
'claude-sonnet-5-20260101',
|
||||
'anthropic.claude-sonnet-5',
|
||||
'us.anthropic.claude-sonnet-5',
|
||||
'claude-sonnet-9',
|
||||
];
|
||||
|
||||
models.forEach((model) => {
|
||||
expect(omitsSamplingParameters(model)).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
test('returns false for older Opus and pre-5 Sonnet models', () => {
|
||||
const models = [
|
||||
'claude-opus-4-20250514',
|
||||
'anthropic.claude-opus-4-20250514-v1:0',
|
||||
'claude-opus-4-1-20250805',
|
||||
'claude-opus-4-6',
|
||||
'claude-sonnet-4-6',
|
||||
'claude-sonnet-4-7',
|
||||
];
|
||||
|
||||
|
|
@ -349,6 +372,24 @@ describe('omitsSamplingParameters', () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe('requiresExplicitThinkingDisabled', () => {
|
||||
test('returns true for Sonnet 5+ (omitted thinking runs adaptive by default)', () => {
|
||||
expect(requiresExplicitThinkingDisabled('claude-sonnet-5')).toBe(true);
|
||||
expect(requiresExplicitThinkingDisabled('claude-sonnet-5-20260101')).toBe(true);
|
||||
expect(requiresExplicitThinkingDisabled('anthropic.claude-sonnet-5')).toBe(true);
|
||||
expect(requiresExplicitThinkingDisabled('claude-sonnet-9')).toBe(true);
|
||||
});
|
||||
|
||||
test('returns false for pre-5 Sonnet, Opus, and Mythos-class models', () => {
|
||||
// Opus 4.7+ omit -> off; Fable/Mythos reject an explicit disabled config (400)
|
||||
expect(requiresExplicitThinkingDisabled('claude-sonnet-4-6')).toBe(false);
|
||||
expect(requiresExplicitThinkingDisabled('claude-opus-4-8')).toBe(false);
|
||||
expect(requiresExplicitThinkingDisabled('claude-fable-5')).toBe(false);
|
||||
expect(requiresExplicitThinkingDisabled('claude-mythos-5')).toBe(false);
|
||||
expect(requiresExplicitThinkingDisabled('gpt-4o')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveThinkingDisplay', () => {
|
||||
test('returns "summarized" for Opus 4.7 when explicit is auto/null/undefined', () => {
|
||||
expect(resolveThinkingDisplay('claude-opus-4-7', ThinkingDisplay.auto)).toBe('summarized');
|
||||
|
|
@ -539,6 +580,29 @@ describe('bedrockInputParser', () => {
|
|||
expect(additionalFields.anthropic_beta).toEqual(BEDROCK_CLAUDE_4_BETAS);
|
||||
});
|
||||
|
||||
test('should send explicit disabled thinking for Bedrock Sonnet 5 when thinking is off', () => {
|
||||
const input = {
|
||||
model: 'anthropic.claude-sonnet-5',
|
||||
thinking: false,
|
||||
};
|
||||
const result = bedrockInputParser.parse(input) as Record<string, unknown>;
|
||||
const additionalFields = result.additionalModelRequestFields as Record<string, unknown>;
|
||||
expect(additionalFields.thinking).toEqual({ type: 'disabled' });
|
||||
});
|
||||
|
||||
test('should keep Bedrock Sonnet 5 thinking off when only the persisted AMRF disabled config remains', () => {
|
||||
// initializeBedrock feeds persisted model_parameters straight through this
|
||||
// parser. A prior "thinking off" survives only as AMRF.thinking; it must
|
||||
// not be rebuilt as adaptive on reload.
|
||||
const input = {
|
||||
model: 'anthropic.claude-sonnet-5',
|
||||
additionalModelRequestFields: { thinking: { type: 'disabled' } },
|
||||
};
|
||||
const result = bedrockInputParser.parse(input) as Record<string, unknown>;
|
||||
const additionalFields = result.additionalModelRequestFields as Record<string, unknown>;
|
||||
expect(additionalFields.thinking).toEqual({ type: 'disabled' });
|
||||
});
|
||||
|
||||
test('should respect custom thinking budget', () => {
|
||||
const input = {
|
||||
model: 'anthropic.claude-sonnet-4',
|
||||
|
|
@ -804,6 +868,18 @@ describe('bedrockInputParser', () => {
|
|||
expect(persisted.thinkingDisplay).toBe('summarized');
|
||||
});
|
||||
|
||||
test('coerces a persisted disabled AMRF.thinking back to thinking=false (Sonnet 5)', () => {
|
||||
/** A persisted Sonnet 5 "thinking off" carries AMRF.thinking =
|
||||
* { type: 'disabled' }. The schema must coerce that to top-level
|
||||
* thinking=false (not the truthy-object default of true), so the parser
|
||||
* re-emits the disabled config instead of flipping back to adaptive. */
|
||||
const persisted = bedrockInputSchema.parse({
|
||||
model: 'anthropic.claude-sonnet-5',
|
||||
additionalModelRequestFields: { thinking: { type: 'disabled' } },
|
||||
}) as Record<string, unknown>;
|
||||
expect(persisted.thinking).toBe(false);
|
||||
});
|
||||
|
||||
test('ignores unknown display values during round-trip extraction', () => {
|
||||
const persisted = bedrockInputSchema.parse({
|
||||
model: 'anthropic.claude-opus-4-7',
|
||||
|
|
|
|||
|
|
@ -10,7 +10,8 @@ const bedrockReasoningConfigValues = new Set<string>(Object.values(s.BedrockReas
|
|||
|
||||
type ThinkingConfig =
|
||||
| { type: 'enabled'; budget_tokens: number }
|
||||
| { type: 'adaptive'; display?: s.ThinkingDisplayWireValue };
|
||||
| { type: 'adaptive'; display?: s.ThinkingDisplayWireValue }
|
||||
| { type: 'disabled' };
|
||||
|
||||
/**
|
||||
* Resolves the final `thinking.display` value for an adaptive-thinking request.
|
||||
|
|
@ -142,6 +143,10 @@ export function omitsThinkingByDefault(model: string): boolean {
|
|||
if (opus && (opus.major > 4 || (opus.major === 4 && opus.minor >= 7))) {
|
||||
return true;
|
||||
}
|
||||
const sonnet = parseSonnetVersion(model);
|
||||
if (sonnet != null && sonnet.major >= 5) {
|
||||
return true;
|
||||
}
|
||||
if (s.isMythosClassModel(model)) {
|
||||
return true;
|
||||
}
|
||||
|
|
@ -153,12 +158,33 @@ export function omitsSamplingParameters(model: string): boolean {
|
|||
if (opus && (opus.major > 4 || (opus.major === 4 && opus.minor >= 7))) {
|
||||
return true;
|
||||
}
|
||||
const sonnet = parseSonnetVersion(model);
|
||||
if (sonnet != null && sonnet.major >= 5) {
|
||||
return true;
|
||||
}
|
||||
if (s.isMythosClassModel(model)) {
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether disabling thinking requires sending an explicit `{ type: 'disabled' }`
|
||||
* config rather than simply omitting the `thinking` field.
|
||||
*
|
||||
* Sonnet 5 treats an omitted `thinking` field as adaptive thinking ON by
|
||||
* default, so honoring a user who turns thinking off means sending the disabled
|
||||
* config explicitly. Opus 4.7+ run without thinking when the field is omitted,
|
||||
* and Fable/Mythos reject an explicit disabled config (400, thinking always
|
||||
* on), so both are excluded.
|
||||
*
|
||||
* See https://platform.claude.com/docs/en/about-claude/models/migration-guide#migrating-to-claude-sonnet-5
|
||||
*/
|
||||
export function requiresExplicitThinkingDisabled(model: string): boolean {
|
||||
const sonnet = parseSonnetVersion(model);
|
||||
return sonnet != null && sonnet.major >= 5;
|
||||
}
|
||||
|
||||
/** Checks if a model has a 1M context window (Sonnet 4.6+, Opus 4.6+, Opus 5+, Fable/Mythos) */
|
||||
export function supportsContext1m(model: string): boolean {
|
||||
const sonnet = parseSonnetVersion(model);
|
||||
|
|
@ -265,7 +291,11 @@ export const bedrockInputSchema = s.tConversationSchema
|
|||
if ((obj as AnthropicInput).additionalModelRequestFields?.thinking != null) {
|
||||
const _obj = obj as AnthropicInput;
|
||||
const thinking = _obj.additionalModelRequestFields.thinking;
|
||||
obj.thinking = !!thinking;
|
||||
const isDisabled =
|
||||
typeof thinking === 'object' &&
|
||||
thinking !== null &&
|
||||
(thinking as { type?: string }).type === 'disabled';
|
||||
obj.thinking = isDisabled ? false : !!thinking;
|
||||
obj.thinkingBudget =
|
||||
typeof thinking === 'object' && 'budget_tokens' in thinking
|
||||
? thinking.budget_tokens
|
||||
|
|
@ -357,6 +387,26 @@ export const bedrockInputParser = s.tConversationSchema
|
|||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Persisted `model_parameters` can carry a prior "thinking off" only inside
|
||||
* `additionalModelRequestFields.thinking = { type: 'disabled' }` (a known
|
||||
* key that isn't spread into `additionalFields`). `initializeBedrock` feeds
|
||||
* those params straight through this parser, so surface that as
|
||||
* `thinking: false` — otherwise the disabled branch is skipped and the
|
||||
* config rebuilds adaptive, flipping a user's Sonnet 5 setting back on.
|
||||
*/
|
||||
const persistedThinking = (
|
||||
typedData.additionalModelRequestFields as { thinking?: unknown } | undefined
|
||||
)?.thinking;
|
||||
if (
|
||||
additionalFields.thinking === undefined &&
|
||||
typeof persistedThinking === 'object' &&
|
||||
persistedThinking !== null &&
|
||||
(persistedThinking as { type?: string }).type === 'disabled'
|
||||
) {
|
||||
additionalFields.thinking = false;
|
||||
}
|
||||
|
||||
/** Configure thinking for Bedrock Anthropic models: 3.7 Sonnet, Claude 4+ (opus/sonnet/haiku), and Mythos-class (Fable/Mythos). */
|
||||
if (
|
||||
typeof typedData.model === 'string' &&
|
||||
|
|
@ -376,9 +426,13 @@ export const bedrockInputParser = s.tConversationSchema
|
|||
delete additionalFields.effort;
|
||||
|
||||
if (additionalFields.thinking === false) {
|
||||
delete additionalFields.thinking;
|
||||
delete additionalFields.thinkingBudget;
|
||||
delete additionalFields.thinkingDisplay;
|
||||
if (requiresExplicitThinkingDisabled(typedData.model as string)) {
|
||||
additionalFields.thinking = { type: 'disabled' };
|
||||
} else {
|
||||
delete additionalFields.thinking;
|
||||
}
|
||||
} else {
|
||||
/**
|
||||
* Persisted agent `model_parameters` round-trip back through this
|
||||
|
|
|
|||
|
|
@ -1996,6 +1996,7 @@ const sharedAnthropicModels = [
|
|||
'claude-fable-5',
|
||||
'claude-opus-4-8',
|
||||
'claude-opus-4-7',
|
||||
'claude-sonnet-5',
|
||||
'claude-sonnet-4-6',
|
||||
'claude-opus-4-6',
|
||||
'claude-sonnet-4-5',
|
||||
|
|
@ -2021,6 +2022,7 @@ export const bedrockModels = [
|
|||
'anthropic.claude-fable-5',
|
||||
'anthropic.claude-opus-4-8',
|
||||
'anthropic.claude-opus-4-7',
|
||||
'anthropic.claude-sonnet-5',
|
||||
'anthropic.claude-sonnet-4-6',
|
||||
'anthropic.claude-opus-4-6-v1',
|
||||
'anthropic.claude-sonnet-4-5-20250929-v1:0',
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ describe('anthropicSettings', () => {
|
|||
describe('maxOutputTokens.reset()', () => {
|
||||
const { reset } = anthropicSettings.maxOutputTokens;
|
||||
|
||||
describe('Claude Sonnet models', () => {
|
||||
describe('Claude Sonnet 4.x models (64K limit)', () => {
|
||||
it('should return 64K for claude-sonnet-4', () => {
|
||||
expect(reset('claude-sonnet-4')).toBe(64000);
|
||||
});
|
||||
|
|
@ -19,12 +19,22 @@ describe('anthropicSettings', () => {
|
|||
expect(reset('claude-sonnet-4-5')).toBe(64000);
|
||||
});
|
||||
|
||||
it('should return 64K for claude-sonnet-5', () => {
|
||||
expect(reset('claude-sonnet-5')).toBe(64000);
|
||||
it('should return 64K for claude-sonnet-4-6', () => {
|
||||
expect(reset('claude-sonnet-4-6')).toBe(64000);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Claude Sonnet 5+ models (128K limit)', () => {
|
||||
it('should return 128K for claude-sonnet-5', () => {
|
||||
expect(reset('claude-sonnet-5')).toBe(128000);
|
||||
});
|
||||
|
||||
it('should return 64K for future versions like claude-sonnet-9', () => {
|
||||
expect(reset('claude-sonnet-9')).toBe(64000);
|
||||
it('should return 128K for future versions like claude-sonnet-9', () => {
|
||||
expect(reset('claude-sonnet-9')).toBe(128000);
|
||||
});
|
||||
|
||||
it('should return 128K for double-digit minor versions like claude-sonnet-5-10', () => {
|
||||
expect(reset('claude-sonnet-5-10')).toBe(128000);
|
||||
});
|
||||
});
|
||||
|
||||
|
|
@ -265,6 +275,20 @@ describe('anthropicSettings', () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe('Claude Sonnet 5+ models (128K cap)', () => {
|
||||
it('should allow 100K for claude-sonnet-5', () => {
|
||||
expect(set(100000, 'claude-sonnet-5')).toBe(100000);
|
||||
});
|
||||
|
||||
it('should cap at 128K for claude-sonnet-5 when value exceeds', () => {
|
||||
expect(set(150000, 'claude-sonnet-5')).toBe(128000);
|
||||
});
|
||||
|
||||
it('should cap at 128K for future versions like claude-sonnet-9', () => {
|
||||
expect(set(150000, 'claude-sonnet-9')).toBe(128000);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Claude Opus 4.5+ models (64K cap)', () => {
|
||||
it('should cap at 64K for claude-opus-4-5 when value exceeds', () => {
|
||||
expect(set(100000, 'claude-opus-4-5')).toBe(64000);
|
||||
|
|
|
|||
|
|
@ -547,6 +547,10 @@ export const anthropicSettings = {
|
|||
return ANTHROPIC_MAX_OUTPUT;
|
||||
}
|
||||
|
||||
if (/claude-sonnet[-.]?(?:[5-9]|\d{2,})/.test(modelName)) {
|
||||
return ANTHROPIC_MAX_OUTPUT;
|
||||
}
|
||||
|
||||
if (/claude-(?:sonnet|haiku)[-.]?[4-9]/.test(modelName)) {
|
||||
return CLAUDE_4_64K_MAX_OUTPUT;
|
||||
}
|
||||
|
|
@ -576,6 +580,13 @@ export const anthropicSettings = {
|
|||
return value;
|
||||
}
|
||||
|
||||
if (/claude-sonnet[-.]?(?:[5-9]|\d{2,})/.test(modelName)) {
|
||||
if (value > ANTHROPIC_MAX_OUTPUT) {
|
||||
return ANTHROPIC_MAX_OUTPUT;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
if (/claude-(?:sonnet|haiku)[-.]?[4-9]/.test(modelName) && value > CLAUDE_4_64K_MAX_OUTPUT) {
|
||||
return CLAUDE_4_64K_MAX_OUTPUT;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -18,6 +18,7 @@ import logger from '~/config/winston';
|
|||
* These are the standard Anthropic model names as served by Vertex AI
|
||||
*/
|
||||
export const defaultVertexModels: string[] = [
|
||||
'claude-sonnet-5',
|
||||
'claude-sonnet-4-6',
|
||||
'claude-3-7-sonnet-20250219',
|
||||
'claude-3-5-sonnet-v2@20241022',
|
||||
|
|
|
|||
|
|
@ -2473,6 +2473,54 @@ describe('Claude Model Tests', () => {
|
|||
cacheTokenValues['claude-mythos-5'].read,
|
||||
);
|
||||
});
|
||||
|
||||
it('should return correct prompt and completion rates for Claude Sonnet 5', () => {
|
||||
expect(getMultiplier({ model: 'claude-sonnet-5', tokenType: 'prompt' })).toBe(
|
||||
tokenValues['claude-sonnet-5'].prompt,
|
||||
);
|
||||
expect(getMultiplier({ model: 'claude-sonnet-5', tokenType: 'completion' })).toBe(
|
||||
tokenValues['claude-sonnet-5'].completion,
|
||||
);
|
||||
});
|
||||
|
||||
it('should pin Claude Sonnet 5 to introductory $2 / $10 per MTok (through 2026-08-31)', () => {
|
||||
expect(tokenValues['claude-sonnet-5']).toEqual({ prompt: 2, completion: 10 });
|
||||
});
|
||||
|
||||
it('should apply introductory cache rates ($2.50 / $0.20) for Claude Sonnet 5', () => {
|
||||
expect(cacheTokenValues['claude-sonnet-5']).toEqual({ write: 2.5, read: 0.2 });
|
||||
});
|
||||
|
||||
it('should handle Claude Sonnet 5 model name variations without matching Sonnet 4', () => {
|
||||
const modelVariations = [
|
||||
'claude-sonnet-5',
|
||||
'claude-sonnet-5-20260101',
|
||||
'claude-sonnet-5-latest',
|
||||
'anthropic/claude-sonnet-5',
|
||||
'claude-sonnet-5/anthropic',
|
||||
'anthropic.claude-sonnet-5',
|
||||
];
|
||||
|
||||
modelVariations.forEach((model) => {
|
||||
const valueKey = getValueKey(model);
|
||||
expect(valueKey).toBe('claude-sonnet-5');
|
||||
expect(getMultiplier({ model, tokenType: 'prompt' })).toBe(
|
||||
tokenValues['claude-sonnet-5'].prompt,
|
||||
);
|
||||
expect(getMultiplier({ model, tokenType: 'completion' })).toBe(
|
||||
tokenValues['claude-sonnet-5'].completion,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
it('should return correct cache rates for Claude Sonnet 5', () => {
|
||||
expect(getCacheMultiplier({ model: 'claude-sonnet-5', cacheType: 'write' })).toBe(
|
||||
cacheTokenValues['claude-sonnet-5'].write,
|
||||
);
|
||||
expect(getCacheMultiplier({ model: 'claude-sonnet-5', cacheType: 'read' })).toBe(
|
||||
cacheTokenValues['claude-sonnet-5'].read,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Premium Token Pricing', () => {
|
||||
|
|
@ -2500,6 +2548,7 @@ describe('Premium Token Pricing', () => {
|
|||
'claude-fable-5',
|
||||
'claude-mythos-5',
|
||||
'claude-sonnet-4-6',
|
||||
'claude-sonnet-5',
|
||||
];
|
||||
claudeModels.forEach((model) => {
|
||||
expect(premiumTokenValues[model]).toBeUndefined();
|
||||
|
|
|
|||
|
|
@ -172,6 +172,8 @@ export const tokenValues: Record<string, { prompt: number; completion: number }>
|
|||
'claude-sonnet-4': { prompt: 3, completion: 15 },
|
||||
'claude-sonnet-4-5': { prompt: 3, completion: 15 },
|
||||
'claude-sonnet-4-6': { prompt: 3, completion: 15 },
|
||||
// Sonnet 5 introductory pricing through 2026-08-31; revert to { prompt: 3, completion: 15 } after.
|
||||
'claude-sonnet-5': { prompt: 2, completion: 10 },
|
||||
'command-r': { prompt: 0.5, completion: 1.5 },
|
||||
'command-r-plus': { prompt: 3, completion: 15 },
|
||||
'command-text': { prompt: 1.5, completion: 2.0 },
|
||||
|
|
@ -305,6 +307,8 @@ export const cacheTokenValues: Record<string, { write: number; read: number }> =
|
|||
'claude-sonnet-4': { write: 3.75, read: 0.3 },
|
||||
'claude-sonnet-4-5': { write: 3.75, read: 0.3 },
|
||||
'claude-sonnet-4-6': { write: 3.75, read: 0.3 },
|
||||
// Sonnet 5 introductory pricing through 2026-08-31; revert to { write: 3.75, read: 0.3 } after.
|
||||
'claude-sonnet-5': { write: 2.5, read: 0.2 },
|
||||
'claude-opus-4': { write: 18.75, read: 1.5 },
|
||||
'claude-opus-4-5': { write: 6.25, read: 0.5 },
|
||||
'claude-opus-4-6': { write: 6.25, read: 0.5 },
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue