mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-10-22 18:13:27 +00:00
🫧 feat: Claude Opus 4.7 Reasoning Visibility (#12701)
* 🫧 fix: Restore Claude Opus 4.7 Reasoning Visibility Claude Opus 4.7 omits `thinking` content from Messages API responses by default — empty thinking blocks still stream, but the `thinking` field is blank unless the caller passes `display: "summarized"` in the adaptive thinking config. This silenced the LibreChat "Thoughts" UI for Anthropic (and Anthropic-on-Bedrock) adaptive models. - Extend `ThinkingConfigAdaptive` in `packages/api/src/types/anthropic.ts` with an optional `display: 'summarized' | 'omitted'` field - Emit `{ type: 'adaptive', display: 'summarized' }` from `configureReasoning` in `packages/api/src/endpoints/anthropic/helpers.ts` - Emit `{ type: 'adaptive', display: 'summarized' }` from `bedrockInputParser` in `packages/data-provider/src/bedrock.ts` and update the local `ThinkingConfig` union - Update existing adaptive-thinking assertions to include the new field - Add dedicated tests asserting `display: 'summarized'` flows through both the Anthropic endpoint and the Bedrock parser See https://platform.claude.com/docs/en/about-claude/models/whats-new-claude-4-7#thinking-content-omitted-by-default * refactor: Gate `display: summarized` on Opus 4.7+ Narrow the reasoning-visibility opt-in to the models that actually omit thinking content by default, instead of applying it to every adaptive model. Pre-Opus-4.7 adaptive models (Opus 4.6, Sonnet 4.6) already return summaries, so sending the field is unnecessary noise. - Add `omitsThinkingByDefault(model)` in `packages/data-provider/src/bedrock.ts` that returns true only for Opus 4.7+ (including future majors like Opus 5+) - Bedrock parser now only attaches `display: 'summarized'` when the helper matches, keeping the adaptive object unchanged for older models - Anthropic endpoint `configureReasoning` uses the same helper so its emit path matches the Bedrock one - Tests: replace the blanket `display: 'summarized'` assertions with model-specific ones (Opus 4.7 gets it, Opus 4.6 / Sonnet 4.6 do not), add a dedicated `omitsThinkingByDefault` suite covering naming variants and future versions * feat: Configurable Thought Visibility for Anthropic Adaptive Models Expose the Anthropic `thinking.display` API field as a user-facing parameter so users can override the `auto` default (which stays as the Opus-4.7+ opt-in added earlier in this PR). Also fixes the CI type error by widening the adaptive thinking type assignment via a resolver helper that returns a properly-typed object. - Add `ThinkingDisplay` enum (`auto` | `summarized` | `omitted`) and matching zod schema in `packages/data-provider/src/schemas.ts` - Add `thinkingDisplay` to `tConversationSchema`, `anthropicSettings`, and the pick lists for Bedrock input/parser + Anthropic agent params - Add `resolveThinkingDisplay(model, explicit)` helper in `packages/data-provider/src/bedrock.ts` that returns the wire value or undefined (auto → model default, explicit → always honored) - `bedrockInputParser` now reads `thinkingDisplay` from input and emits `display` only when the resolver returns a value; strips the field on non-adaptive-model branches so it does not leak - `configureReasoning` in the Anthropic endpoint threads `thinkingDisplay` through, uses the resolver, and casts the adaptive config to `AnthropicClientOptions['thinking']` so the widened shape compiles against the stale installed SDK types - Add UI slider for `thinkingDisplay` in `parameterSettings.ts` next to `effort`, with three-position `com_ui_auto` / `com_ui_summarized` / `com_ui_omitted` labels - Add translation keys `com_endpoint_anthropic_thinking_display`, `com_endpoint_anthropic_thinking_display_desc`, `com_ui_summarized`, `com_ui_omitted` - Add tests: `resolveThinkingDisplay` suite (5 cases covering auto / explicit / unknown input), parser round-trip tests for all three modes on Opus 4.6 and Opus 4.7, Anthropic endpoint tests for explicit summarized/omitted overrides * fix: Drop `thinkingDisplay` When Adaptive Thinking Is Disabled If a user turns adaptive thinking off but had previously selected a `thinkingDisplay` value, the stale field was left in `additionalFields` and ended up merged into the Bedrock request's `additionalModelRequestFields`. That leaks a non-Bedrock key into the payload and can round-trip back into `llmConfig`. - Delete `additionalFields.thinkingDisplay` alongside `thinking` and `thinkingBudget` in the `thinking === false` branch of `bedrockInputParser` - Add a regression test asserting `thinking`, `thinkingBudget`, and `thinkingDisplay` are all absent when adaptive thinking is disabled on an Opus 4.7 request Reported by chatgpt-codex-connector on PR #12701. * refactor: Consolidate `ThinkingDisplay` Types and Preserve Persisted Display Address review findings on PR #12701: - [Codex P2] `bedrockInputSchema.transform` now extracts `thinking.display` from persisted `additionalModelRequestFields` back into the top-level `thinkingDisplay` field so explicit `'omitted'` round-trips through storage instead of being silently reverted to `'summarized'` on the next parse. - [Codex P2] `getLLMConfig` in the Anthropic endpoint now reads `.display` from a persisted `thinking` object (agents store the full Anthropic shape) and uses it as the fallback for `thinkingDisplay` when no top-level override is present. - [Audit #2] Collapse the three parallel wire-value types into a single `ThinkingDisplayWireValue = Exclude<ThinkingDisplay, 'auto'>` exported from `schemas.ts`; remove the duplicate `ThinkingDisplay` alias in `packages/api/src/types/anthropic.ts` (which collided with the enum name) and the `ThinkingDisplayValue` alias in `bedrock.ts`. - [Audit #3] Add `thinkingDisplay` to the `TEndpointOption` pick list next to `effort`. - [Audit #4] Add a TODO comment next to the `as AnthropicClientOptions['thinking']` cast explaining the stale `@librechat/agents` SDK types that require it. - Add tests: four round-trip cases asserting `bedrockInputSchema` recovers `display` from persisted AMRF (Opus 4.7 omitted, pre-4.7 summarized, unknown-value ignore, explicit top-level wins), and two `getLLMConfig` cases asserting the Anthropic endpoint preserves and overrides persisted `thinking.display`. * fix: Preserve Persisted `thinking.display` in bedrockInputParser The parser constructed a fresh adaptive thinking config without looking at any `display` already embedded in the incoming `additionalModelRequestFields.thinking`. On round-trip through `initializeBedrock`, a persisted user choice of `'omitted'` on Opus 4.7+ was silently reverted to `'summarized'` by the auto fallback. - Extract `extractPersistedDisplay` helper and reuse it in both the schema transform (form-state round-trip) and the parser (wire-request round-trip) - `bedrockInputParser` now feeds the persisted display as the resolver's explicit value when no top-level `thinkingDisplay` override is set - Add regression tests: parser preserves `display: 'omitted'` for persisted Opus 4.7 AMRF, and top-level `thinkingDisplay` still wins over persisted AMRF display Reported by chatgpt-codex-connector (P1) on PR #12701.
This commit is contained in:
Danny Avila
GitHub
parent
e2e3284713
commit
034b672d0c
10 changed files with 571 additions and 7 deletions
|
|
@ -2,9 +2,11 @@ import { logger } from '@librechat/data-schemas';
|
|||
import { AnthropicClientOptions } from '@librechat/agents';
|
||||
import {
|
||||
EModelEndpoint,
|
||||
ThinkingDisplay,
|
||||
AnthropicEffort,
|
||||
anthropicSettings,
|
||||
supportsContext1m,
|
||||
resolveThinkingDisplay,
|
||||
supportsAdaptiveThinking,
|
||||
} from 'librechat-data-provider';
|
||||
import { matchModelName } from '~/utils/tokens';
|
||||
|
|
@ -73,6 +75,7 @@ function configureReasoning(
|
|||
thinking?: boolean;
|
||||
thinkingBudget?: number | null;
|
||||
effort?: AnthropicEffort | string | null;
|
||||
thinkingDisplay?: ThinkingDisplay | string | null;
|
||||
} = {},
|
||||
): AnthropicClientOptions & { max_tokens?: number } {
|
||||
const updatedOptions = { ...anthropicInput };
|
||||
|
|
@ -80,7 +83,25 @@ function configureReasoning(
|
|||
const modelName = updatedOptions.model ?? '';
|
||||
|
||||
if (extendedOptions.thinking && modelName && supportsAdaptiveThinking(modelName)) {
|
||||
updatedOptions.thinking = { type: 'adaptive' };
|
||||
/**
|
||||
* For Opus 4.7+, Anthropic omits thinking content from responses by
|
||||
* default. Resolver returns `'summarized'` for those models (so the
|
||||
* LibreChat "Thoughts" UI keeps working) and leaves the field off for
|
||||
* older adaptive models, while honoring an explicit user choice.
|
||||
*
|
||||
* https://platform.claude.com/docs/en/about-claude/models/whats-new-claude-4-7#thinking-content-omitted-by-default
|
||||
*/
|
||||
const display = resolveThinkingDisplay(modelName, extendedOptions.thinkingDisplay);
|
||||
const adaptive = display
|
||||
? { type: 'adaptive' as const, display }
|
||||
: { type: 'adaptive' as const };
|
||||
/**
|
||||
* TODO: Remove the cast once `@librechat/agents` updates its
|
||||
* `ChatAnthropicMessages['thinking']` type to include the `display` field
|
||||
* added with Claude Opus 4.7. The cast is required because the installed
|
||||
* agents SDK still uses the pre-4.7 `ThinkingConfigAdaptive` shape.
|
||||
*/
|
||||
updatedOptions.thinking = adaptive as AnthropicClientOptions['thinking'];
|
||||
|
||||
const effort = extendedOptions.effort;
|
||||
if (effort && effort !== AnthropicEffort.unset) {
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
import { AnthropicEffort } from 'librechat-data-provider';
|
||||
import { AnthropicEffort, ThinkingDisplay } from 'librechat-data-provider';
|
||||
import type * as t from '~/types';
|
||||
import { getLLMConfig } from './llm';
|
||||
|
||||
|
|
@ -1041,6 +1041,105 @@ describe('getLLMConfig', () => {
|
|||
});
|
||||
});
|
||||
|
||||
it('should request summarized thinking display for Opus 4.7 (opt back in)', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: { model: 'claude-opus-4-7', thinking: true },
|
||||
});
|
||||
|
||||
const thinking = result.llmConfig.thinking as unknown as {
|
||||
type: string;
|
||||
display?: string;
|
||||
};
|
||||
expect(thinking.type).toBe('adaptive');
|
||||
expect(thinking.display).toBe('summarized');
|
||||
});
|
||||
|
||||
it('should NOT set thinking.display for pre-Opus-4.7 adaptive models', () => {
|
||||
const pre47Models = ['claude-opus-4-6', 'claude-sonnet-4-6'];
|
||||
|
||||
pre47Models.forEach((model) => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: { model, thinking: true },
|
||||
});
|
||||
|
||||
const thinking = result.llmConfig.thinking as unknown as {
|
||||
type: string;
|
||||
display?: string;
|
||||
};
|
||||
expect(thinking.type).toBe('adaptive');
|
||||
expect(thinking.display).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
it('should honor explicit thinkingDisplay="summarized" on Opus 4.6', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: {
|
||||
model: 'claude-opus-4-6',
|
||||
thinking: true,
|
||||
thinkingDisplay: ThinkingDisplay.summarized,
|
||||
},
|
||||
});
|
||||
|
||||
const thinking = result.llmConfig.thinking as unknown as {
|
||||
type: string;
|
||||
display?: string;
|
||||
};
|
||||
expect(thinking.type).toBe('adaptive');
|
||||
expect(thinking.display).toBe('summarized');
|
||||
});
|
||||
|
||||
it('should honor explicit thinkingDisplay="omitted" on Opus 4.7', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: {
|
||||
model: 'claude-opus-4-7',
|
||||
thinking: true,
|
||||
thinkingDisplay: ThinkingDisplay.omitted,
|
||||
},
|
||||
});
|
||||
|
||||
const thinking = result.llmConfig.thinking as unknown as {
|
||||
type: string;
|
||||
display?: string;
|
||||
};
|
||||
expect(thinking.type).toBe('adaptive');
|
||||
expect(thinking.display).toBe('omitted');
|
||||
});
|
||||
|
||||
it('should recover display from persisted agent thinking object (Opus 4.7 omitted)', () => {
|
||||
/** Agents persist `thinking` as the full Anthropic object. Without
|
||||
* extracting `.display` back into `thinkingDisplay`, Opus 4.7's auto
|
||||
* resolver would silently flip it to 'summarized'. */
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: {
|
||||
model: 'claude-opus-4-7',
|
||||
thinking: { type: 'adaptive', display: 'omitted' },
|
||||
} as unknown as t.AnthropicModelOptions,
|
||||
});
|
||||
|
||||
const thinking = result.llmConfig.thinking as unknown as {
|
||||
type: string;
|
||||
display?: string;
|
||||
};
|
||||
expect(thinking.type).toBe('adaptive');
|
||||
expect(thinking.display).toBe('omitted');
|
||||
});
|
||||
|
||||
it('explicit thinkingDisplay wins over persisted thinking.display', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: {
|
||||
model: 'claude-opus-4-7',
|
||||
thinking: { type: 'adaptive', display: 'summarized' },
|
||||
thinkingDisplay: ThinkingDisplay.omitted,
|
||||
} as unknown as t.AnthropicModelOptions,
|
||||
});
|
||||
|
||||
const thinking = result.llmConfig.thinking as unknown as {
|
||||
type: string;
|
||||
display?: string;
|
||||
};
|
||||
expect(thinking.display).toBe('omitted');
|
||||
});
|
||||
|
||||
it('should exclude topP/topK for Sonnet 4.6 with adaptive thinking', () => {
|
||||
const result = getLLMConfig('test-key', {
|
||||
modelOptions: {
|
||||
|
|
|
|||
|
|
@ -1,7 +1,12 @@
|
|||
import { Dispatcher, ProxyAgent } from 'undici';
|
||||
import { logger } from '@librechat/data-schemas';
|
||||
import { AnthropicClientOptions } from '@librechat/agents';
|
||||
import { anthropicSettings, removeNullishValues, AuthKeys } from 'librechat-data-provider';
|
||||
import {
|
||||
anthropicSettings,
|
||||
removeNullishValues,
|
||||
ThinkingDisplay,
|
||||
AuthKeys,
|
||||
} from 'librechat-data-provider';
|
||||
import type {
|
||||
AnthropicLLMConfigResult,
|
||||
AnthropicConfigOptions,
|
||||
|
|
@ -83,12 +88,32 @@ function getLLMConfig(
|
|||
credentials: string | AnthropicCredentials | undefined,
|
||||
options: AnthropicConfigOptions = {},
|
||||
): AnthropicLLMConfigResult {
|
||||
/**
|
||||
* Persisted agent `model_parameters` may round-trip `thinking` as the full
|
||||
* Anthropic object `{ type: 'adaptive', display: 'omitted' }` rather than a
|
||||
* boolean. Pull any `.display` out of it so an explicit user choice is not
|
||||
* silently demoted to `'auto'` (which would then resolve to `'summarized'`
|
||||
* on Opus 4.7+).
|
||||
*/
|
||||
const persistedThinking = options.modelOptions?.thinking;
|
||||
const persistedDisplay =
|
||||
typeof persistedThinking === 'object' &&
|
||||
persistedThinking != null &&
|
||||
'display' in persistedThinking &&
|
||||
typeof (persistedThinking as { display?: unknown }).display === 'string'
|
||||
? ((persistedThinking as { display: string }).display as ThinkingDisplay | string)
|
||||
: undefined;
|
||||
|
||||
const systemOptions = {
|
||||
thinking: options.modelOptions?.thinking ?? anthropicSettings.thinking.default,
|
||||
promptCache: options.modelOptions?.promptCache ?? anthropicSettings.promptCache.default,
|
||||
thinkingBudget:
|
||||
options.modelOptions?.thinkingBudget ?? anthropicSettings.thinkingBudget.default,
|
||||
effort: options.modelOptions?.effort ?? anthropicSettings.effort.default,
|
||||
thinkingDisplay:
|
||||
options.modelOptions?.thinkingDisplay ??
|
||||
persistedDisplay ??
|
||||
anthropicSettings.thinkingDisplay.default,
|
||||
};
|
||||
|
||||
if (options.modelOptions) {
|
||||
|
|
@ -96,6 +121,7 @@ function getLLMConfig(
|
|||
delete options.modelOptions.promptCache;
|
||||
delete options.modelOptions.thinkingBudget;
|
||||
delete options.modelOptions.effort;
|
||||
delete options.modelOptions.thinkingDisplay;
|
||||
} else {
|
||||
throw new Error('No modelOptions provided');
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ import { z } from 'zod';
|
|||
import { Dispatcher } from 'undici';
|
||||
import { AuthKeys, anthropicSchema, TVertexAISchema } from 'librechat-data-provider';
|
||||
import type { AnthropicClientOptions } from '@librechat/agents';
|
||||
import type { ThinkingDisplayWireValue } from 'librechat-data-provider';
|
||||
import type { LLMConfigResult } from './openai';
|
||||
import type { GoogleServiceKey } from '../utils/key';
|
||||
|
||||
|
|
@ -46,6 +47,7 @@ export interface ThinkingConfigEnabled {
|
|||
|
||||
export interface ThinkingConfigAdaptive {
|
||||
type: 'adaptive';
|
||||
display?: ThinkingDisplayWireValue;
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue