feat: Model-Aware Max Output Tokens for Google/Gemini (#13390)

* 📤 feat: Model-Aware Max Output Tokens for Google/Gemini

Resolves #13384.

Current Gemini text models (2.5 and 3+, including Gemini 3.5 Flash)
support 64K output tokens, but LibreChat defaulted every Google model
to the legacy 8K value — most visibly in the Agents model-parameter
panel.

- Add model-aware `reset`/`set` to `googleSettings.maxOutputTokens`,
  mirroring the Anthropic pattern: Gemini 2.5/3+ -> 65536, legacy
  (2.0 and earlier) and Gemma -> 8192.
- Resolve the default server-side in `getGoogleConfig` and in the
  Agents, preset, and standard Google settings panels via a shared
  `applyModelAwareDefaults` helper.
- Make `compactGoogleSchema` and `generateGoogleSchema` model-aware so
  explicit user values are preserved and not overwritten.

* 🛡️ fix: Cap Google max output at Vertex-safe limits

Addresses Codex review (P1) on #13390. Vertex AI caps current Gemini
text models at 65,535 output tokens (vs 65,536 on AI Studio) and image
models at 32,768, so an unconditional 65,536 default could make
otherwise-default Vertex requests fail validation.

- Lower the modern text default/ceiling to 65535 (valid on both Vertex
  and AI Studio).
- Resolve Gemini image models (e.g. gemini-2.5-flash-image) to 32768.
- Add reset/set + getGoogleConfig tests for image models and the Vertex
  default path.

* 🧮 fix: Respect configured Google defaults and legacy image caps

Addresses Codex review round 2 on #13390 (one P2, two P3).

- P2 (llm.ts): apply the model-aware maxOutputTokens default as the final
  fallback instead of pre-filling it, so an explicit value, `defaultParams`,
  and `addParams` all take precedence and `dropParams` is honored. Empty-string
  values stay stripped (preserves prior Gemini empty-payload handling).
- P3 (panels): pass the resolved params endpoint (`overriddenEndpointKey`) to
  `applyModelAwareDefaults`, so custom endpoints with
  `defaultParamsEndpoint: 'google'` also surface the model-aware default.
- P3 (schemas): nest the image-model check inside the 2.5+/3+ version check, so
  legacy image IDs (e.g. gemini-2.0-flash-preview-image-generation) keep the 8K
  cap instead of being treated as 32K models.
- Add tests for defaultParams precedence, dropParams, legacy image models, and
  the Vertex default path.

* 🧭 fix: Base Google defaults on final model and configured overrides

Addresses Codex review round 3 on #13390 (two P2).

- llm.ts: resolve the model-aware maxOutputTokens default from the final
  `llmConfig.model` (after defaultParams/addParams) instead of the model
  captured from modelOptions, so a model forced via addParams/paramDefinitions
  on a Google-compatible custom endpoint gets its correct limit.
- Panels: apply model-aware defaults to the built-in settings first, then
  overlay `customParams.paramDefinitions`, so an admin-configured
  maxOutputTokens default wins in the UI (consistent with backend precedence).
- Add parameterSettings.spec for applyModelAwareDefaults (incl. override
  precedence) and a getGoogleConfig final-model test.
This commit is contained in:
Danny Avila 2026-05-29 08:09:32 -07:00 committed by GitHub
parent 56a203e65f
commit 06a6c42435
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 349 additions and 14 deletions

View file

@ -49,6 +49,7 @@ export default function Settings({ conversation, setOption, models, readonly }:
const setTopP = setOption('topP');
const setTopK = setOption('topK');
const setMaxOutputTokens = setOption('maxOutputTokens');
const maxOutputTokensDefault = google.maxOutputTokens.reset(model ?? '');
return (
<div className="grid grid-cols-5 gap-6">
@ -267,7 +268,7 @@ export default function Settings({ conversation, setOption, models, readonly }:
<small className="opacity-40">
(
{localize('com_endpoint_default_with_num', {
0: google.maxOutputTokens.default + '',
0: maxOutputTokensDefault + '',
})}
)
</small>
@ -292,9 +293,9 @@ export default function Settings({ conversation, setOption, models, readonly }:
</div>
<Slider
disabled={readonly}
value={[maxOutputTokens ?? google.maxOutputTokens.default]}
value={[maxOutputTokens ?? maxOutputTokensDefault]}
onValueChange={(value) => setMaxOutputTokens(value[0])}
onDoubleClick={() => setMaxOutputTokens(google.maxOutputTokens.default)}
onDoubleClick={() => setMaxOutputTokens(maxOutputTokensDefault)}
max={google.maxOutputTokens.max}
min={google.maxOutputTokens.min}
step={google.maxOutputTokens.step}

View file

@ -11,6 +11,7 @@ import {
LocalStorageKeys,
SettingDefinition,
agentParamSettings,
applyModelAwareDefaults,
} from 'librechat-data-provider';
import type * as t from 'librechat-data-provider';
import type { AgentForm, AgentModelPanelProps, StringOption } from '~/common';
@ -82,9 +83,14 @@ export default function ModelPanel({
agentParamSettings[combinedKey] ?? agentParamSettings[overriddenEndpointKey] ?? [];
const overriddenParams = endpointsConfig[provider]?.customParams?.paramDefinitions ?? [];
const overriddenParamsMap = keyBy(overriddenParams, 'key');
return defaultParams
.filter((param) => param != null)
.map((param) => (overriddenParamsMap[param.key] as SettingDefinition) ?? param);
const modelAwareParams = applyModelAwareDefaults(
defaultParams.filter((param) => param != null),
overriddenEndpointKey,
model ?? '',
);
return modelAwareParams.map(
(param) => (overriddenParamsMap[param.key] as SettingDefinition) ?? param,
);
}, [endpointType, endpointsConfig, model, provider]);
const setOption = (optionKey: keyof t.AgentModelParameters) => (value: t.AgentParameterValue) => {

View file

@ -8,6 +8,7 @@ import {
getEndpointField,
SettingDefinition,
tConvoUpdateSchema,
applyModelAwareDefaults,
} from 'librechat-data-provider';
import type { TPreset } from 'librechat-data-provider';
import { SaveAsPresetDialog } from '~/components/Endpoints';
@ -45,9 +46,14 @@ export default function Parameters() {
const defaultParams = paramSettings[combinedKey] ?? paramSettings[overriddenEndpointKey] ?? [];
const overriddenParams = endpointsConfig[provider]?.customParams?.paramDefinitions ?? [];
const overriddenParamsMap = keyBy(overriddenParams, 'key');
return defaultParams
.filter((param) => param != null)
.map((param) => (overriddenParamsMap[param.key] as SettingDefinition) ?? param);
const modelAwareParams = applyModelAwareDefaults(
defaultParams.filter((param) => param != null),
overriddenEndpointKey,
model,
);
return modelAwareParams.map(
(param) => (overriddenParamsMap[param.key] as SettingDefinition) ?? param,
);
}, [endpointType, endpointsConfig, model, provider]);
useEffect(() => {