mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-10-25 03:32:12 +00:00
🪟 fix: Resolve Context Windows and Pricing for Newer Model Families (#14877)
* fix: resolve context windows for Qwen3.5+ and newer model families Qwen3.5 model ids matched the `qwen3` key and inherited its 40,960 token window instead of their native 262,144, so pruneMessages dropped whole conversations down to the system message once tool output grew large. Add the 3.5 through 3.8 generations, Meta Muse Spark, Muse Glimmer and Llama 4, plus newer DeepSeek, GLM, Kimi, Grok, Mistral, Nova, Cohere and MiniMax entries. Llama 4 Scout is capped at 1M rather than its 10M native ceiling, since no host serves near that and a 10M value would stop pruning from ever firing. findMatchingPattern now matches a vendor-prefixed id on its model segment first. No map key contains a slash, so a prefix could only ever contribute a spurious longer match: moonshotai/kimi-k2 matched moonshot (8 chars) over kimi-k2 (7) and reported half the real window. * fix: price newer models and make the pricing test helper faithful Models added to the context map billed either at defaultRate or at an older generation's rate. Llama 4 and Muse Spark fell to $6/1M, Kimi K3 undercharged 5x through the kimi key, and Grok 4.5/4.6 overcharged through grok-4. Add rates for 21 models. tx.ts needs no matcher change of its own, since it receives findMatchingPattern by injection from @librechat/api. The vendor-prefix fix therefore already corrects moonshotai/kimi-k2, which was matching moonshot at $2.00/$5.00 rather than kimi-k2 at $0.60/$2.50. The data-schemas test helper implemented a different algorithm from the function production injects, returning the first reverse-order match instead of the longest, so the pricing suite was not describing real billing behavior. Mirror the real implementation. Every existing test still passes, so nothing was relying on the lossy version. Rates for the Qwen 3.x plus/flash tiers, DeepSeek V3.1/V3.2, Kimi K2.6/K2.7 and Nova 2 Pro are left as they are, since published figures disagreed by more than the current fuzzy match is wrong. * test: assert parity between the context and pricing maps Neither map imports the other, so a model added to one and forgotten in the other is silently wrong at runtime: it bills at defaultRate, or it has no window to prune against. A comment in tx.spec.ts claimed such a test lived in packages/api, but none existed. Exact key parity is the wrong invariant. Both maps resolve by longest substring match, so gpt-4-32k legitimately prices through the 32k bucket without a row of its own. Assert instead that every key resolves in the other map, with exemption lists covering the genuine gaps and a further test that fails once an exemption goes stale, so the lists shrink rather than rot. Verified the assertions actually fail: injecting a context key with no price, a price with no context window, and a rate for an exempted model each fails the expected test and names the offending model. * fix: correct GLM 5.3, Kimi K3 aliases, Grok tiering and prefixed overrides Four gaps found in review. glm-5.3 matched the glm-5 key and reported 204,800 tokens rather than the 1,048,576 it shares with 5.2. Add both the context and pricing rows. The dot-prefixed Kimi K3 ids were added to the context map without pricing counterparts, so moonshot.kimi-k3 resolved to moonshot.kimi and moonshotai.kimi-k3 to moonshot, billing completion at 2.5 and 5.0 rather than 15.0. Grok 4.5/4.6 carry a 500K window, which makes xAI's doubled rate above a 200K prompt reachable, but neither key existed in premiumTokenValues so long prompts stayed on the base rate. Add the premium entries. Matching the model segment of a vendor-prefixed id was too eager. EndpointTokenConfig is an arbitrary record and one built from OpenRouter is keyed by org/model, so an alias like org/model-latest could resolve to a bare model entry and pick the wrong configured context and rates. Keep a matched key that carries the vendor, and retry on the segment only when the whole id resolved to a prefix-only match. * refactor: type the model matcher as keys-only and drop the caller casts findMatchingPattern reads keys and never inspects a value, but its parameter was typed as a union of the two maps it happens to be called with, so every injection site cast into it. pricing.spec.ts already carried that cast and parity.spec.ts copied it. Name the intent instead. Both adapters now pass their map through unchanged, and a future change that starts reading values will fail to compile rather than slip past a cast.
This commit is contained in:
Marco Beretta
GitHub
parent
5fc05ac037
commit
5d88b8453e
7 changed files with 479 additions and 19 deletions
|
|
@ -8,8 +8,7 @@ import { buildTokenConfigMap } from './pricing';
|
|||
/** Adapters: TxDeps types are looser than the utils signatures (string endpoint, undefined miss) */
|
||||
const { getValueKey, getMultiplier, getCacheMultiplier } = createTxMethods(mongoose, {
|
||||
matchModelName: (model, endpoint) => matchModelName(model, endpoint as EModelEndpoint),
|
||||
findMatchingPattern: (model, values) =>
|
||||
findMatchingPattern(model, values as Record<string, number>) ?? undefined,
|
||||
findMatchingPattern: (model, values) => findMatchingPattern(model, values) ?? undefined,
|
||||
});
|
||||
|
||||
const deps = { getValueKey, getMultiplier, getCacheMultiplier };
|
||||
|
|
|
|||
89
packages/api/src/utils/parity.spec.ts
Normal file
89
packages/api/src/utils/parity.spec.ts
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
import { EModelEndpoint } from 'librechat-data-provider';
|
||||
import { createTxMethods, tokenValues, defaultRate } from '@librechat/data-schemas';
|
||||
import { maxTokensMap, getModelMaxTokens, matchModelName, findMatchingPattern } from './tokens';
|
||||
|
||||
/**
|
||||
* Parity between the context-window map here and the pricing map in
|
||||
* `@librechat/data-schemas`. The two are maintained by hand and neither imports
|
||||
* the other, so a model added to one and forgotten in the other is silently
|
||||
* wrong at runtime: it either bills at `defaultRate` or has no window to prune
|
||||
* against. These tests fail on the half-added model rather than on the invoice.
|
||||
*
|
||||
* Exact key parity is not the invariant. Both maps resolve by longest-substring
|
||||
* match, so `gpt-4-32k` legitimately prices via the `32k` bucket without its own
|
||||
* row. What must hold is that every key *resolves* in the other map.
|
||||
*/
|
||||
|
||||
/** Context-map keys with no pricing key. Shrink this list, do not grow it. */
|
||||
const UNPRICED = new Set([
|
||||
/** Degenerate prefix key: no pricing key is a substring of it. */
|
||||
'mistral-',
|
||||
/** AWS has not published a Nova 2 Pro rate. */
|
||||
'nova-2-pro',
|
||||
]);
|
||||
|
||||
/** Pricing keys with no context window. Shrink this list, do not grow it. */
|
||||
const NO_CONTEXT = new Set([
|
||||
/** Legacy OpenAI pricing buckets, not model ids. */
|
||||
'8k',
|
||||
'32k',
|
||||
'4k',
|
||||
'16k',
|
||||
/** Priced, but no published window has been added to the context map. */
|
||||
'devstral',
|
||||
'voxtral',
|
||||
'holo2',
|
||||
]);
|
||||
|
||||
const { getValueKey } = createTxMethods({} as typeof import('mongoose'), {
|
||||
matchModelName: (model: string, endpoint?: string) =>
|
||||
matchModelName(model, endpoint as EModelEndpoint),
|
||||
findMatchingPattern: (model: string, values: Record<string, number | Record<string, number>>) =>
|
||||
findMatchingPattern(model, values) ?? undefined,
|
||||
});
|
||||
|
||||
const contextKeys = Object.keys(maxTokensMap[EModelEndpoint.openAI]);
|
||||
const priceKeys = Object.keys(tokenValues);
|
||||
|
||||
describe('context map and pricing map parity', () => {
|
||||
it('prices every model that has a context window', () => {
|
||||
const unpriced = contextKeys.filter((model) => !getValueKey(model) && !UNPRICED.has(model));
|
||||
expect(unpriced).toEqual([]);
|
||||
});
|
||||
|
||||
it('gives every priced model a context window', () => {
|
||||
const windowless = priceKeys.filter(
|
||||
(model) =>
|
||||
getModelMaxTokens(model, EModelEndpoint.openAI) === undefined && !NO_CONTEXT.has(model),
|
||||
);
|
||||
expect(windowless).toEqual([]);
|
||||
});
|
||||
|
||||
it('never lets a context-map model silently fall to the default rate', () => {
|
||||
for (const model of contextKeys) {
|
||||
if (UNPRICED.has(model)) {
|
||||
continue;
|
||||
}
|
||||
const valueKey = getValueKey(model);
|
||||
expect(valueKey).toBeDefined();
|
||||
expect(tokenValues[valueKey as string]).toBeDefined();
|
||||
}
|
||||
});
|
||||
|
||||
it('keeps both exemption lists free of stale entries', () => {
|
||||
const fixed = {
|
||||
unpriced: [...UNPRICED].filter((model) => getValueKey(model)),
|
||||
windowless: [...NO_CONTEXT].filter(
|
||||
(model) => getModelMaxTokens(model, EModelEndpoint.openAI) !== undefined,
|
||||
),
|
||||
};
|
||||
expect(fixed).toEqual({ unpriced: [], windowless: [] });
|
||||
});
|
||||
|
||||
it('bills a representative new model at its own rate, not the default', () => {
|
||||
for (const model of ['muse-spark-1.1', 'kimi-k3', 'glm-5.2', 'minimax-m3']) {
|
||||
expect(getModelMaxTokens(model, EModelEndpoint.openAI)).toBeGreaterThan(0);
|
||||
expect(tokenValues[getValueKey(model) as string].prompt).not.toBe(defaultRate);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
|
@ -60,3 +60,107 @@ describe('Gemini 3.7 Flash', () => {
|
|||
expect(getModelMaxTokens('gemini-3', EModelEndpoint.google)).toBe(1000000);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Qwen3.5 and later generations', () => {
|
||||
it('resolves 262K for the vLLM model id that regressed to the qwen3 window', () => {
|
||||
expect(getModelMaxTokens('Qwen/Qwen3.5-397B-A17B-FP8', EModelEndpoint.openAI)).toBe(262144);
|
||||
});
|
||||
|
||||
it('resolves 262K natively for every 3.5 through 3.8 generation', () => {
|
||||
for (const model of ['qwen3.5', 'qwen3.6', 'qwen3.7', 'qwen3.8']) {
|
||||
expect(getModelMaxTokens(model, EModelEndpoint.openAI)).toBe(262144);
|
||||
expect(getModelMaxTokens(`Qwen/${model}-27B`, EModelEndpoint.openAI)).toBe(262144);
|
||||
}
|
||||
});
|
||||
|
||||
it('prefers the hosted plus/flash tiers over the generation window', () => {
|
||||
for (const model of ['qwen3.5-plus', 'qwen3.6-flash', 'qwen3.7-plus']) {
|
||||
expect(getModelMaxTokens(model, EModelEndpoint.openAI)).toBe(1000000);
|
||||
}
|
||||
});
|
||||
|
||||
it('leaves the qwen3 generation and its parameter sizes untouched', () => {
|
||||
expect(getModelMaxTokens('qwen3', EModelEndpoint.openAI)).toBe(40960);
|
||||
expect(getModelMaxTokens('qwen3-30b-a3b', EModelEndpoint.openAI)).toBe(40960);
|
||||
expect(getModelMaxTokens('qwen3-235b-a22b', EModelEndpoint.openAI)).toBe(40960);
|
||||
expect(getModelMaxTokens('qwen2.5-72b', EModelEndpoint.openAI)).toBe(32000);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Meta Muse and Llama 4', () => {
|
||||
it('resolves the 1M Muse Spark window across id spellings', () => {
|
||||
for (const model of ['muse-spark-1.1', 'meta/muse-spark-1.2', 'musespark']) {
|
||||
expect(getModelMaxTokens(model, EModelEndpoint.openAI)).toBe(1000000);
|
||||
}
|
||||
});
|
||||
|
||||
it('resolves the 131K Muse Glimmer window', () => {
|
||||
expect(getModelMaxTokens('meta-models/Muse-Glimmer-30B', EModelEndpoint.openAI)).toBe(131072);
|
||||
});
|
||||
|
||||
it('resolves Llama 4 instead of falling through to no match', () => {
|
||||
for (const model of [
|
||||
'meta.llama4-scout-17b-instruct-v1:0',
|
||||
'meta-llama/Llama-4-Maverick-17B-128E-Instruct',
|
||||
]) {
|
||||
expect(getModelMaxTokens(model, EModelEndpoint.bedrock)).toBe(1048576);
|
||||
}
|
||||
});
|
||||
|
||||
it('keeps Llama 3.x on its own window', () => {
|
||||
expect(getModelMaxTokens('llama-3.3-70b', EModelEndpoint.openAI)).toBe(127500);
|
||||
});
|
||||
});
|
||||
|
||||
describe('newer windows for existing families', () => {
|
||||
it.each([
|
||||
['grok-4.6', 500000],
|
||||
['x-ai/grok-4-5', 500000],
|
||||
['deepseek-v3.2', 163840],
|
||||
['deepseek-v4-flash', 1048576],
|
||||
['glm-4.7', 204800],
|
||||
['z-ai/glm-5.2', 1048576],
|
||||
['glm-5.3', 1048576],
|
||||
['moonshotai/kimi-k3', 1048576],
|
||||
['kimi-k2.7-code', 262144],
|
||||
['minimax-m3', 1048576],
|
||||
['mistral-medium-3-5', 262144],
|
||||
['nova-2-lite', 995000],
|
||||
])('resolves %s to %i tokens', (model, expected) => {
|
||||
expect(getModelMaxTokens(model, EModelEndpoint.openAI)).toBe(expected);
|
||||
});
|
||||
|
||||
it('keeps command-a-plus off the wider command-a window', () => {
|
||||
expect(getModelMaxTokens('command-a', EModelEndpoint.openAI)).toBe(255500);
|
||||
expect(getModelMaxTokens('command-a-plus-05-2026', EModelEndpoint.openAI)).toBe(127500);
|
||||
});
|
||||
});
|
||||
|
||||
describe('vendor-prefixed model ids', () => {
|
||||
it('matches the model segment rather than the vendor prefix', () => {
|
||||
expect(getModelMaxTokens('moonshotai/kimi-k2', EModelEndpoint.openAI)).toBe(262144);
|
||||
expect(getModelMaxTokens('moonshot/kimi-k2', EModelEndpoint.openAI)).toBe(262144);
|
||||
expect(getModelMaxTokens('moonshotai/kimi-k3', EModelEndpoint.openAI)).toBe(1048576);
|
||||
});
|
||||
|
||||
it('still resolves prefixed ids whose model segment alone has no match', () => {
|
||||
expect(getModelMaxTokens('moonshot/v1-8k', EModelEndpoint.openAI)).toBe(
|
||||
getModelMaxTokens('moonshot', EModelEndpoint.openAI),
|
||||
);
|
||||
});
|
||||
|
||||
it('keeps a vendor-keyed override entry over a bare one', () => {
|
||||
const config: EndpointTokenConfig = {
|
||||
'openrouter/foo': { prompt: 1, completion: 2, context: 900000, output: 1 },
|
||||
foo: { prompt: 1, completion: 2, context: 1000, output: 1 },
|
||||
};
|
||||
expect(getModelMaxTokens('openrouter/foo', EModelEndpoint.custom, config)).toBe(900000);
|
||||
expect(getModelMaxTokens('openrouter/foo-latest', EModelEndpoint.custom, config)).toBe(900000);
|
||||
expect(getModelMaxTokens('other/foo-latest', EModelEndpoint.custom, config)).toBe(1000);
|
||||
});
|
||||
|
||||
it('leaves dot-separated bedrock ids alone', () => {
|
||||
expect(getModelMaxTokens('moonshot.kimi-k2-thinking', EModelEndpoint.bedrock)).toBe(262144);
|
||||
expect(getModelMaxTokens('us.anthropic.claude-3-5-sonnet-20241022-v2:0')).toBe(200000);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -87,6 +87,11 @@ const mistralModels = {
|
|||
'mistral-nemo': 131000,
|
||||
'pixtral-large': 131000,
|
||||
codestral: 256000,
|
||||
'mistral-medium-3': 131000,
|
||||
'mistral-medium-3.5': 262144,
|
||||
'mistral-medium-3-5': 262144,
|
||||
'ministral-8b-2512': 262144,
|
||||
'ministral-14b-2512': 262144,
|
||||
};
|
||||
|
||||
const cohereModels = {
|
||||
|
|
@ -97,6 +102,8 @@ const cohereModels = {
|
|||
'command-text': 4086, // -10 from max
|
||||
'command-r': 127500, // -500 from max
|
||||
'command-r-plus': 127500, // -500 from max
|
||||
'command-a': 255500, // -500 from max
|
||||
'command-a-plus': 127500, // -500 from max
|
||||
};
|
||||
|
||||
const googleModels = {
|
||||
|
|
@ -220,6 +227,12 @@ const deepseekModels = {
|
|||
'deepseek.r1': 128000,
|
||||
'deepseek-r1': 128000,
|
||||
'deepseek-reasoner': 128000,
|
||||
'deepseek-v3.1': 163840,
|
||||
'deepseek-v3.2': 163840,
|
||||
'deepseek-v3.2-speciale': 131072,
|
||||
'deepseek-v4': 1048576,
|
||||
'deepseek-v4-flash': 1048576,
|
||||
'deepseek-v4-pro': 1048576,
|
||||
};
|
||||
|
||||
const moonshotModels = {
|
||||
|
|
@ -249,6 +262,9 @@ const moonshotModels = {
|
|||
'kimi-k2-thinking': 262144,
|
||||
'kimi-k2-thinking-turbo': 262144,
|
||||
'kimi-k2.5': 262144,
|
||||
'kimi-k2.6': 262144,
|
||||
'kimi-k2.7': 262144,
|
||||
'kimi-k3': 1048576,
|
||||
// Bedrock moonshot models
|
||||
'moonshot.kimi-k2-0711': 131072,
|
||||
'moonshot.kimi': 262144,
|
||||
|
|
@ -257,6 +273,8 @@ const moonshotModels = {
|
|||
'moonshotai.kimi': 262144,
|
||||
'moonshot.kimi-k2.5': 262144,
|
||||
'moonshotai.kimi-k2.5': 262144,
|
||||
'moonshot.kimi-k3': 1048576,
|
||||
'moonshotai.kimi-k3': 1048576,
|
||||
};
|
||||
|
||||
const metaModels = {
|
||||
|
|
@ -302,13 +320,28 @@ const metaModels = {
|
|||
'llama-3.2-3b': 127500,
|
||||
'llama-3.2-11b': 127500,
|
||||
'llama-3.2-90b': 127500,
|
||||
// Llama 3.3 (newest)
|
||||
// Llama 3.3
|
||||
'llama3.3': 127500,
|
||||
'llama3-3': 127500,
|
||||
'llama-3.3': 127500,
|
||||
'llama3.3:70b': 127500,
|
||||
'llama3-3-70b': 127500,
|
||||
'llama-3.3-70b': 127500,
|
||||
/* Llama 4. Scout's native ceiling is 10M, but no host serves anywhere near it,
|
||||
so capping at 1M keeps pruning useful instead of never firing. */
|
||||
llama4: 1048576,
|
||||
'llama-4': 1048576,
|
||||
'llama4-scout': 1048576,
|
||||
'llama-4-scout': 1048576,
|
||||
'llama4-maverick': 1048576,
|
||||
'llama-4-maverick': 1048576,
|
||||
// Muse (newest)
|
||||
musespark: 1000000,
|
||||
'muse-spark': 1000000,
|
||||
'muse-spark-1.1': 1000000,
|
||||
'muse-spark-1.2': 1000000,
|
||||
'muse-glimmer': 131072,
|
||||
'muse-glimmer-30b': 131072,
|
||||
};
|
||||
|
||||
const qwenModels = {
|
||||
|
|
@ -335,6 +368,21 @@ const qwenModels = {
|
|||
'qwen3-coder-plus': 128000,
|
||||
'qwen3-coder-flash': 128000,
|
||||
'qwen3-next-80b-a3b': 262144,
|
||||
'qwen3-coder-next': 262144,
|
||||
/* Qwen3.5 onward: every open-weight size ships a 262K native window, while the
|
||||
hosted plus/flash tiers serve 1M. */
|
||||
'qwen3.5': 262144,
|
||||
'qwen3.5-plus': 1000000,
|
||||
'qwen3.5-flash': 1000000,
|
||||
'qwen3.6': 262144,
|
||||
'qwen3.6-plus': 1000000,
|
||||
'qwen3.6-flash': 1000000,
|
||||
'qwen3.7': 262144,
|
||||
'qwen3.7-plus': 1000000,
|
||||
'qwen3.7-flash': 1000000,
|
||||
'qwen3.8': 262144,
|
||||
'qwen3.8-plus': 1000000,
|
||||
'qwen3.8-flash': 1000000,
|
||||
};
|
||||
|
||||
const ai21Models = {
|
||||
|
|
@ -354,6 +402,8 @@ const amazonModels = {
|
|||
'nova-lite': 295000, // -5000 from max
|
||||
'nova-pro': 295000, // -5000 from max
|
||||
'nova-premier': 995000, // -5000 from max
|
||||
'nova-2-lite': 995000, // -5000 from max
|
||||
'nova-2-pro': 995000, // -5000 from max
|
||||
};
|
||||
|
||||
const openAIBedrockModels = {
|
||||
|
|
@ -391,6 +441,10 @@ const xAIModels = {
|
|||
'grok-4': 256000, // 256K context
|
||||
'grok-4-fast': 2000000, // 2M context
|
||||
'grok-4-1-fast': 2000000, // 2M context (covers reasoning & non-reasoning variants)
|
||||
'grok-4.5': 500000,
|
||||
'grok-4-5': 500000,
|
||||
'grok-4.6': 500000,
|
||||
'grok-4-6': 500000,
|
||||
};
|
||||
|
||||
const aggregateModels = {
|
||||
|
|
@ -402,6 +456,17 @@ const aggregateModels = {
|
|||
'glm-4.5-air': 131000,
|
||||
'glm-4.5v': 66000,
|
||||
'glm-4.6': 200000,
|
||||
'glm-4.7': 204800,
|
||||
'glm-4.7-flash': 202752,
|
||||
'glm-5': 204800,
|
||||
'glm-5.1': 204800,
|
||||
'glm-5.2': 1048576,
|
||||
'glm-5.3': 1048576,
|
||||
// MiniMax
|
||||
'minimax-m1': 1000000,
|
||||
'minimax-m2': 204800,
|
||||
'minimax-m2.7': 204800,
|
||||
'minimax-m3': 1048576,
|
||||
// GPT-OSS
|
||||
'gpt-oss': 131000,
|
||||
'gpt-oss:20b': 131000,
|
||||
|
|
@ -500,13 +565,15 @@ export const maxOutputTokensMap: Record<string, Record<string, number>> = {
|
|||
[EModelEndpoint.custom]: { ...modelMaxOutputs, ...deepseekMaxOutputs },
|
||||
};
|
||||
|
||||
/** Finds the longest matching key in the tokens map via substring match. */
|
||||
export function findMatchingPattern(
|
||||
modelName: string,
|
||||
tokensMap: Record<string, number> | EndpointTokenConfig,
|
||||
): string | null {
|
||||
/**
|
||||
* Any model-keyed map. Matching reads keys only and never inspects a value, so
|
||||
* the value type is deliberately open rather than a union that every caller has
|
||||
* to cast into.
|
||||
*/
|
||||
type ModelKeyedMap = Readonly<Record<string, unknown>>;
|
||||
|
||||
function findLongestKey(lowerModelName: string, tokensMap: ModelKeyedMap): string | null {
|
||||
const keys = Object.keys(tokensMap);
|
||||
const lowerModelName = modelName.toLowerCase();
|
||||
let bestMatch: string | null = null;
|
||||
let bestLength = 0;
|
||||
for (let i = keys.length - 1; i >= 0; i--) {
|
||||
|
|
@ -524,6 +591,31 @@ export function findMatchingPattern(
|
|||
return bestMatch;
|
||||
}
|
||||
|
||||
/**
|
||||
* Finds the longest matching key in the tokens map via substring match.
|
||||
*
|
||||
* A vendor prefix can outrank the id it precedes: `moonshotai/kimi-k2` matched
|
||||
* `moonshot` (8) over `kimi-k2` (7) and reported half the real window. When the
|
||||
* whole id resolves to a prefix-only match, retry against the model segment.
|
||||
*
|
||||
* A key that carries the vendor itself is already the most specific answer and
|
||||
* is kept. `EndpointTokenConfig` is an arbitrary record, and one built from
|
||||
* OpenRouter is keyed by `org/model`, so those must still beat a bare `model`.
|
||||
*/
|
||||
export function findMatchingPattern(modelName: string, tokensMap: ModelKeyedMap): string | null {
|
||||
const lowerModelName = modelName.toLowerCase();
|
||||
const slashIndex = lowerModelName.lastIndexOf('/');
|
||||
if (slashIndex === -1) {
|
||||
return findLongestKey(lowerModelName, tokensMap);
|
||||
}
|
||||
|
||||
const fullMatch = findLongestKey(lowerModelName, tokensMap);
|
||||
if (fullMatch != null && fullMatch.includes('/')) {
|
||||
return fullMatch;
|
||||
}
|
||||
return findLongestKey(lowerModelName.slice(slashIndex + 1), tokensMap) ?? fullMatch;
|
||||
}
|
||||
|
||||
/**
|
||||
* Retrieves a token value for a given model name from a tokens map.
|
||||
*
|
||||
|
|
|
|||
|
|
@ -3,25 +3,51 @@
|
|||
* These are used only by test files in data-schemas.
|
||||
*/
|
||||
|
||||
function findLongestKey(
|
||||
lowerModelName: string,
|
||||
tokensMap: Record<string, number | Record<string, number>>,
|
||||
): string | undefined {
|
||||
const keys = Object.keys(tokensMap);
|
||||
let bestMatch: string | undefined;
|
||||
let bestLength = 0;
|
||||
for (let i = keys.length - 1; i >= 0; i--) {
|
||||
const key = keys[i];
|
||||
const lowerKey = key.toLowerCase();
|
||||
if (lowerKey.length > bestLength && lowerModelName.includes(lowerKey)) {
|
||||
if (lowerKey.length === lowerModelName.length) {
|
||||
return key;
|
||||
}
|
||||
bestMatch = key;
|
||||
bestLength = lowerKey.length;
|
||||
}
|
||||
}
|
||||
|
||||
return bestMatch;
|
||||
}
|
||||
|
||||
/**
|
||||
* Finds the first matching pattern in a tokens/values map by reverse-iterating
|
||||
* and checking if the model name (lowercased) includes the key.
|
||||
* Finds the longest matching key in a tokens/values map by substring match,
|
||||
* preferring the model segment of a vendor-prefixed id.
|
||||
*
|
||||
* Inlined from @librechat/api findMatchingPattern
|
||||
* Mirrors @librechat/api findMatchingPattern. Keep the two in step: this is the
|
||||
* function production injects into `createTxMethods`, so a divergence here means
|
||||
* the pricing tests stop describing real billing behavior.
|
||||
*/
|
||||
export function findMatchingPattern(
|
||||
modelName: string,
|
||||
tokensMap: Record<string, number | Record<string, number>>,
|
||||
): string | undefined {
|
||||
const keys = Object.keys(tokensMap);
|
||||
const lowerModelName = modelName.toLowerCase();
|
||||
for (let i = keys.length - 1; i >= 0; i--) {
|
||||
const modelKey = keys[i];
|
||||
if (lowerModelName.includes(modelKey)) {
|
||||
return modelKey;
|
||||
}
|
||||
const slashIndex = lowerModelName.lastIndexOf('/');
|
||||
if (slashIndex === -1) {
|
||||
return findLongestKey(lowerModelName, tokensMap);
|
||||
}
|
||||
return undefined;
|
||||
|
||||
const fullMatch = findLongestKey(lowerModelName, tokensMap);
|
||||
if (fullMatch != null && fullMatch.includes('/')) {
|
||||
return fullMatch;
|
||||
}
|
||||
return findLongestKey(lowerModelName.slice(slashIndex + 1), tokensMap) ?? fullMatch;
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -2915,5 +2915,109 @@ describe('Long-Context Premium Cache Pricing', () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe('Newer model pricing', () => {
|
||||
const newlyPriced = [
|
||||
'llama4-scout',
|
||||
'llama-4-maverick',
|
||||
'muse-spark-1.1',
|
||||
'musespark',
|
||||
'muse-glimmer-30b',
|
||||
'deepseek-v4-flash',
|
||||
'deepseek-v4-pro',
|
||||
'kimi-k3',
|
||||
'glm-4.7',
|
||||
'glm-4.7-flash',
|
||||
'glm-5.1',
|
||||
'glm-5.2',
|
||||
'grok-4.5',
|
||||
'grok-4.6',
|
||||
'ministral-8b-2512',
|
||||
'ministral-14b-2512',
|
||||
'nova-2-lite',
|
||||
'command-a',
|
||||
'minimax-m2.7',
|
||||
'minimax-m3',
|
||||
'qwen3-coder-next',
|
||||
];
|
||||
|
||||
it.each(newlyPriced)('bills %s at its own rate rather than the default', (model) => {
|
||||
const valueKey = getValueKey(model);
|
||||
expect(valueKey).toBe(model);
|
||||
expect(getMultiplier({ model, tokenType: 'prompt' })).toBe(tokenValues[model].prompt);
|
||||
expect(getMultiplier({ model, tokenType: 'completion' })).toBe(tokenValues[model].completion);
|
||||
expect(getMultiplier({ model, tokenType: 'prompt' })).not.toBe(defaultRate);
|
||||
});
|
||||
|
||||
it('keeps the preceding generation on its own rate', () => {
|
||||
for (const model of [
|
||||
'grok-4',
|
||||
'grok-4-fast',
|
||||
'ministral-8b',
|
||||
'qwen3-coder',
|
||||
'command-r-plus',
|
||||
]) {
|
||||
expect(getValueKey(model)).toBe(model);
|
||||
expect(getMultiplier({ model, tokenType: 'prompt' })).toBe(tokenValues[model].prompt);
|
||||
}
|
||||
});
|
||||
|
||||
it('resolves dated and vendor-prefixed variants to the same rate', () => {
|
||||
expect(getMultiplier({ model: 'moonshotai/kimi-k3', tokenType: 'completion' })).toBe(
|
||||
tokenValues['kimi-k3'].completion,
|
||||
);
|
||||
expect(getMultiplier({ model: 'x-ai/grok-4-6', tokenType: 'prompt' })).toBe(
|
||||
tokenValues['grok-4-6'].prompt,
|
||||
);
|
||||
expect(getMultiplier({ model: 'command-a-plus-05-2026', tokenType: 'prompt' })).toBe(
|
||||
tokenValues['command-a'].prompt,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('dot-prefixed Kimi K3 aliases', () => {
|
||||
it('bills both bedrock forms at the K3 rate rather than an older Kimi row', () => {
|
||||
for (const model of ['moonshot.kimi-k3', 'moonshotai.kimi-k3']) {
|
||||
expect(getMultiplier({ model, tokenType: 'completion' })).toBe(
|
||||
tokenValues['kimi-k3'].completion,
|
||||
);
|
||||
expect(getMultiplier({ model, tokenType: 'completion' })).not.toBe(
|
||||
tokenValues['moonshot.kimi'].completion,
|
||||
);
|
||||
expect(getMultiplier({ model, tokenType: 'completion' })).not.toBe(
|
||||
tokenValues['moonshot'].completion,
|
||||
);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('Grok long-context premium tier', () => {
|
||||
it('doubles the rate once a prompt passes the 200K threshold', () => {
|
||||
for (const model of ['grok-4.5', 'grok-4.6']) {
|
||||
const { threshold } = premiumTokenValues[model];
|
||||
expect(getMultiplier({ model, tokenType: 'prompt', inputTokenCount: threshold })).toBe(
|
||||
tokenValues[model].prompt,
|
||||
);
|
||||
expect(getMultiplier({ model, tokenType: 'prompt', inputTokenCount: threshold + 1 })).toBe(
|
||||
premiumTokenValues[model].prompt,
|
||||
);
|
||||
expect(
|
||||
getMultiplier({ model, tokenType: 'completion', inputTokenCount: threshold + 1 }),
|
||||
).toBe(premiumTokenValues[model].completion);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('vendor-prefixed pricing keys', () => {
|
||||
it('prices the model segment rather than the vendor prefix', () => {
|
||||
expect(getValueKey('moonshotai/kimi-k2')).toBe('kimi-k2');
|
||||
expect(getMultiplier({ model: 'moonshotai/kimi-k2', tokenType: 'prompt' })).toBe(
|
||||
tokenValues['kimi-k2'].prompt,
|
||||
);
|
||||
expect(getMultiplier({ model: 'moonshotai/kimi-k2', tokenType: 'prompt' })).not.toBe(
|
||||
tokenValues['moonshot'].prompt,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// Cross-package sync validation tests (tokens.ts ↔ tx.ts) moved to
|
||||
// packages/api tests since they require maxTokensMap from @librechat/api.
|
||||
|
|
|
|||
|
|
@ -74,6 +74,14 @@ const bedrockValues: Record<string, { prompt: number; completion: number }> = {
|
|||
'llama-3.2-90b': { prompt: 0.72, completion: 0.72 },
|
||||
'llama-3.3': { prompt: 2.65, completion: 3.5 },
|
||||
'llama-3.3-70b': { prompt: 2.65, completion: 3.5 },
|
||||
/* Llama 4. Per-host rates vary widely (0.08 to 0.22 prompt); these follow the
|
||||
published Bedrock/OpenRouter figures. */
|
||||
llama4: { prompt: 0.1, completion: 0.3 },
|
||||
'llama-4': { prompt: 0.1, completion: 0.3 },
|
||||
'llama4-scout': { prompt: 0.1, completion: 0.3 },
|
||||
'llama-4-scout': { prompt: 0.1, completion: 0.3 },
|
||||
'llama4-maverick': { prompt: 0.2, completion: 0.7 },
|
||||
'llama-4-maverick': { prompt: 0.2, completion: 0.7 },
|
||||
'mistral-7b': { prompt: 0.15, completion: 0.2 },
|
||||
'mistral-small': { prompt: 0.15, completion: 0.2 },
|
||||
'mixtral-8x7b': { prompt: 0.45, completion: 0.7 },
|
||||
|
|
@ -91,11 +99,16 @@ const bedrockValues: Record<string, { prompt: number; completion: number }> = {
|
|||
'nova-lite': { prompt: 0.06, completion: 0.24 },
|
||||
'nova-pro': { prompt: 0.8, completion: 3.2 },
|
||||
'nova-premier': { prompt: 2.5, completion: 12.5 },
|
||||
'nova-2-lite': { prompt: 0.3, completion: 2.5 },
|
||||
'deepseek.r1': { prompt: 1.35, completion: 5.4 },
|
||||
'moonshot.kimi': { prompt: 0.6, completion: 2.5 },
|
||||
'moonshot.kimi-k2': { prompt: 0.6, completion: 2.5 },
|
||||
'moonshot.kimi-k2.5': { prompt: 0.6, completion: 3.0 },
|
||||
'moonshot.kimi-k2-thinking': { prompt: 0.6, completion: 2.5 },
|
||||
/* Both dot-prefixed forms need their own row: longest-substring matching
|
||||
otherwise reaches `moonshot.kimi` or `moonshot` before `kimi-k3`. */
|
||||
'moonshot.kimi-k3': { prompt: 3.0, completion: 15.0 },
|
||||
'moonshotai.kimi-k3': { prompt: 3.0, completion: 15.0 },
|
||||
};
|
||||
|
||||
/**
|
||||
|
|
@ -181,10 +194,13 @@ export const tokenValues: Record<string, { prompt: number; completion: number }>
|
|||
'command-r': { prompt: 0.5, completion: 1.5 },
|
||||
'command-r-plus': { prompt: 3, completion: 15 },
|
||||
'command-text': { prompt: 1.5, completion: 2.0 },
|
||||
'command-a': { prompt: 2.5, completion: 10.0 },
|
||||
'deepseek-chat': { prompt: 0.28, completion: 0.42 },
|
||||
'deepseek-reasoner': { prompt: 0.28, completion: 0.42 },
|
||||
'deepseek-r1': { prompt: 0.4, completion: 2.0 },
|
||||
'deepseek-v3': { prompt: 0.2, completion: 0.8 },
|
||||
'deepseek-v4-flash': { prompt: 0.14, completion: 0.28 },
|
||||
'deepseek-v4-pro': { prompt: 0.435, completion: 0.87 },
|
||||
'gemma-2': { prompt: 0.01, completion: 0.03 },
|
||||
'gemma-3': { prompt: 0.02, completion: 0.04 },
|
||||
'gemma-3-27b': { prompt: 0.09, completion: 0.16 },
|
||||
|
|
@ -225,6 +241,12 @@ export const tokenValues: Record<string, { prompt: number; completion: number }>
|
|||
'grok-4': { prompt: 3.0, completion: 15.0 },
|
||||
'grok-4-fast': { prompt: 0.2, completion: 0.5 },
|
||||
'grok-4-1-fast': { prompt: 0.2, completion: 0.5 },
|
||||
/* Standard tier. The doubled rate above a 200K prompt lives in
|
||||
`premiumTokenValues`, since the 500K window makes it reachable. */
|
||||
'grok-4.5': { prompt: 2.0, completion: 6.0 },
|
||||
'grok-4-5': { prompt: 2.0, completion: 6.0 },
|
||||
'grok-4.6': { prompt: 2.0, completion: 6.0 },
|
||||
'grok-4-6': { prompt: 2.0, completion: 6.0 },
|
||||
'grok-code-fast': { prompt: 0.2, completion: 1.5 },
|
||||
codestral: { prompt: 0.3, completion: 0.9 },
|
||||
devstral: { prompt: 0.4, completion: 2.0 },
|
||||
|
|
@ -233,6 +255,8 @@ export const tokenValues: Record<string, { prompt: number; completion: number }>
|
|||
holo2: { prompt: 0.3, completion: 0.7 },
|
||||
'ministral-3b': { prompt: 0.04, completion: 0.04 },
|
||||
'ministral-8b': { prompt: 0.1, completion: 0.1 },
|
||||
'ministral-8b-2512': { prompt: 0.15, completion: 0.15 },
|
||||
'ministral-14b-2512': { prompt: 0.2, completion: 0.2 },
|
||||
'mistral-nemo': { prompt: 0.15, completion: 0.15 },
|
||||
'mistral-saba': { prompt: 0.2, completion: 0.6 },
|
||||
'pixtral-large': { prompt: 2.0, completion: 6.0 },
|
||||
|
|
@ -251,6 +275,7 @@ export const tokenValues: Record<string, { prompt: number; completion: number }>
|
|||
'kimi-k2-0711-preview': { prompt: 0.6, completion: 2.5 },
|
||||
'kimi-k2-thinking': { prompt: 0.6, completion: 2.5 },
|
||||
'kimi-k2-thinking-turbo': { prompt: 1.15, completion: 8.0 },
|
||||
'kimi-k3': { prompt: 3.0, completion: 15.0 },
|
||||
'moonshot-v1': { prompt: 2.0, completion: 5.0 },
|
||||
'moonshot-v1-auto': { prompt: 2.0, completion: 5.0 },
|
||||
'moonshot-v1-8k': { prompt: 0.2, completion: 2.0 },
|
||||
|
|
@ -273,6 +298,16 @@ export const tokenValues: Record<string, { prompt: number; completion: number }>
|
|||
'glm-4.5-air': { prompt: 0.14, completion: 0.86 },
|
||||
'glm-4.5v': { prompt: 0.6, completion: 1.8 },
|
||||
'glm-4.6': { prompt: 0.5, completion: 1.75 },
|
||||
'glm-4.7': { prompt: 0.6, completion: 2.2 },
|
||||
'glm-4.7-flash': { prompt: 0.06, completion: 0.4 },
|
||||
'glm-5': { prompt: 1.4, completion: 4.4 },
|
||||
'glm-5.1': { prompt: 1.4, completion: 4.4 },
|
||||
'glm-5.2': { prompt: 1.4, completion: 4.4 },
|
||||
'glm-5.3': { prompt: 1.4, completion: 4.4 },
|
||||
'minimax-m1': { prompt: 0.55, completion: 2.2 },
|
||||
'minimax-m2': { prompt: 0.3, completion: 1.2 },
|
||||
'minimax-m2.7': { prompt: 0.3, completion: 1.2 },
|
||||
'minimax-m3': { prompt: 0.3, completion: 1.2 },
|
||||
qwen: { prompt: 0.08, completion: 0.33 },
|
||||
'qwen2.5': { prompt: 0.08, completion: 0.33 },
|
||||
'qwen-turbo': { prompt: 0.05, completion: 0.2 },
|
||||
|
|
@ -295,6 +330,13 @@ export const tokenValues: Record<string, { prompt: number; completion: number }>
|
|||
'qwen3-coder-plus': { prompt: 1, completion: 5 },
|
||||
'qwen3-coder-flash': { prompt: 0.3, completion: 1.5 },
|
||||
'qwen3-next-80b-a3b': { prompt: 0.1, completion: 0.8 },
|
||||
'qwen3-coder-next': { prompt: 0.12, completion: 0.8 },
|
||||
musespark: { prompt: 1.25, completion: 4.25 },
|
||||
'muse-spark': { prompt: 1.25, completion: 4.25 },
|
||||
'muse-spark-1.1': { prompt: 1.25, completion: 4.25 },
|
||||
'muse-spark-1.2': { prompt: 1.25, completion: 4.25 },
|
||||
'muse-glimmer': { prompt: 0.3, completion: 1.2 },
|
||||
'muse-glimmer-30b': { prompt: 0.3, completion: 1.2 },
|
||||
},
|
||||
bedrockValues,
|
||||
);
|
||||
|
|
@ -396,6 +438,10 @@ export const premiumTokenValues: Record<
|
|||
'gpt-5.6': { threshold: 272000, prompt: 10, completion: 45 },
|
||||
'gpt-5.6-terra': { threshold: 272000, prompt: 5, completion: 22.5 },
|
||||
'gpt-5.6-luna': { threshold: 272000, prompt: 2, completion: 9 },
|
||||
'grok-4.5': { threshold: 200000, prompt: 4, completion: 12 },
|
||||
'grok-4-5': { threshold: 200000, prompt: 4, completion: 12 },
|
||||
'grok-4.6': { threshold: 200000, prompt: 4, completion: 12 },
|
||||
'grok-4-6': { threshold: 200000, prompt: 4, completion: 12 },
|
||||
};
|
||||
|
||||
/**
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue