LibreChat/api/server/services/Config/loadCustomConfig.spec.js
Danny Avila 9f617fc4ad
🧬 feat: Re-Derive Azure Groups, Vertex AI, Custom Parameters and Spec Defaults for Config Profiles (#17399)
* 🧬 feat: Re-Derive Azure Groups, Vertex AI, Custom Parameters and Spec Defaults for Config Profiles

AppService derives part of the served config from librechat.yaml at startup, and config
profiles merged into that output without deriving it again. Azure `groups` now merge by `group`
like `endpoints.custom` by `name`: a profile's partial group merges over the group of that
name, groups it does not name stay, indexed writes are refused, change batches take entry ops
on them, and groups that do not resolve over librechat.yaml and Base (or whose written `${VAR}`
is not set) answer `invalid_azure_groups`. Each merged group keeps the key below it unless the
profile supplies one, so the rebinding guard drops a key a partial item moves and a key a lower
layer dropped stays dropped; Azure re-entry compares groups merged the same way.

With `CONFIG_REDERIVE_ENDPOINTS=true` (opt-in for this release), a profile's Azure `groups`
resolve into what the Azure clients read (unconditional before, now behind the switch so stored
group overrides do not go live on upgrade), a profile's `endpoints.anthropic.vertex` derives
`vertexConfig` and the Anthropic models it serves where librechat.yaml configures Vertex AI, and
the Azure Assistants endpoint follows a profile's `assistants`. Startup logs the documents that
hold such overrides, the groups that would not resolve over librechat.yaml and Base, and the
Vertex AI overrides that never apply. Always on: Vertex AI's key file and region are base-only,
a profile may turn Vertex AI on only where librechat.yaml configures it, its settings may
reference only librechat.yaml's own Vertex AI variables (reads return them resolved), a
profile's `vertexConfig` is never applied and writes skip it (`derived_path`,
`rules.derivedPaths`), a model's own `deploymentName` resolves its `${VAR}` like the default one,
the region, project ID and the deployment name each model's requests send (the model name where
the client sends it as written) must stay in their URL host label or path segment, a `${VAR}`
there is a whole `${NAME}` reference only, the Anthropic client refuses any other deployment name
before building a request, and an invalid setup answers `invalid_vertex_config`. The rebinding
guard no longer binds the derived Vertex AI region, which is librechat.yaml's.

Custom endpoint `customParams` are expanded by `expandCustomEndpoint` into the served copy, so
`appConfig.config` stays librechat.yaml as written, and each endpoint a profile writes is
expanded again from its merged form (a changed `defaultParamsEndpoint`, OpenRouter defaults
for a profile's own endpoint); invalid definitions answer `invalid_param_definitions`. After
tenant scoping, `getAppConfig` derives `modelSelect`, `parameters` and `presets` again for the
specs a principal is served where nothing sets them, the agents `memory` capability follows
the merged memory config, and a model spec naming an unconfigured endpoint answers
`unknown_spec_endpoint`. Meta flags: `azureGroupMergeKey`, `customParamsExpansion`,
`specInterfaceRederive` and `endpointRederive`.

* 🪜 fix: Judge Re-Derived Config on What Every Profile Layer Leaves

Two re-derivations decided whether a profile had set a field from whether any layer ever set
it, so a lower profile's value tombstoned by a higher one still counted as set.

- Vertex AI (CONFIG_REDERIVE_ENDPOINTS=true): the marker that keeps a profile's
  `endpoints.anthropic.models` over Vertex AI's is cleared by a later tombstone of those
  models (or of `endpoints.anthropic` or `endpoints`), and such a tombstone derives Vertex AI
  again, so the Anthropic endpoint serves Vertex AI's models instead of none. A tombstone of
  librechat.yaml's own models does the same.
- Spec-dependent controls: `modelSelect`, `parameters` and `presets` are derived again where
  their served value differs from the base config's, not only where the librechat.yaml-shaped
  value does, so a control one profile sets and a higher one tombstones, or a tombstone of one
  librechat.yaml leaves unset, follows the specs served instead of being dropped.

The memory capability, Azure Assistants, Azure groups and custom parameters already derive
from the merged result; tests now cover each set in one layer and tombstoned in a higher one,
and the reverse.

* 🧮 fix: Derive Each Config Part From What the Layers Leave

A tombstone of an object above a derived part's inputs, alone or followed by a layer that
recreates part of that object, left the part without what startup derives it to:

- An Azure endpoint recreated after a tombstone of `endpoints.azureOpenAI` or `endpoints` had
  no groupMap, modelGroupMap or model lists, so its model lists fell back to the environment's
  default Azure models, which requests could not map to a group. It now serves no group, with or
  without CONFIG_REDERIVE_ENDPOINTS (completeAzureResolution).
- A tombstone of the whole `interface` left modelSelect, parameters and presets out, which the
  client reads as off. They are derived from the specs served into a new interface.
- A tombstone of `endpoints.agents` or `endpoints` left no agents endpoint, while a tombstone of
  its capabilities served the defaults. The agents endpoint startup derives is served there.
- A tombstone of a Vertex AI setting that leaves the setup unable to load kept the vertexConfig
  derived below it. Vertex AI is off there, with a warning.

A table in service.spec.ts runs every derived part against every way a layer changes its inputs
(set, unset, a tombstone of the path and of each object above it, a layer recreating after such
a tombstone, Base on either side, tenant scoping, the switch off) through getAppConfig, and
compares each cell with what startup derives from the librechat.yaml the layers leave.

* 🧭 fix: Serve Only Model Specs Whose Endpoint the Layers Leave

A tombstone of `endpoints.custom` or `endpoints` in a higher profile (or the Base profile)
left the model specs of the custom endpoints it removes served, librechat.yaml's or a lower
profile's: the spec list pointed at endpoints /api/endpoints no longer offers, with the model
selector, parameters and presets still off for them. The write check only ran when the write
held `modelSpecs.list`, and the merge served stored specs as they were.

After the last layer, the merge now serves only the specs whose endpoint is a built-in endpoint
or a custom endpoint the YAML-shaped config the layers leave defines, the rule startup skips
librechat.yaml's specs by (deriveServedSpecs). A removed endpoint hides its specs until a layer
defines it again; where no spec is left, none is served (as with a tenant-hidden endpoint) and
the spec-dependent controls come back on. Tombstones of a built-in endpoint's settings keep its
specs, as startup does, and `addedEndpoints` stays as merged. A stored spec for an endpoint that
is not configured is left out on its own instead of being served; writes of one still answer
400 unknown_spec_endpoint.

The derived-part table in service.spec.ts gains the model specs served (with addedEndpoints and
the controls they decide) against tombstones of `endpoints.custom`, `endpoints`, and each of
`endpoints.openAI`, `azureOpenAI` and `anthropic`: by a profile, by Base under a profile, a lower
profile under a higher one, recreated above or by the same layer, with no spec left, and with
the switch off.

* 🧩 fix: Reconcile Re-Derived Config With Dev's Keyed Tombstones, Write Report and Read APIs

- Keyed tombstones of Azure groups (endpoints.azureOpenAI.groups["g"], now merged by key)
  resolve the groups left again where the merge resolves them, and stay inert without
  CONFIG_REDERIVE_ENDPOINTS.
- Keyed tombstones count as touching endpoints.custom, so served specs and custom parameters
  are derived again after one.
- The served-spec derivation keeps modelSpecs while added endpoints remain, as the hidden
  endpoint scoping does; hidden endpoint names leave addedEndpoints.
- Derived paths join the held paths every write skips, built on first use.
- Tests follow the merged behaviour: Azure groups merge by group, the agents endpoint is
  derived again below a tombstone.
2026-10-24 11:46:01 +02:00

602 lines
19 KiB
JavaScript

jest.mock('axios');
jest.mock('~/cache/getLogStores');
jest.mock('@librechat/api', () => ({
...jest.requireActual('@librechat/api'),
loadYaml: jest.fn(),
}));
jest.mock('librechat-data-provider', () => {
const actual = jest.requireActual('librechat-data-provider');
return {
...actual,
paramSettings: {
foo: {},
bar: {},
custom: [],
openrouter: [
{
key: 'promptCache',
type: 'boolean',
component: 'switch',
default: true,
},
],
},
agentParamSettings: {
custom: [],
google: [
{
key: 'pressure',
type: 'string',
component: 'input',
},
{
key: 'temperature',
type: 'number',
component: 'slider',
default: 0.5,
range: {
min: 0,
max: 2,
step: 0.01,
},
},
],
},
};
});
jest.mock('@librechat/data-schemas', () => {
return {
logger: {
info: jest.fn(),
warn: jest.fn(),
debug: jest.fn(),
error: jest.fn(),
},
};
});
const axios = require('axios');
const { loadYaml, expandCustomEndpoint } = require('@librechat/api');
const { logger } = require('@librechat/data-schemas');
const { ReasoningParameterFormat, ReasoningResponseKey } = require('librechat-data-provider');
const loadCustomConfig = require('./loadCustomConfig');
describe('loadCustomConfig', () => {
const originalExit = process.exit;
const mockExit = jest.fn((code) => {
throw new Error(`process.exit called with "${code}"`);
});
beforeAll(() => {
process.exit = mockExit;
});
afterAll(() => {
process.exit = originalExit;
});
beforeEach(() => {
jest.resetAllMocks();
// Re-apply the exit mock implementation after resetAllMocks
mockExit.mockImplementation((code) => {
throw new Error(`process.exit called with "${code}"`);
});
delete process.env.CONFIG_PATH;
});
it('should return null and log error if remote config fetch fails', async () => {
process.env.CONFIG_PATH = 'http://example.com/config.yaml';
axios.get.mockRejectedValue(new Error('Network error'));
const result = await loadCustomConfig();
expect(logger.error).toHaveBeenCalledTimes(1);
expect(result).toBeNull();
});
it('should return null for an invalid local config file', async () => {
process.env.CONFIG_PATH = 'localConfig.yaml';
loadYaml.mockReturnValueOnce(null);
const result = await loadCustomConfig();
expect(result).toBeNull();
});
it('should parse, validate, and cache a valid local configuration', async () => {
const mockConfig = {
version: '1.0',
cache: true,
endpoints: {
custom: [
{
name: 'mistral',
apiKey: 'user_provided',
baseURL: 'https://api.mistral.ai/v1',
},
],
},
};
process.env.CONFIG_PATH = 'validConfig.yaml';
loadYaml.mockReturnValueOnce(mockConfig);
const result = await loadCustomConfig();
expect(result).toEqual(mockConfig);
});
it('should return null and log if config schema validation fails', async () => {
const invalidConfig = { invalidField: true };
process.env.CONFIG_PATH = 'invalidConfig.yaml';
process.env.CONFIG_BYPASS_VALIDATION = 'true';
loadYaml.mockReturnValueOnce(invalidConfig);
const result = await loadCustomConfig();
expect(result).toBeNull();
expect(logger.warn).toHaveBeenCalledWith(
'CONFIG_BYPASS_VALIDATION is enabled. Continuing with default configuration despite validation errors.',
);
delete process.env.CONFIG_BYPASS_VALIDATION;
});
it('should call process.exit(1) when config validation fails without bypass', async () => {
const invalidConfig = { invalidField: true };
process.env.CONFIG_PATH = 'invalidConfig.yaml';
loadYaml.mockReturnValueOnce(invalidConfig);
await expect(loadCustomConfig()).rejects.toThrow('process.exit called with "1"');
expect(logger.error).toHaveBeenCalledWith(
'Exiting due to invalid configuration. Set CONFIG_BYPASS_VALIDATION=true to bypass this check.',
);
});
it('does not exit when config validation fails during reload', async () => {
process.env.CONFIG_PATH = 'invalidConfig.yaml';
loadYaml.mockReturnValueOnce({ invalidField: true });
await expect(loadCustomConfig(true, { mode: 'reload' })).rejects.toMatchObject({
name: 'ConfigReloadError',
validationErrors: expect.any(Array),
});
expect(mockExit).not.toHaveBeenCalled();
});
it('rejects a reload when the local config file is missing', async () => {
process.env.CONFIG_PATH = 'missingConfig.yaml';
loadYaml.mockReturnValueOnce(null);
await expect(loadCustomConfig(true, { mode: 'reload' })).rejects.toMatchObject({
name: 'ConfigReloadError',
});
expect(mockExit).not.toHaveBeenCalled();
});
it('rejects malformed local YAML during reload without exiting', async () => {
process.env.CONFIG_PATH = 'invalidConfig.yaml';
loadYaml.mockReturnValueOnce(new Error('Malformed YAML'));
await expect(loadCustomConfig(true, { mode: 'reload' })).rejects.toMatchObject({
name: 'ConfigReloadError',
});
expect(mockExit).not.toHaveBeenCalled();
});
it('rejects malformed remote YAML during reload without exiting', async () => {
process.env.CONFIG_PATH = 'https://example.com/config.yaml';
axios.get.mockResolvedValueOnce({ data: 'version: [unterminated' });
await expect(loadCustomConfig(true, { mode: 'reload' })).rejects.toMatchObject({
name: 'ConfigReloadError',
});
expect(mockExit).not.toHaveBeenCalled();
});
it('rejects a reload when the remote config fetch fails', async () => {
process.env.CONFIG_PATH = 'https://example.com/config.yaml';
axios.get.mockRejectedValueOnce(new Error('Network error'));
await expect(loadCustomConfig(true, { mode: 'reload' })).rejects.toMatchObject({
name: 'ConfigReloadError',
});
expect(mockExit).not.toHaveBeenCalled();
});
it('should handle and return null on YAML parse error for a string response from remote', async () => {
process.env.CONFIG_PATH = 'http://example.com/config.yaml';
process.env.CONFIG_BYPASS_VALIDATION = 'true';
axios.get.mockResolvedValue({ data: 'invalidYAMLContent' });
const result = await loadCustomConfig();
expect(result).toBeNull();
delete process.env.CONFIG_BYPASS_VALIDATION;
});
it('should return the custom config object for a valid remote config file', async () => {
const mockConfig = {
version: '1.0',
cache: true,
endpoints: {
custom: [
{
name: 'mistral',
apiKey: 'user_provided',
baseURL: 'https://api.mistral.ai/v1',
},
],
},
};
process.env.CONFIG_PATH = 'http://example.com/config.yaml';
axios.get.mockResolvedValue({ data: mockConfig });
const result = await loadCustomConfig();
expect(result).toEqual(mockConfig);
});
it('should return null if the remote config file is not found', async () => {
process.env.CONFIG_PATH = 'http://example.com/config.yaml';
axios.get.mockRejectedValue({ response: { status: 404 } });
const result = await loadCustomConfig();
expect(result).toBeNull();
});
it('should return null if the local config file is not found', async () => {
process.env.CONFIG_PATH = 'nonExistentConfig.yaml';
loadYaml.mockReturnValueOnce(null);
const result = await loadCustomConfig();
expect(result).toBeNull();
});
it('should not cache the config if cache is set to false', async () => {
const mockConfig = {
version: '1.0',
cache: false,
endpoints: {
custom: [
{
name: 'mistral',
apiKey: 'user_provided',
baseURL: 'https://api.mistral.ai/v1',
},
],
},
};
process.env.CONFIG_PATH = 'validConfig.yaml';
loadYaml.mockReturnValueOnce(mockConfig);
const result = await loadCustomConfig();
expect(result).toEqual(mockConfig);
});
it('should log the loaded custom config', async () => {
const mockConfig = {
version: '1.0',
cache: true,
endpoints: {
custom: [
{
name: 'mistral',
apiKey: 'user_provided',
baseURL: 'https://api.mistral.ai/v1',
},
],
},
};
process.env.CONFIG_PATH = 'validConfig.yaml';
loadYaml.mockReturnValueOnce(mockConfig);
await loadCustomConfig();
expect(logger.info).toHaveBeenCalledWith('Custom config file loaded:');
expect(logger.info).toHaveBeenCalledWith(JSON.stringify(mockConfig, null, 2));
expect(logger.debug).toHaveBeenCalledWith('Custom config:', mockConfig);
});
it('masks literal Langfuse header credentials in the startup log', async () => {
const mockConfig = {
version: '1.0',
cache: true,
langfuse: {
enabled: true,
publicKey: 'pk-lf-1',
headers: {
'CF-Access-Client-Id': 'client-id',
'CF-Access-Client-Secret': 'gateway-credential',
},
},
};
process.env.CONFIG_PATH = 'validConfig.yaml';
loadYaml.mockReturnValueOnce(mockConfig);
const result = await loadCustomConfig();
const logged = logger.info.mock.calls.map(([value]) => value).join('\n');
const debugged = JSON.stringify(logger.debug.mock.calls);
expect(logged).not.toContain('gateway-credential');
expect(debugged).not.toContain('gateway-credential');
expect(logged).toContain('***');
// The masking is for logging only — the live config keeps real values.
expect(result.langfuse.headers['CF-Access-Client-Secret']).toBe('gateway-credential');
});
/**
* The loader checks each custom endpoint's `customParams` and returns them as written; the
* served copy is expanded by `expandCustomEndpoint` (AppService), so `appConfig.config` stays
* librechat.yaml as written.
*/
describe('customParams', () => {
const mockConfig = {
version: '1.0',
cache: false,
endpoints: {
custom: [
{
name: 'Google',
apiKey: 'user_provided',
customParams: {},
},
],
},
};
async function loadCustomParams(customParams) {
mockConfig.endpoints.custom[0].customParams = customParams;
loadYaml.mockReturnValue(mockConfig);
return await loadCustomConfig();
}
beforeEach(() => {
jest.resetAllMocks();
process.env.CONFIG_PATH = 'validConfig.yaml';
});
it('returns no error when customParams is undefined', async () => {
const result = await loadCustomParams(undefined);
expect(result).toEqual(mockConfig);
});
it('returns no error when customParams is valid', async () => {
const result = await loadCustomParams({
defaultParamsEndpoint: 'google',
paramDefinitions: [
{
key: 'temperature',
default: 0.5,
},
],
});
expect(result).toEqual(mockConfig);
});
it('throws an error when paramDefinitions contain unsupported keys', async () => {
const malformedCustomParams = {
defaultParamsEndpoint: 'google',
paramDefinitions: [
{ key: 'temperature', default: 0.5 },
{ key: 'unsupportedKey', range: 0.5 },
],
};
await expect(loadCustomParams(malformedCustomParams)).rejects.toThrow(
'paramDefinitions of "Google" endpoint contains invalid key(s). Valid parameter keys are pressure, temperature',
);
});
it('throws an error when paramDefinitions is malformed', async () => {
const malformedCustomParams = {
defaultParamsEndpoint: 'google',
paramDefinitions: [
{
key: 'temperature',
type: 'noomba',
component: 'inpoot',
optionType: 'custom',
},
],
};
await expect(loadCustomParams(malformedCustomParams)).rejects.toThrow(
/Custom parameter definitions for "Google" endpoint is malformed:/,
);
});
it('defaults defaultParamsEndpoint when only reasoningFormat is provided', async () => {
const parsedConfig = await loadCustomParams({
reasoningFormat: ReasoningParameterFormat.reasoningObject,
});
expect(parsedConfig.endpoints.custom[0].customParams).toEqual({
reasoningFormat: ReasoningParameterFormat.reasoningObject,
});
expect(expandCustomEndpoint(parsedConfig.endpoints.custom[0]).customParams).toEqual({
defaultParamsEndpoint: 'custom',
reasoningFormat: ReasoningParameterFormat.reasoningObject,
paramDefinitions: [],
});
});
it('defaults defaultParamsEndpoint when only reasoningKey is provided', async () => {
const parsedConfig = await loadCustomParams({
reasoningKey: ReasoningResponseKey.reasoning,
});
expect(parsedConfig.endpoints.custom[0].customParams).toEqual({
reasoningKey: ReasoningResponseKey.reasoning,
});
expect(expandCustomEndpoint(parsedConfig.endpoints.custom[0]).customParams).toEqual({
defaultParamsEndpoint: 'custom',
reasoningKey: ReasoningResponseKey.reasoning,
paramDefinitions: [],
});
});
it('fills the paramDefinitions with missing values', async () => {
const customParams = {
defaultParamsEndpoint: 'google',
paramDefinitions: [
{ key: 'temperature', default: 0.7, range: { min: 0.1, max: 0.9, step: 0.1 } },
{ key: 'pressure', component: 'textarea' },
],
};
const written = structuredClone(customParams);
const parsedConfig = await loadCustomParams(customParams);
expect(parsedConfig.endpoints.custom[0].customParams).toEqual(written);
const paramDefinitions = expandCustomEndpoint(parsedConfig.endpoints.custom[0]).customParams
.paramDefinitions;
expect(paramDefinitions).toEqual([
{
columnSpan: 1,
component: 'slider',
default: 0.7, // overridden
includeInput: true,
key: 'temperature',
label: 'temperature',
optionType: 'custom',
range: {
// overridden
max: 0.9,
min: 0.1,
step: 0.1,
},
type: 'number',
},
{
columnSpan: 1,
component: 'textarea', // overridden
key: 'pressure',
label: 'pressure',
optionType: 'custom',
placeholder: '',
type: 'string',
},
]);
});
it('adds OpenRouter promptCache defaults when custom endpoint name is OpenRouter', async () => {
const openRouterConfig = {
version: '1.0',
cache: false,
endpoints: {
custom: [
{
name: 'OpenRouter',
apiKey: 'user_provided',
baseURL: 'https://proxy.example.com/v1',
models: {
default: ['anthropic/claude-sonnet-4.6'],
},
},
],
},
};
loadYaml.mockReturnValue(openRouterConfig);
const parsedConfig = await loadCustomConfig();
expect(parsedConfig.endpoints.custom[0].customParams).toBeUndefined();
expect(expandCustomEndpoint(parsedConfig.endpoints.custom[0]).customParams).toEqual({
defaultParamsEndpoint: 'openrouter',
paramDefinitions: [
{
columnSpan: 1,
component: 'switch',
default: true,
key: 'promptCache',
label: 'promptCache',
optionType: 'custom',
type: 'boolean',
},
],
});
});
it('adds OpenRouter promptCache defaults when custom endpoint URL is OpenRouter', async () => {
const openRouterConfig = {
version: '1.0',
cache: false,
endpoints: {
custom: [
{
name: 'Company Gateway',
apiKey: 'user_provided',
baseURL: 'https://openrouter.ai/api/v1',
models: {
default: ['anthropic/claude-sonnet-4.6'],
},
},
],
},
};
loadYaml.mockReturnValue(openRouterConfig);
const parsedConfig = await loadCustomConfig();
expect(parsedConfig.endpoints.custom[0].customParams).toBeUndefined();
expect(expandCustomEndpoint(parsedConfig.endpoints.custom[0]).customParams).toMatchObject({
defaultParamsEndpoint: 'openrouter',
paramDefinitions: [
{
default: true,
key: 'promptCache',
},
],
});
});
it('treats an empty defaultParamsEndpoint as absent for OpenRouter', async () => {
const config = {
version: '1.0',
endpoints: {
custom: [
{
name: 'OpenRouter',
apiKey: 'user_provided',
baseURL: 'https://openrouter.ai/api/v1',
models: { default: ['test-model'] },
customParams: { defaultParamsEndpoint: '' },
},
],
},
};
loadYaml.mockReturnValueOnce(config);
const loaded = await loadCustomConfig();
expect(loaded.endpoints.custom[0].customParams.defaultParamsEndpoint).toBe('');
expect(
expandCustomEndpoint(loaded.endpoints.custom[0]).customParams.defaultParamsEndpoint,
).toBe('openrouter');
});
it('preserves explicit OpenRouter promptCache defaults', async () => {
const openRouterConfig = {
version: '1.0',
cache: false,
endpoints: {
custom: [
{
name: 'OpenRouter',
apiKey: 'user_provided',
baseURL: 'https://openrouter.ai/api/v1',
models: {
default: ['anthropic/claude-sonnet-4.6'],
},
customParams: {
defaultParamsEndpoint: 'openrouter',
paramDefinitions: [{ key: 'promptCache', default: false }],
},
},
],
},
};
loadYaml.mockReturnValue(openRouterConfig);
const parsedConfig = await loadCustomConfig();
expect(parsedConfig.endpoints.custom[0].customParams.paramDefinitions).toEqual([
{ key: 'promptCache', default: false },
]);
expect(
expandCustomEndpoint(parsedConfig.endpoints.custom[0]).customParams.paramDefinitions,
).toEqual([
{
columnSpan: 1,
component: 'switch',
default: false,
key: 'promptCache',
label: 'promptCache',
optionType: 'custom',
type: 'boolean',
},
]);
});
});
});