LibreChat/packages/api/src/app/AppService.spec.ts
Danny Avila 397ddc5366
🧠 feat: Add Memory as an Agent Capability with Inline Tools and Ephemeral Badge (#13869)
* 🧠 feat: Memory Agent Capability with Inline Tools and Ephemeral Badge

Add `AgentCapabilities.memory`, which expands into the inline set_memory/delete_memory tool pair (mirroring the execute_code expansion via registerMemoryTools) when a run-level memoryAvailable gate holds: capability enabled, memory configured, MEMORIES.USE permission, and personalization not opted out. Surfaces the memory artifact as an attachment in the agents tool-end callback.

Adds the ephemeral path (TEphemeralAgent.memory, load/added agent tool injection), a fully-gated memory badge plus tools-dropdown entry, the agent-builder Memory toggle with form round-trip, and a mock e2e test asserting the badge reaches the request payload. Additive to and independent of the existing post-turn memory extraction agent.

* 🩹 fix: Address Codex review on memory capability (gating, validKeys, usage guard)

- Strip the memory capability from the served agents capabilities when memory is not configured/enabled, so the badge, tools dropdown, agent-builder toggle, and backend capability gate stay consistent instead of exposing an inert toggle on default installs (where MEMORIES.USE defaults true).
- Surface configured memory.validKeys in the inline tool definitions so the model is told the allowed keys up front, matching the runtime createMemoryTool schema.
- Append a strict explicit-request usage guard to the agent instructions when inline memory tools are registered, preserving the memory-agent's privacy behavior.
- Add AppService tests covering memory-capability stripping.

*  test: Update AppService capability snapshots for memory strip

AppService now strips the memory capability from the served agents defaults when no memory block is configured; update the spec's expected capability lists to defaultAgentCapabilitiesWithoutMemory for the no-memory-config cases.

* 🛡️ fix: Address Codex re-review on memory capability (round 2)

- Strip the memory capability from the FINAL served agents config, not just defaults; loadEndpoints reparses any endpoints.agents block, so memory was still exposed in that common shape (packages/data-schemas/src/app/service.ts) + regression test.
- Re-check the full memory gate (config, opt-out, MEMORIES.USE) inside handleTools before constructing set_memory/delete_memory, so an unsolicited tool call from a model/custom endpoint can't bypass the runtime gates (api/app/clients/tools/util/handleTools.js).
- Restore the persisted memory toggle for model-spec conversations via applyModelSpecEphemeralAgent (client/src/utils/endpoints.ts).
- Clear LAST_MEMORY_TOGGLE_ on logout and clear-all-chats so a stale memory preference can't leak across users on a shared browser (client/src/utils/localStorage.ts).

* 🧠 fix: Address Codex re-review on memory capability (round 3)

- Serialize set_memory writes and advance a running token total inside createMemoryTool, so parallel batched calls in one event-driven turn can't each pass the limit check against a stale total and collectively exceed memory.tokenLimit (packages/api/src/agents/memory.ts) + tests.
- Inject the keyed memory context (withKeys) instead of withoutKeys when the running agent has the inline memory capability, so delete_memory has a visible key to target (api/server/controllers/agents/client.js).

* 🔐 fix: Address Codex re-review on memory capability (round 4)

- Detect inline memory by tool NAME (set_memory/delete_memory) across an initialized agent's tools + toolDefinitions, since the 'memory' marker is expanded at init and the prior string check never matched; inject the keyed memory context for any primary OR sub-agent that carries the inline memory tools (api/server/controllers/agents/client.js).
- Enforce memory WRITE permissions in the inline tool gate: set_memory requires CREATE+UPDATE and delete_memory requires UPDATE (matching the REST memory routes), so a USE-only role can't mutate/delete memories via agent tool calls (api/app/clients/tools/util/handleTools.js).

* 🔒 fix: Address Codex re-review on memory capability (round 5)

- Gate inline memory registration (memoryAvailable) on the memory WRITE permissions (USE+CREATE+UPDATE), so a read-only-memory role no longer has set_memory/delete_memory shown to the model only for the runtime loader to refuse them (api/server/services/Endpoints/agents/initialize.js).
- Enforce the per-agent memory opt-in at execution: handleTools now refuses to construct set_memory/delete_memory unless the agent actually declared them (toolDefinitions/tools), blocking hallucinated/undeclared memory tool calls from mutating memory.
- Fail closed when getFormattedMemories errors with a configured tokenLimit, instead of writing as if storage were empty and bypassing the cap (api/app/clients/tools/util/handleTools.js).

* 🩹 fix: Address Codex re-review on memory capability (round 6)

- Fix a P1 regression from the prior round: the execution-context agent keeps the raw 'memory' capability marker (not the expanded set_memory/delete_memory names), so the opt-in check now matches the marker. This restores memory writes/deletes AND avoids hijacking an MCP tool that merely shares the set_memory/delete_memory name (api/app/clients/tools/util/handleTools.js).
- Count repeated set_memory writes to the same key as replacements, not additions, against tokenLimit — set_memory upserts, so a same-key rewrite swaps its prior token contribution instead of double-counting (packages/api/src/agents/memory.ts) + test.
- Gate the memory badge, tools dropdown, and agent-builder toggle on the full memory write permissions (USE+CREATE+UPDATE) via a shared useHasMemoryAccess hook, so a read-only-memory role no longer sees an enabled Memory control the backend would refuse to wire up.

* 🧷 fix: Address Codex re-review on memory capability (round 7)

- Recognize inline memory across both execution-context agent shapes: initializeAgent now sets a LibreChat-only memoryToolsRegistered flag on the InitializedAgent, and the opt-in/detection checks accept that flag OR the raw 'memory' marker. Fixes memory failing for processAddedConvo agents (which store the initialized config, marker already expanded) while staying MCP-name-collision-safe (api/app/clients/tools/util/handleTools.js, packages/api/src/agents/initialize.ts, api/server/controllers/agents/client.js).
- Scope keyed memory context to memory-enabled agents only: useMemory now returns both keyed and unkeyed contexts, and buildMessages injects the keyed one (memory keys + token metadata) only to agents that can call delete_memory, while the primary/post-turn path keeps the unkeyed values — so a primary without memory tools no longer sees memory keys it doesn't need.

* 🔏 fix: Address Codex re-review on memory capability (round 8)

- Enforce memory size limits on inline writes: createMemoryTool now rejects keys over 1000 chars and values over memory.charLimit, matching the REST memory routes, so an inline-memory agent can't persist blobs the memory UI/API would reject (packages/api/src/agents/memory.ts, api/app/clients/tools/util/handleTools.js) + test.
- Recheck the agents 'memory' endpoint capability at execution time, so a stale/hallucinated set_memory/delete_memory call can't mutate memory after an admin removes the capability while the agent document still carries the marker (api/app/clients/tools/util/handleTools.js).

* ♻️ refactor: Move inline-memory backend logic into packages/api + share memory load

Workspace boundary: the inline-memory gating/detection logic that had crept into /api now lives in packages/api/src/agents/memory.ts (TS), with /api kept as thin wrappers.

- Add agentHasInlineMemoryTools, isMemoryToolAllowed, and buildInlineMemoryTool to packages/api; handleTools.js now calls buildInlineMemoryTool instead of constructing/gating the tools inline, and client.js imports agentHasInlineMemoryTools instead of redefining it.
- Optimize repeated memory loads: getRequestMemories memoizes getFormattedMemories per request (WeakMap keyed by req), so the run's memory-context load and every memory-enabled agent's set_memory token-usage load share a single DB fetch instead of one per agent.

* 🧠 fix: Invalidate request memory cache after inline writes

Inline set_memory/delete_memory now invalidate the request-scoped
getFormattedMemories cache on a successful write, so a later tool round
in the same response is seeded with the post-write usage total instead
of the stale pre-write one (multi-round writes no longer collectively
exceed tokenLimit, and a set after a delete is not over-counted). The
within-round sharing across multiple memory-enabled agents is preserved.

* 🧠 fix: Persist memory capability on saved agents; honor registration flag

- Add Tools.memory to the v1 systemTools allowlist so filterAuthorizedTools
  no longer silently drops the memory marker when an agent with the Memory
  capability is created/updated/duplicated through the builder (previously
  the capability only worked for ephemeral chats, not persisted agents).
- agentHasInlineMemoryTools now honors an explicit memoryToolsRegistered
  boolean before falling back to the raw `memory` marker, so an initialized
  config whose registration was denied (memoryAvailable false) is not given
  keyed memory context just because the marker survives in tools.

* 🧩 fix: Bring memory tool to parity with other ephemeral tools

- Add `memory` to the model-spec schema/type and honor `modelSpec.memory`
  in both ephemeral paths (load.ts, added.ts) and the frontend spec
  application, so admins can pre-enable Memory from a model spec exactly
  like webSearch/fileSearch/executeCode.
- Add LAST_MEMORY_TOGGLE_ to the timestamped-storage cleanup list so stale
  per-conversation memory toggles are purged on startup like the others.
- Hide the agent-builder Memory toggle for users who disabled memory in
  personalization (memories === false), mirroring the chat badge's opt-out
  gate, so the setting isn't shown as inert/misleading.

*  test: Cover memory in applyModelSpecEphemeralAgent spec defaults

Update the exact-object assertions to include the new `memory` field and
add positive coverage that `modelSpec.memory` maps to the ephemeral
agent's `memory` flag. Fixes the shard 2/4 failure from 672a03b05.
2026-06-24 17:14:13 -04:00

929 lines
29 KiB
TypeScript

import {
OCRStrategy,
FileSources,
EModelEndpoint,
EImageOutputType,
AgentCapabilities,
defaultSocialLogins,
validateAzureGroups,
defaultAgentCapabilities,
} from 'librechat-data-provider';
import type { TCustomConfig } from 'librechat-data-provider';
import type { FunctionTool } from '@librechat/data-schemas';
jest.mock('@librechat/data-schemas', () => ({
...jest.requireActual('@librechat/data-schemas'),
logger: {
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn(),
},
}));
import { AppService } from '@librechat/data-schemas';
const azureGroups = [
{
group: 'librechat-westus',
apiKey: '${WESTUS_API_KEY}',
instanceName: 'librechat-westus',
version: '2023-12-01-preview',
models: {
'gpt-4-vision-preview': {
deploymentName: 'gpt-4-vision-preview',
version: '2024-02-15-preview',
},
'gpt-3.5-turbo': {
deploymentName: 'gpt-35-turbo',
},
'gpt-3.5-turbo-1106': {
deploymentName: 'gpt-35-turbo-1106',
},
'gpt-4': {
deploymentName: 'gpt-4',
},
'gpt-4-1106-preview': {
deploymentName: 'gpt-4-1106-preview',
},
},
},
{
group: 'librechat-eastus',
apiKey: '${EASTUS_API_KEY}',
instanceName: 'librechat-eastus',
deploymentName: 'gpt-4-turbo',
version: '2024-02-15-preview',
models: {
'gpt-4-turbo': true,
},
} as const,
];
/** Default agent capabilities served when no `memory` block is configured —
* `AppService` strips `memory` from the defaults since the capability is inert
* without a memory config. */
const defaultAgentCapabilitiesWithoutMemory = defaultAgentCapabilities.filter(
(capability) => capability !== AgentCapabilities.memory,
);
describe('AppService', () => {
const mockSystemTools: Record<string, FunctionTool> = {
ExampleTool: {
type: 'function',
function: {
description: 'Example tool function',
name: 'exampleFunction',
parameters: {
type: 'object',
properties: {
param1: { type: 'string', description: 'An example parameter' },
},
required: ['param1'],
},
},
},
};
beforeEach(() => {
process.env.CDN_PROVIDER = undefined;
jest.clearAllMocks();
});
it('should correctly assign process.env and initialize app config based on custom config', async () => {
const config: Partial<TCustomConfig> = {
registration: { socialLogins: ['testLogin'] },
fileStrategy: FileSources.s3,
balance: {
enabled: true,
},
};
const result = await AppService({ config, systemTools: mockSystemTools });
expect(process.env.CDN_PROVIDER).toEqual('s3');
expect(result).toEqual(
expect.objectContaining({
config: expect.objectContaining({
fileStrategy: 's3',
}),
registration: expect.objectContaining({
socialLogins: ['testLogin'],
}),
fileStrategy: 's3',
interfaceConfig: expect.objectContaining({
modelSelect: true,
parameters: true,
presets: true,
}),
mcpConfig: null,
imageOutputType: expect.any(String),
fileConfig: undefined,
secureImageLinks: undefined,
balance: { enabled: true },
filteredTools: undefined,
includedTools: undefined,
webSearch: expect.objectContaining({
safeSearch: 1,
jinaApiKey: '${JINA_API_KEY}',
jinaApiUrl: '${JINA_API_URL}',
cohereApiKey: '${COHERE_API_KEY}',
serperApiKey: '${SERPER_API_KEY}',
searxngApiKey: '${SEARXNG_API_KEY}',
firecrawlApiKey: '${FIRECRAWL_API_KEY}',
firecrawlApiUrl: '${FIRECRAWL_API_URL}',
searxngInstanceUrl: '${SEARXNG_INSTANCE_URL}',
}),
memory: undefined,
endpoints: expect.objectContaining({
agents: expect.objectContaining({
disableBuilder: false,
capabilities: expect.arrayContaining([...defaultAgentCapabilitiesWithoutMemory]),
maxCitations: 30,
maxCitationsPerFile: 7,
minRelevanceScore: 0.45,
}),
}),
}),
);
});
it('should change the `imageOutputType` based on config value', async () => {
const config = {
version: '0.10.0',
imageOutputType: EImageOutputType.WEBP,
};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
imageOutputType: EImageOutputType.WEBP,
}),
);
});
it('should default to `PNG` `imageOutputType` with no provided type', async () => {
const config = {
version: '0.10.0',
};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
imageOutputType: EImageOutputType.PNG,
}),
);
});
it('should default to `PNG` `imageOutputType` with no provided config', async () => {
const config = {};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
imageOutputType: EImageOutputType.PNG,
}),
);
});
it('should enable summarization when it is configured without enabled flag', async () => {
const config = {
summarization: {
prompt: 'Summarize with emphasis on next actions',
},
} as Partial<TCustomConfig> & { summarization: Record<string, unknown> };
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
summarization: expect.objectContaining({
enabled: true,
prompt: 'Summarize with emphasis on next actions',
}),
}),
);
});
it('should preserve explicit summarization disable flag', async () => {
const config = {
summarization: {
enabled: false,
prompt: 'Ignored while disabled',
},
} as Partial<TCustomConfig> & { summarization: Record<string, unknown> };
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
summarization: expect.objectContaining({
enabled: false,
prompt: 'Ignored while disabled',
}),
}),
);
});
it('should load and format tools accurately with defined structure', async () => {
const config = {};
const result = await AppService({ config, systemTools: mockSystemTools });
// Verify tools are included in the returned config
expect(result.availableTools).toBeDefined();
expect(result.availableTools?.ExampleTool).toEqual({
type: 'function',
function: {
description: 'Example tool function',
name: 'exampleFunction',
parameters: {
type: 'object',
properties: {
param1: { type: 'string', description: 'An example parameter' },
},
required: ['param1'],
},
},
});
});
it('should correctly configure Assistants endpoint based on custom config', async () => {
const config: Partial<TCustomConfig> = {
endpoints: {
[EModelEndpoint.assistants]: {
disableBuilder: true,
pollIntervalMs: 5000,
timeoutMs: 30000,
supportedIds: ['id1', 'id2'],
privateAssistants: false,
},
},
};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
endpoints: expect.objectContaining({
[EModelEndpoint.assistants]: expect.objectContaining({
disableBuilder: true,
pollIntervalMs: 5000,
timeoutMs: 30000,
supportedIds: expect.arrayContaining(['id1', 'id2']),
privateAssistants: false,
}),
}),
}),
);
});
it('should correctly configure Agents endpoint based on custom config', async () => {
const config: Partial<TCustomConfig> = {
endpoints: {
[EModelEndpoint.agents]: {
disableBuilder: true,
recursionLimit: 10,
maxRecursionLimit: 20,
allowedProviders: ['openai', 'anthropic'],
capabilities: [AgentCapabilities.tools, AgentCapabilities.actions],
},
},
};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
endpoints: expect.objectContaining({
[EModelEndpoint.agents]: expect.objectContaining({
disableBuilder: true,
recursionLimit: 10,
maxRecursionLimit: 20,
allowedProviders: expect.arrayContaining(['openai', 'anthropic']),
capabilities: expect.arrayContaining([
AgentCapabilities.tools,
AgentCapabilities.actions,
]),
}),
}),
}),
);
});
it('should configure Agents endpoint with defaults when no config is provided', async () => {
const config = {};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
endpoints: expect.objectContaining({
[EModelEndpoint.agents]: expect.objectContaining({
disableBuilder: false,
capabilities: expect.arrayContaining([...defaultAgentCapabilitiesWithoutMemory]),
}),
}),
}),
);
});
it('should configure Agents endpoint with defaults when endpoints exist but agents is not defined', async () => {
const config = {
endpoints: {
[EModelEndpoint.openAI]: {
titleConvo: true,
},
},
};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
endpoints: expect.objectContaining({
[EModelEndpoint.agents]: expect.objectContaining({
disableBuilder: false,
capabilities: expect.arrayContaining([...defaultAgentCapabilitiesWithoutMemory]),
}),
[EModelEndpoint.openAI]: expect.objectContaining({
titleConvo: true,
}),
}),
}),
);
});
it('should correctly configure minimum Azure OpenAI Assistant values', async () => {
const assistantGroups = [azureGroups[0], { ...azureGroups[1], assistants: true }];
const config = {
endpoints: {
[EModelEndpoint.azureOpenAI]: {
groups: assistantGroups,
assistants: true,
},
},
};
process.env.WESTUS_API_KEY = 'westus-key';
process.env.EASTUS_API_KEY = 'eastus-key';
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
endpoints: expect.objectContaining({
[EModelEndpoint.azureAssistants]: expect.objectContaining({
capabilities: expect.arrayContaining([
expect.any(String),
expect.any(String),
expect.any(String),
]),
}),
}),
}),
);
});
it('should correctly configure Azure OpenAI endpoint based on custom config', async () => {
const config: Partial<TCustomConfig> = {
endpoints: {
[EModelEndpoint.azureOpenAI]: {
groups: azureGroups,
},
},
};
process.env.WESTUS_API_KEY = 'westus-key';
process.env.EASTUS_API_KEY = 'eastus-key';
const result = await AppService({ config });
const { modelNames, modelGroupMap, groupMap } = validateAzureGroups(azureGroups);
expect(result).toEqual(
expect.objectContaining({
endpoints: expect.objectContaining({
[EModelEndpoint.azureOpenAI]: expect.objectContaining({
modelNames,
modelGroupMap,
groupMap,
}),
}),
}),
);
});
it('should not modify FILE_UPLOAD environment variables without rate limits', async () => {
// Setup initial environment variables
process.env.FILE_UPLOAD_IP_MAX = '10';
process.env.FILE_UPLOAD_IP_WINDOW = '15';
process.env.FILE_UPLOAD_USER_MAX = '5';
process.env.FILE_UPLOAD_USER_WINDOW = '20';
const initialEnv = { ...process.env };
const config = {};
await AppService({ config });
// Expect environment variables to remain unchanged
expect(process.env.FILE_UPLOAD_IP_MAX).toEqual(initialEnv.FILE_UPLOAD_IP_MAX);
expect(process.env.FILE_UPLOAD_IP_WINDOW).toEqual(initialEnv.FILE_UPLOAD_IP_WINDOW);
expect(process.env.FILE_UPLOAD_USER_MAX).toEqual(initialEnv.FILE_UPLOAD_USER_MAX);
expect(process.env.FILE_UPLOAD_USER_WINDOW).toEqual(initialEnv.FILE_UPLOAD_USER_WINDOW);
});
it('should fallback to default FILE_UPLOAD environment variables when rate limits are unspecified', async () => {
// Setup initial environment variables to non-default values
process.env.FILE_UPLOAD_IP_MAX = 'initialMax';
process.env.FILE_UPLOAD_IP_WINDOW = 'initialWindow';
process.env.FILE_UPLOAD_USER_MAX = 'initialUserMax';
process.env.FILE_UPLOAD_USER_WINDOW = 'initialUserWindow';
const config = {};
await AppService({ config });
// Verify that process.env falls back to the initial values
expect(process.env.FILE_UPLOAD_IP_MAX).toEqual('initialMax');
expect(process.env.FILE_UPLOAD_IP_WINDOW).toEqual('initialWindow');
expect(process.env.FILE_UPLOAD_USER_MAX).toEqual('initialUserMax');
expect(process.env.FILE_UPLOAD_USER_WINDOW).toEqual('initialUserWindow');
});
it('should not modify IMPORT environment variables without rate limits', async () => {
// Setup initial environment variables
process.env.IMPORT_IP_MAX = '10';
process.env.IMPORT_IP_WINDOW = '15';
process.env.IMPORT_USER_MAX = '5';
process.env.IMPORT_USER_WINDOW = '20';
const initialEnv = { ...process.env };
const config = {};
await AppService({ config });
// Expect environment variables to remain unchanged
expect(process.env.IMPORT_IP_MAX).toEqual(initialEnv.IMPORT_IP_MAX);
expect(process.env.IMPORT_IP_WINDOW).toEqual(initialEnv.IMPORT_IP_WINDOW);
expect(process.env.IMPORT_USER_MAX).toEqual(initialEnv.IMPORT_USER_MAX);
expect(process.env.IMPORT_USER_WINDOW).toEqual(initialEnv.IMPORT_USER_WINDOW);
});
it('should fallback to default IMPORT environment variables when rate limits are unspecified', async () => {
// Setup initial environment variables to non-default values
process.env.IMPORT_IP_MAX = 'initialMax';
process.env.IMPORT_IP_WINDOW = 'initialWindow';
process.env.IMPORT_USER_MAX = 'initialUserMax';
process.env.IMPORT_USER_WINDOW = 'initialUserWindow';
const config = {};
await AppService({ config });
// Verify that process.env falls back to the initial values
expect(process.env.IMPORT_IP_MAX).toEqual('initialMax');
expect(process.env.IMPORT_IP_WINDOW).toEqual('initialWindow');
expect(process.env.IMPORT_USER_MAX).toEqual('initialUserMax');
expect(process.env.IMPORT_USER_WINDOW).toEqual('initialUserWindow');
});
it('should correctly configure endpoint with titlePrompt, titleMethod, and titlePromptTemplate', async () => {
const config: Partial<TCustomConfig> = {
endpoints: {
[EModelEndpoint.openAI]: {
titleConvo: true,
titleModel: 'gpt-3.5-turbo',
titleMethod: 'structured',
titlePrompt: 'Custom title prompt for conversation',
titlePromptTemplate: 'Summarize this conversation: {{conversation}}',
},
[EModelEndpoint.assistants]: {
titleMethod: 'functions',
titlePrompt: 'Generate a title for this assistant conversation',
titlePromptTemplate: 'Assistant conversation template: {{messages}}',
},
[EModelEndpoint.azureOpenAI]: {
groups: azureGroups,
titleConvo: true,
titleMethod: 'completion',
titleModel: 'gpt-4',
titlePrompt: 'Azure title prompt',
titlePromptTemplate: 'Azure conversation: {{context}}',
},
},
};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
endpoints: expect.objectContaining({
// Check OpenAI endpoint configuration
[EModelEndpoint.openAI]: expect.objectContaining({
titleConvo: true,
titleModel: 'gpt-3.5-turbo',
titleMethod: 'structured',
titlePrompt: 'Custom title prompt for conversation',
titlePromptTemplate: 'Summarize this conversation: {{conversation}}',
}),
// Check Assistants endpoint configuration
[EModelEndpoint.assistants]: expect.objectContaining({
titleMethod: 'functions',
titlePrompt: 'Generate a title for this assistant conversation',
titlePromptTemplate: 'Assistant conversation template: {{messages}}',
}),
// Check Azure OpenAI endpoint configuration
[EModelEndpoint.azureOpenAI]: expect.objectContaining({
titleConvo: true,
titleMethod: 'completion',
titleModel: 'gpt-4',
titlePrompt: 'Azure title prompt',
titlePromptTemplate: 'Azure conversation: {{context}}',
}),
}),
}),
);
});
it('should configure Agent endpoint with title generation settings', async () => {
const config: Partial<TCustomConfig> = {
endpoints: {
[EModelEndpoint.agents]: {
disableBuilder: false,
titleConvo: true,
titleModel: 'gpt-4',
titleMethod: 'structured',
titlePrompt: 'Generate a descriptive title for this agent conversation',
titlePromptTemplate: 'Agent conversation summary: {{content}}',
recursionLimit: 15,
capabilities: [AgentCapabilities.tools, AgentCapabilities.actions],
maxCitations: 30,
maxCitationsPerFile: 7,
minRelevanceScore: 0.45,
},
},
};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
endpoints: expect.objectContaining({
[EModelEndpoint.agents]: expect.objectContaining({
disableBuilder: false,
titleConvo: true,
titleModel: 'gpt-4',
titleMethod: 'structured',
titlePrompt: 'Generate a descriptive title for this agent conversation',
titlePromptTemplate: 'Agent conversation summary: {{content}}',
recursionLimit: 15,
capabilities: expect.arrayContaining([
AgentCapabilities.tools,
AgentCapabilities.actions,
]),
}),
}),
}),
);
});
it('should handle missing title configuration options with defaults', async () => {
const config = {
endpoints: {
[EModelEndpoint.openAI]: {
titleConvo: true,
// titlePrompt and titlePromptTemplate are not provided
},
},
};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
endpoints: expect.objectContaining({
[EModelEndpoint.openAI]: expect.objectContaining({
titleConvo: true,
}),
}),
}),
);
// Verify that optional fields are not set when not provided
expect(result.endpoints![EModelEndpoint.openAI]!.titlePrompt).toBeUndefined();
expect(result.endpoints![EModelEndpoint.openAI]!.titlePromptTemplate).toBeUndefined();
expect(result.endpoints![EModelEndpoint.openAI]!.titleMethod).toBeUndefined();
});
it('should correctly configure titleEndpoint when specified', async () => {
const config: Partial<TCustomConfig> = {
endpoints: {
[EModelEndpoint.openAI]: {
titleConvo: true,
titleModel: 'gpt-3.5-turbo',
titleEndpoint: EModelEndpoint.anthropic,
titlePrompt: 'Generate a concise title',
},
[EModelEndpoint.agents]: {
disableBuilder: false,
capabilities: [AgentCapabilities.tools],
maxCitations: 30,
maxCitationsPerFile: 7,
minRelevanceScore: 0.45,
titleEndpoint: 'custom-provider',
titleMethod: 'structured',
},
},
};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
endpoints: expect.objectContaining({
// Check OpenAI endpoint has titleEndpoint
[EModelEndpoint.openAI]: expect.objectContaining({
titleConvo: true,
titleModel: 'gpt-3.5-turbo',
titleEndpoint: EModelEndpoint.anthropic,
titlePrompt: 'Generate a concise title',
}),
// Check Agents endpoint has titleEndpoint
[EModelEndpoint.agents]: expect.objectContaining({
titleEndpoint: 'custom-provider',
titleMethod: 'structured',
}),
}),
}),
);
});
it('should correctly configure Bedrock endpoint with models and inferenceProfiles', async () => {
const config: Partial<TCustomConfig> = {
endpoints: {
[EModelEndpoint.bedrock]: {
models: [
'us.anthropic.claude-3-7-sonnet-20250219-v1:0',
'us.anthropic.claude-sonnet-4-5-20250929-v1:0',
'global.anthropic.claude-opus-4-5-20251101-v1:0',
],
inferenceProfiles: {
'us.anthropic.claude-3-7-sonnet-20250219-v1:0':
'arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/abc123',
'us.anthropic.claude-sonnet-4-5-20250929-v1:0': '${BEDROCK_SONNET_45_PROFILE}',
},
availableRegions: ['us-east-1', 'us-west-2'],
titleConvo: true,
titleModel: 'us.anthropic.claude-3-7-sonnet-20250219-v1:0',
},
},
};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
endpoints: expect.objectContaining({
[EModelEndpoint.bedrock]: expect.objectContaining({
models: expect.arrayContaining([
'us.anthropic.claude-3-7-sonnet-20250219-v1:0',
'us.anthropic.claude-sonnet-4-5-20250929-v1:0',
'global.anthropic.claude-opus-4-5-20251101-v1:0',
]),
inferenceProfiles: expect.objectContaining({
'us.anthropic.claude-3-7-sonnet-20250219-v1:0':
'arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/abc123',
'us.anthropic.claude-sonnet-4-5-20250929-v1:0': '${BEDROCK_SONNET_45_PROFILE}',
}),
availableRegions: expect.arrayContaining(['us-east-1', 'us-west-2']),
titleConvo: true,
titleModel: 'us.anthropic.claude-3-7-sonnet-20250219-v1:0',
}),
}),
}),
);
});
it('should configure Bedrock endpoint with only inferenceProfiles (no models array)', async () => {
const config: Partial<TCustomConfig> = {
endpoints: {
[EModelEndpoint.bedrock]: {
inferenceProfiles: {
'us.anthropic.claude-3-7-sonnet-20250219-v1:0': '${BEDROCK_INFERENCE_PROFILE_ARN}',
},
},
},
};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
endpoints: expect.objectContaining({
[EModelEndpoint.bedrock]: expect.objectContaining({
inferenceProfiles: expect.objectContaining({
'us.anthropic.claude-3-7-sonnet-20250219-v1:0': '${BEDROCK_INFERENCE_PROFILE_ARN}',
}),
}),
}),
}),
);
});
it('should correctly configure all endpoint when specified', async () => {
const config: Partial<TCustomConfig> = {
endpoints: {
all: {
titleConvo: true,
titleModel: 'gpt-4o-mini',
titleMethod: 'structured',
titlePrompt: 'Default title prompt for all endpoints',
titlePromptTemplate: 'Default template: {{conversation}}',
titleEndpoint: EModelEndpoint.anthropic,
streamRate: 50,
},
[EModelEndpoint.openAI]: {
titleConvo: true,
titleModel: 'gpt-3.5-turbo',
},
},
};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
// Check that 'all' endpoint config is loaded
endpoints: expect.objectContaining({
all: expect.objectContaining({
titleConvo: true,
titleModel: 'gpt-4o-mini',
titleMethod: 'structured',
titlePrompt: 'Default title prompt for all endpoints',
titlePromptTemplate: 'Default template: {{conversation}}',
titleEndpoint: EModelEndpoint.anthropic,
streamRate: 50,
}),
// Check that OpenAI endpoint has its own config
[EModelEndpoint.openAI]: expect.objectContaining({
titleConvo: true,
titleModel: 'gpt-3.5-turbo',
}),
}),
}),
);
});
});
describe('AppService updating app config and issuing warnings', () => {
let initialEnv: NodeJS.ProcessEnv;
beforeEach(() => {
// Store initial environment variables to restore them after each test
initialEnv = { ...process.env };
process.env.CDN_PROVIDER = undefined;
jest.clearAllMocks();
});
afterEach(() => {
// Restore initial environment variables
process.env = { ...initialEnv };
});
it('should initialize app config with default values if config is empty', async () => {
const config = {};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
config: {},
fileStrategy: FileSources.local,
registration: expect.objectContaining({
socialLogins: defaultSocialLogins,
}),
balance: expect.objectContaining({
enabled: false,
startBalance: undefined,
}),
}),
);
});
it('should initialize app config with values from config', async () => {
// Mock loadCustomConfig to return a specific config object with a complete balance config
const config: Partial<TCustomConfig> = {
fileStrategy: FileSources.firebase,
registration: { socialLogins: ['testLogin'] },
balance: {
enabled: false,
startBalance: 5000,
autoRefillEnabled: true,
refillIntervalValue: 15,
refillIntervalUnit: 'hours',
refillAmount: 5000,
},
};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
config,
fileStrategy: config.fileStrategy,
registration: expect.objectContaining({
socialLogins: config.registration?.socialLogins,
}),
balance: config.balance,
}),
);
});
it('should apply the assistants endpoint configuration correctly to app config', async () => {
const config: Partial<TCustomConfig> = {
endpoints: {
assistants: {
version: 'v2',
retrievalModels: ['gpt-4', 'gpt-3.5-turbo'],
capabilities: [],
disableBuilder: true,
pollIntervalMs: 5000,
timeoutMs: 30000,
supportedIds: ['id1', 'id2'],
},
},
};
const result = await AppService({ config });
expect(result).toEqual(
expect.objectContaining({
endpoints: expect.objectContaining({
assistants: expect.objectContaining({
disableBuilder: true,
pollIntervalMs: 5000,
timeoutMs: 30000,
supportedIds: ['id1', 'id2'],
}),
}),
}),
);
// Verify excludedIds is undefined when not provided
expect(result.endpoints!.assistants!.excludedIds).toBeUndefined();
});
it('should not parse environment variable references in OCR config', async () => {
// Mock custom configuration with env variable references in OCR config
const config: Partial<TCustomConfig> = {
ocr: {
apiKey: '${OCR_API_KEY_CUSTOM_VAR_NAME}',
baseURL: '${OCR_BASEURL_CUSTOM_VAR_NAME}',
strategy: OCRStrategy.MISTRAL_OCR,
mistralModel: 'mistral-medium',
},
};
// Set actual environment variables with different values
process.env.OCR_API_KEY_CUSTOM_VAR_NAME = 'actual-api-key';
process.env.OCR_BASEURL_CUSTOM_VAR_NAME = 'https://actual-ocr-url.com';
const result = await AppService({ config });
// Verify that the raw string references were preserved and not interpolated
expect(result).toEqual(
expect.objectContaining({
ocr: expect.objectContaining({
apiKey: '${OCR_API_KEY_CUSTOM_VAR_NAME}',
baseURL: '${OCR_BASEURL_CUSTOM_VAR_NAME}',
strategy: 'mistral_ocr',
mistralModel: 'mistral-medium',
}),
}),
);
});
it('should correctly configure peoplePicker permissions when specified', async () => {
const config = {
interface: {
peoplePicker: {
users: true,
groups: true,
roles: true,
},
},
};
const result = await AppService({ config });
// Check that interface config includes the permissions
expect(result).toEqual(
expect.objectContaining({
interfaceConfig: expect.objectContaining({
peoplePicker: expect.objectContaining({
users: true,
groups: true,
roles: true,
}),
}),
}),
);
});
});