From 397ddc5366c839110ff1ca24704b87d16f6d7943 Mon Sep 17 00:00:00 2001 From: Danny Avila Date: Wed, 24 Jun 2026 17:14:13 -0400 Subject: [PATCH] =?UTF-8?q?=F0=9F=A7=A0=20feat:=20Add=20Memory=20as=20an?= =?UTF-8?q?=20Agent=20Capability=20with=20Inline=20Tools=20and=20Ephemeral?= =?UTF-8?q?=20Badge=20(#13869)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * ๐Ÿง  feat: Memory Agent Capability with Inline Tools and Ephemeral Badge Add `AgentCapabilities.memory`, which expands into the inline set_memory/delete_memory tool pair (mirroring the execute_code expansion via registerMemoryTools) when a run-level memoryAvailable gate holds: capability enabled, memory configured, MEMORIES.USE permission, and personalization not opted out. Surfaces the memory artifact as an attachment in the agents tool-end callback. Adds the ephemeral path (TEphemeralAgent.memory, load/added agent tool injection), a fully-gated memory badge plus tools-dropdown entry, the agent-builder Memory toggle with form round-trip, and a mock e2e test asserting the badge reaches the request payload. Additive to and independent of the existing post-turn memory extraction agent. * ๐Ÿฉน fix: Address Codex review on memory capability (gating, validKeys, usage guard) - Strip the memory capability from the served agents capabilities when memory is not configured/enabled, so the badge, tools dropdown, agent-builder toggle, and backend capability gate stay consistent instead of exposing an inert toggle on default installs (where MEMORIES.USE defaults true). - Surface configured memory.validKeys in the inline tool definitions so the model is told the allowed keys up front, matching the runtime createMemoryTool schema. - Append a strict explicit-request usage guard to the agent instructions when inline memory tools are registered, preserving the memory-agent's privacy behavior. - Add AppService tests covering memory-capability stripping. * โœ… test: Update AppService capability snapshots for memory strip AppService now strips the memory capability from the served agents defaults when no memory block is configured; update the spec's expected capability lists to defaultAgentCapabilitiesWithoutMemory for the no-memory-config cases. * ๐Ÿ›ก๏ธ fix: Address Codex re-review on memory capability (round 2) - Strip the memory capability from the FINAL served agents config, not just defaults; loadEndpoints reparses any endpoints.agents block, so memory was still exposed in that common shape (packages/data-schemas/src/app/service.ts) + regression test. - Re-check the full memory gate (config, opt-out, MEMORIES.USE) inside handleTools before constructing set_memory/delete_memory, so an unsolicited tool call from a model/custom endpoint can't bypass the runtime gates (api/app/clients/tools/util/handleTools.js). - Restore the persisted memory toggle for model-spec conversations via applyModelSpecEphemeralAgent (client/src/utils/endpoints.ts). - Clear LAST_MEMORY_TOGGLE_ on logout and clear-all-chats so a stale memory preference can't leak across users on a shared browser (client/src/utils/localStorage.ts). * ๐Ÿง  fix: Address Codex re-review on memory capability (round 3) - Serialize set_memory writes and advance a running token total inside createMemoryTool, so parallel batched calls in one event-driven turn can't each pass the limit check against a stale total and collectively exceed memory.tokenLimit (packages/api/src/agents/memory.ts) + tests. - Inject the keyed memory context (withKeys) instead of withoutKeys when the running agent has the inline memory capability, so delete_memory has a visible key to target (api/server/controllers/agents/client.js). * ๐Ÿ” fix: Address Codex re-review on memory capability (round 4) - Detect inline memory by tool NAME (set_memory/delete_memory) across an initialized agent's tools + toolDefinitions, since the 'memory' marker is expanded at init and the prior string check never matched; inject the keyed memory context for any primary OR sub-agent that carries the inline memory tools (api/server/controllers/agents/client.js). - Enforce memory WRITE permissions in the inline tool gate: set_memory requires CREATE+UPDATE and delete_memory requires UPDATE (matching the REST memory routes), so a USE-only role can't mutate/delete memories via agent tool calls (api/app/clients/tools/util/handleTools.js). * ๐Ÿ”’ fix: Address Codex re-review on memory capability (round 5) - Gate inline memory registration (memoryAvailable) on the memory WRITE permissions (USE+CREATE+UPDATE), so a read-only-memory role no longer has set_memory/delete_memory shown to the model only for the runtime loader to refuse them (api/server/services/Endpoints/agents/initialize.js). - Enforce the per-agent memory opt-in at execution: handleTools now refuses to construct set_memory/delete_memory unless the agent actually declared them (toolDefinitions/tools), blocking hallucinated/undeclared memory tool calls from mutating memory. - Fail closed when getFormattedMemories errors with a configured tokenLimit, instead of writing as if storage were empty and bypassing the cap (api/app/clients/tools/util/handleTools.js). * ๐Ÿฉน fix: Address Codex re-review on memory capability (round 6) - Fix a P1 regression from the prior round: the execution-context agent keeps the raw 'memory' capability marker (not the expanded set_memory/delete_memory names), so the opt-in check now matches the marker. This restores memory writes/deletes AND avoids hijacking an MCP tool that merely shares the set_memory/delete_memory name (api/app/clients/tools/util/handleTools.js). - Count repeated set_memory writes to the same key as replacements, not additions, against tokenLimit โ€” set_memory upserts, so a same-key rewrite swaps its prior token contribution instead of double-counting (packages/api/src/agents/memory.ts) + test. - Gate the memory badge, tools dropdown, and agent-builder toggle on the full memory write permissions (USE+CREATE+UPDATE) via a shared useHasMemoryAccess hook, so a read-only-memory role no longer sees an enabled Memory control the backend would refuse to wire up. * ๐Ÿงท fix: Address Codex re-review on memory capability (round 7) - Recognize inline memory across both execution-context agent shapes: initializeAgent now sets a LibreChat-only memoryToolsRegistered flag on the InitializedAgent, and the opt-in/detection checks accept that flag OR the raw 'memory' marker. Fixes memory failing for processAddedConvo agents (which store the initialized config, marker already expanded) while staying MCP-name-collision-safe (api/app/clients/tools/util/handleTools.js, packages/api/src/agents/initialize.ts, api/server/controllers/agents/client.js). - Scope keyed memory context to memory-enabled agents only: useMemory now returns both keyed and unkeyed contexts, and buildMessages injects the keyed one (memory keys + token metadata) only to agents that can call delete_memory, while the primary/post-turn path keeps the unkeyed values โ€” so a primary without memory tools no longer sees memory keys it doesn't need. * ๐Ÿ” fix: Address Codex re-review on memory capability (round 8) - Enforce memory size limits on inline writes: createMemoryTool now rejects keys over 1000 chars and values over memory.charLimit, matching the REST memory routes, so an inline-memory agent can't persist blobs the memory UI/API would reject (packages/api/src/agents/memory.ts, api/app/clients/tools/util/handleTools.js) + test. - Recheck the agents 'memory' endpoint capability at execution time, so a stale/hallucinated set_memory/delete_memory call can't mutate memory after an admin removes the capability while the agent document still carries the marker (api/app/clients/tools/util/handleTools.js). * โ™ป๏ธ refactor: Move inline-memory backend logic into packages/api + share memory load Workspace boundary: the inline-memory gating/detection logic that had crept into /api now lives in packages/api/src/agents/memory.ts (TS), with /api kept as thin wrappers. - Add agentHasInlineMemoryTools, isMemoryToolAllowed, and buildInlineMemoryTool to packages/api; handleTools.js now calls buildInlineMemoryTool instead of constructing/gating the tools inline, and client.js imports agentHasInlineMemoryTools instead of redefining it. - Optimize repeated memory loads: getRequestMemories memoizes getFormattedMemories per request (WeakMap keyed by req), so the run's memory-context load and every memory-enabled agent's set_memory token-usage load share a single DB fetch instead of one per agent. * ๐Ÿง  fix: Invalidate request memory cache after inline writes Inline set_memory/delete_memory now invalidate the request-scoped getFormattedMemories cache on a successful write, so a later tool round in the same response is seeded with the post-write usage total instead of the stale pre-write one (multi-round writes no longer collectively exceed tokenLimit, and a set after a delete is not over-counted). The within-round sharing across multiple memory-enabled agents is preserved. * ๐Ÿง  fix: Persist memory capability on saved agents; honor registration flag - Add Tools.memory to the v1 systemTools allowlist so filterAuthorizedTools no longer silently drops the memory marker when an agent with the Memory capability is created/updated/duplicated through the builder (previously the capability only worked for ephemeral chats, not persisted agents). - agentHasInlineMemoryTools now honors an explicit memoryToolsRegistered boolean before falling back to the raw `memory` marker, so an initialized config whose registration was denied (memoryAvailable false) is not given keyed memory context just because the marker survives in tools. * ๐Ÿงฉ fix: Bring memory tool to parity with other ephemeral tools - Add `memory` to the model-spec schema/type and honor `modelSpec.memory` in both ephemeral paths (load.ts, added.ts) and the frontend spec application, so admins can pre-enable Memory from a model spec exactly like webSearch/fileSearch/executeCode. - Add LAST_MEMORY_TOGGLE_ to the timestamped-storage cleanup list so stale per-conversation memory toggles are purged on startup like the others. - Hide the agent-builder Memory toggle for users who disabled memory in personalization (memories === false), mirroring the chat badge's opt-out gate, so the setting isn't shown as inert/misleading. * โœ… test: Cover memory in applyModelSpecEphemeralAgent spec defaults Update the exact-object assertions to include the new `memory` field and add positive coverage that `modelSpec.memory` maps to the ephemeral agent's `memory` flag. Fixes the shard 2/4 failure from 672a03b05. --- api/app/clients/tools/util/handleTools.js | 16 +- api/server/controllers/agents/callbacks.js | 22 + api/server/controllers/agents/client.js | 49 +- api/server/controllers/agents/client.test.js | 26 +- .../agents/filterAuthorizedTools.spec.js | 4 +- api/server/controllers/agents/v1.js | 1 + .../services/Endpoints/agents/addedConvo.js | 2 + .../services/Endpoints/agents/initialize.js | 26 + api/server/services/ToolService.js | 12 +- client/src/Providers/BadgeRowContext.tsx | 21 + client/src/common/agents-types.ts | 1 + client/src/components/Chat/Input/BadgeRow.tsx | 4 +- client/src/components/Chat/Input/Memory.tsx | 40 ++ .../components/Chat/Input/ToolsDropdown.tsx | 65 ++- .../SidePanel/Agents/AgentConfig.tsx | 19 +- .../SidePanel/Agents/AgentPanel.tsx | 3 + .../SidePanel/Agents/AgentSelect.tsx | 3 +- .../components/SidePanel/Agents/Memory.tsx | 67 +++ .../src/hooks/Agents/useAgentCapabilities.ts | 7 + client/src/hooks/Roles/index.ts | 1 + client/src/hooks/Roles/useHasMemoryAccess.ts | 25 + client/src/locales/en/translation.json | 2 + .../applyModelSpecEphemeralAgent.test.ts | 14 + client/src/utils/__tests__/timestamps.test.ts | 13 + client/src/utils/endpoints.ts | 2 + client/src/utils/localStorage.ts | 2 + client/src/utils/timestamps.ts | 1 + e2e/config/librechat.e2e.yaml | 7 + e2e/specs/mock/helpers.ts | 8 + e2e/specs/mock/memory.spec.ts | 42 ++ packages/api/src/agents/added.ts | 5 + packages/api/src/agents/discovery.ts | 14 +- packages/api/src/agents/initialize.ts | 35 ++ packages/api/src/agents/load.ts | 3 + packages/api/src/agents/memory.spec.ts | 149 +++++- packages/api/src/agents/memory.ts | 499 +++++++++++++++--- packages/api/src/app/AppService.spec.ts | 13 +- packages/data-provider/src/config.ts | 4 + packages/data-provider/src/models.ts | 2 + packages/data-provider/src/schemas.ts | 1 + packages/data-provider/src/types.ts | 1 + packages/data-schemas/src/app/service.spec.ts | 41 +- packages/data-schemas/src/app/service.ts | 29 +- 43 files changed, 1191 insertions(+), 110 deletions(-) create mode 100644 client/src/components/Chat/Input/Memory.tsx create mode 100644 client/src/components/SidePanel/Agents/Memory.tsx create mode 100644 client/src/hooks/Roles/useHasMemoryAccess.ts create mode 100644 e2e/specs/mock/memory.spec.ts diff --git a/api/app/clients/tools/util/handleTools.js b/api/app/clients/tools/util/handleTools.js index adeb9f7ca9..e4e4726610 100644 --- a/api/app/clients/tools/util/handleTools.js +++ b/api/app/clients/tools/util/handleTools.js @@ -6,9 +6,12 @@ const { createSafeUser, mcpToolPattern, loadWebSearchAuth, + buildInlineMemoryTool, getCodeApiAuthHeaders, buildImageToolContext, + SET_MEMORY_TOOL_NAME, buildWebSearchContext, + DELETE_MEMORY_TOOL_NAME, buildWebSearchDynamicContext, } = require('@librechat/api'); const { @@ -48,7 +51,7 @@ const { getUserPluginAuthValue } = require('~/server/services/PluginService'); const { loadAuthValues } = require('~/server/services/Tools/credentials'); const { getMCPServerTools } = require('~/server/services/Config'); const { getMCPServersRegistry } = require('~/config'); -const { getRoleByName } = require('~/models'); +const { getRoleByName, setMemory, deleteMemory, getFormattedMemories } = require('~/models'); /** * Validates the availability and authentication of tools for a user based on environment variables or user-specific plugin authentication values. @@ -357,6 +360,17 @@ const loadTools = async ({ }); }; continue; + } else if (tool === SET_MEMORY_TOOL_NAME || tool === DELETE_MEMORY_TOOL_NAME) { + requestedTools[tool] = () => + buildInlineMemoryTool({ + toolName: tool, + req: options.req, + agent, + userId: user, + memoryMethods: { setMemory, deleteMemory, getFormattedMemories }, + getRoleByName, + }); + continue; } else if (tool && mcpToolPattern.test(tool)) { if (!canUseMCP) { if (!loggedMCPDenied) { diff --git a/api/server/controllers/agents/callbacks.js b/api/server/controllers/agents/callbacks.js index d0b2ea4aaa..8b00ac36d6 100644 --- a/api/server/controllers/agents/callbacks.js +++ b/api/server/controllers/agents/callbacks.js @@ -728,6 +728,28 @@ function createToolEndCallback({ req, res, artifactPromises, streamId = null }) ); } + if (output.artifact[Tools.memory]) { + artifactPromises.push( + (async () => { + const attachment = { + type: Tools.memory, + messageId: metadata.run_id, + toolCallId: output.tool_call_id, + conversationId: metadata.thread_id, + [Tools.memory]: output.artifact[Tools.memory], + }; + if (!streamId && !res.headersSent) { + return attachment; + } + writeAttachment(res, streamId, attachment); + return attachment; + })().catch((error) => { + logger.error('Error processing memory artifact content:', error); + return null; + }), + ); + } + if (output.artifact.content) { /** @type {FormattedContent[]} */ const content = output.artifact.content; diff --git a/api/server/controllers/agents/client.js b/api/server/controllers/agents/client.js index e073c964d7..df2515c2ab 100644 --- a/api/server/controllers/agents/client.js +++ b/api/server/controllers/agents/client.js @@ -33,7 +33,9 @@ const { GenerationJobManager, getTransactionsConfig, resolveRecursionLimit, + getRequestMemories, createMemoryProcessor, + agentHasInlineMemoryTools, loadAgent: loadAgentFn, createMultiAgentMapper, filterMalformedContentParts, @@ -516,11 +518,14 @@ class AgentClient extends BaseClient { } } - /** Memory context (user preferences/memories) */ - const withoutKeys = await this.useMemory(); - const memoryContext = withoutKeys - ? `${memoryInstructions}\n\n# Existing memory about the user:\n${withoutKeys}` - : undefined; + /** Memory context (user preferences/memories). Keyed context (with memory + * keys + token metadata) is reserved for agents that can call + * `delete_memory`; everyone else gets the unkeyed values only. */ + const memories = await this.useMemory(); + const buildMemoryContext = (text) => + text ? `${memoryInstructions}\n\n# Existing memory about the user:\n${text}` : undefined; + const memoryContext = buildMemoryContext(memories?.withoutKeys); + const keyedMemoryContext = buildMemoryContext(memories?.withKeys); const sharedRunContext = sharedRunContextParts.join('\n\n'); const memoryAgentEnabled = isMemoryAgentEnabled(this.options.req.config?.memory); @@ -571,8 +576,13 @@ class AgentClient extends BaseClient { await Promise.all( allAgents.map(({ agent, agentId }) => { const agentRunContextParts = [sharedRunContext]; - if (memoryContext && (agentId === this.options.agent.id || memoryAgentEnabled)) { - agentRunContextParts.push(memoryContext); + const agentHasMemory = agentHasInlineMemoryTools(agent); + const agentMemoryContext = agentHasMemory ? keyedMemoryContext : memoryContext; + if ( + agentMemoryContext && + (agentId === this.options.agent.id || memoryAgentEnabled || agentHasMemory) + ) { + agentRunContextParts.push(agentMemoryContext); } const scopedContext = agentScopedContext.get(agentId); if (scopedContext) { @@ -623,7 +633,7 @@ class AgentClient extends BaseClient { } /** - * @returns {Promise} + * @returns {Promise<{ withKeys?: string; withoutKeys?: string } | undefined>} */ async useMemory() { const user = this.options.req.user; @@ -654,8 +664,12 @@ class AgentClient extends BaseClient { if (!isMemoryAgentEnabled(memoryConfig)) { try { - const { withoutKeys } = await db.getFormattedMemories({ userId }); - return withoutKeys; + const { withKeys, withoutKeys } = await getRequestMemories({ + req: this.options.req, + userId, + getFormattedMemories: db.getFormattedMemories, + }); + return { withKeys, withoutKeys }; } catch (error) { logger.error( '[api/server/controllers/agents/client.js #useMemory] Error loading memories', @@ -772,7 +786,20 @@ class AgentClient extends BaseClient { }); this.processMemory = processMemory; - return withoutKeys; + let withKeys = withoutKeys; + try { + ({ withKeys } = await getRequestMemories({ + req: this.options.req, + userId, + getFormattedMemories: db.getFormattedMemories, + })); + } catch (error) { + logger.error( + '[api/server/controllers/agents/client.js #useMemory] Error loading keyed memories', + error, + ); + } + return { withKeys, withoutKeys }; } /** diff --git a/api/server/controllers/agents/client.test.js b/api/server/controllers/agents/client.test.js index c71ede7b23..1320e362c1 100644 --- a/api/server/controllers/agents/client.test.js +++ b/api/server/controllers/agents/client.test.js @@ -2361,7 +2361,9 @@ describe('AgentClient - titleConvo', () => { it('should only pass memory context to the primary agent by default', async () => { const memoryContent = 'User prefers dark mode. User is a software developer.'; - client.useMemory = jest.fn().mockResolvedValue(memoryContent); + client.useMemory = jest + .fn() + .mockResolvedValue({ withKeys: memoryContent, withoutKeys: memoryContent }); const parallelAgent1 = { id: 'parallel-agent-1', @@ -2414,7 +2416,9 @@ describe('AgentClient - titleConvo', () => { it('should pass memory context to parallel agents when automatic memory updates are enabled', async () => { const memoryContent = 'User prefers dark mode. User is a software developer.'; - client.useMemory = jest.fn().mockResolvedValue(memoryContent); + client.useMemory = jest + .fn() + .mockResolvedValue({ withKeys: memoryContent, withoutKeys: memoryContent }); mockReq.config.memory.agent = { enabled: true, id: 'memory-agent', @@ -2485,7 +2489,9 @@ describe('AgentClient - titleConvo', () => { it('should handle parallel agents without existing instructions when memory stays primary-only', async () => { const memoryContent = 'User is a data scientist.'; - client.useMemory = jest.fn().mockResolvedValue(memoryContent); + client.useMemory = jest + .fn() + .mockResolvedValue({ withKeys: memoryContent, withoutKeys: memoryContent }); const parallelAgentNoInstructions = { id: 'parallel-agent-no-instructions', @@ -2521,7 +2527,9 @@ describe('AgentClient - titleConvo', () => { it('should not modify agentConfigs when none exist', async () => { const memoryContent = 'User prefers concise responses.'; - client.useMemory = jest.fn().mockResolvedValue(memoryContent); + client.useMemory = jest + .fn() + .mockResolvedValue({ withKeys: memoryContent, withoutKeys: memoryContent }); client.agentConfigs = null; @@ -2547,7 +2555,9 @@ describe('AgentClient - titleConvo', () => { it('should handle empty agentConfigs map', async () => { const memoryContent = 'User likes detailed explanations.'; - client.useMemory = jest.fn().mockResolvedValue(memoryContent); + client.useMemory = jest + .fn() + .mockResolvedValue({ withKeys: memoryContent, withoutKeys: memoryContent }); client.agentConfigs = new Map(); @@ -2720,7 +2730,7 @@ describe('AgentClient - titleConvo', () => { const result = await client.useMemory(); - expect(result).toBe('likes pasta'); + expect(result).toEqual({ withKeys: 'food: likes pasta', withoutKeys: 'likes pasta' }); expect(mockGetFormattedMemories).toHaveBeenCalledWith({ userId: 'user-123' }); expect(mockInitializeAgent).not.toHaveBeenCalled(); expect(mockCreateMemoryProcessor).not.toHaveBeenCalled(); @@ -2745,7 +2755,7 @@ describe('AgentClient - titleConvo', () => { const result = await client.useMemory(); - expect(result).toBe(''); + expect(result).toEqual({ withKeys: '', withoutKeys: '' }); expect(mockGetFormattedMemories).toHaveBeenCalledWith({ userId: 'user-123' }); expect(mockInitializeAgent).not.toHaveBeenCalled(); expect(mockCreateMemoryProcessor).not.toHaveBeenCalled(); @@ -2770,7 +2780,7 @@ describe('AgentClient - titleConvo', () => { const result = await client.useMemory(); - expect(result).toBe('prefers concise answers'); + expect(result).toEqual({ withKeys: 'tone: concise', withoutKeys: 'prefers concise answers' }); expect(mockLoadAgent).not.toHaveBeenCalled(); expect(mockInitializeAgent).not.toHaveBeenCalled(); expect(mockCreateMemoryProcessor).not.toHaveBeenCalled(); diff --git a/api/server/controllers/agents/filterAuthorizedTools.spec.js b/api/server/controllers/agents/filterAuthorizedTools.spec.js index 89835fac06..677bccdfe0 100644 --- a/api/server/controllers/agents/filterAuthorizedTools.spec.js +++ b/api/server/controllers/agents/filterAuthorizedTools.spec.js @@ -183,12 +183,12 @@ describe('MCP Tool Authorization', () => { test('should keep system tools without querying MCP registry', async () => { const result = await filterAuthorizedTools({ - tools: ['execute_code', 'file_search', 'web_search'], + tools: ['execute_code', 'file_search', 'web_search', 'memory'], userId, availableTools: {}, }); - expect(result).toEqual(['execute_code', 'file_search', 'web_search']); + expect(result).toEqual(['execute_code', 'file_search', 'web_search', 'memory']); expect(mockGetAllServerConfigs).not.toHaveBeenCalled(); }); diff --git a/api/server/controllers/agents/v1.js b/api/server/controllers/agents/v1.js index ec1c6e62f2..5742a3ad7e 100644 --- a/api/server/controllers/agents/v1.js +++ b/api/server/controllers/agents/v1.js @@ -56,6 +56,7 @@ const systemTools = { [Tools.execute_code]: true, [Tools.file_search]: true, [Tools.web_search]: true, + [Tools.memory]: true, }; const MAX_SEARCH_LEN = 100; diff --git a/api/server/services/Endpoints/agents/addedConvo.js b/api/server/services/Endpoints/agents/addedConvo.js index 847c6c01af..37c703124d 100644 --- a/api/server/services/Endpoints/agents/addedConvo.js +++ b/api/server/services/Endpoints/agents/addedConvo.js @@ -79,6 +79,7 @@ const processAddedConvo = async ({ skillStates, defaultActiveOnShare, codeEnvAvailable, + memoryAvailable, }) => { const addedConvo = endpointOption.addedConvo; if (addedConvo == null) { @@ -173,6 +174,7 @@ const processAddedConvo = async ({ ephemeralSkillsToggle, }), codeEnvAvailable, + memoryAvailable, skillStates, defaultActiveOnShare, }, diff --git a/api/server/services/Endpoints/agents/initialize.js b/api/server/services/Endpoints/agents/initialize.js index 9caee651c4..530901e691 100644 --- a/api/server/services/Endpoints/agents/initialize.js +++ b/api/server/services/Endpoints/agents/initialize.js @@ -1,8 +1,10 @@ const { logger } = require('@librechat/data-schemas'); const { createContentAggregator } = require('@librechat/agents'); const { + checkAccess, loadSkillStates, initializeAgent, + isMemoryEnabled, primeInvokedSkills, validateAgentModel, extractManualSkills, @@ -15,9 +17,11 @@ const { buildAgentContextAttachmentsByAgentId, } = require('@librechat/api'); const { + Permissions, ResourceType, EModelEndpoint, PermissionBits, + PermissionTypes, MAX_SUBAGENT_DEPTH, isAgentsEndpoint, getResponseSender, @@ -148,6 +152,24 @@ const initializeClient = async ({ req, res, signal, endpointOption }) => { const ephemeralSkillsToggle = req.body?.ephemeralAgent?.skills === true; const skillDbMethods = getSkillDbMethods(); + /** Run-level gate for inline memory tools: the `memory` capability must be + * enabled, memory must be configured, and the user must not have opted out. + * Requires the memory WRITE permissions (CREATE + UPDATE) โ€” both inline tools + * mutate memory โ€” so the tools aren't registered (and shown to the model) for + * read-only-memory roles that the runtime loader would then refuse to build. + * Agents (or the ephemeral memory badge) opt in per-agent via the `memory` + * marker on `tools`. */ + const memoryAvailable = + enabledCapabilities.has(AgentCapabilities.memory) && + isMemoryEnabled(appConfig?.memory) && + req.user?.personalization?.memories !== false && + (await checkAccess({ + user: req.user, + permissionType: PermissionTypes.MEMORIES, + permissions: [Permissions.USE, Permissions.CREATE, Permissions.UPDATE], + getRoleByName: db.getRoleByName, + })); + const accessibleSkillIds = skillsCapabilityEnabled ? withDeploymentSkillIds( await findAccessibleResources({ @@ -383,6 +405,7 @@ const initializeClient = async ({ req, res, signal, endpointOption }) => { accessibleSkillIds: primaryScopedSkillIds, skillAuthoringAvailable: primarySkillAuthoringAvailable, codeEnvAvailable, + memoryAvailable, skillStates, defaultActiveOnShare, manualSkills, @@ -458,6 +481,7 @@ const initializeClient = async ({ req, res, signal, endpointOption }) => { skillStates, defaultActiveOnShare, codeEnvAvailable, + memoryAvailable, }, { getAgent: db.getAgent, @@ -529,6 +553,7 @@ const initializeClient = async ({ req, res, signal, endpointOption }) => { skillStates, defaultActiveOnShare, codeEnvAvailable, + memoryAvailable, }); if (updatedMCPAuthMap) { @@ -667,6 +692,7 @@ const initializeClient = async ({ req, res, signal, endpointOption }) => { * false`, so `bash_tool` / `read_file` sandbox fallback are * silently gated off even though the seed walk found it. */ codeEnvAvailable, + memoryAvailable, skillStates, defaultActiveOnShare, }, diff --git a/api/server/services/ToolService.js b/api/server/services/ToolService.js index b11529205b..082217e783 100644 --- a/api/server/services/ToolService.js +++ b/api/server/services/ToolService.js @@ -507,7 +507,12 @@ async function processRequiredActions(client, requiredActions) { * }>} The agent tools and registry. */ /** Native LibreChat tools that are not in the manifest */ -const nativeTools = new Set([Tools.execute_code, Tools.file_search, Tools.web_search]); +const nativeTools = new Set([ + Tools.execute_code, + Tools.file_search, + Tools.web_search, + Tools.memory, +]); /** Checks if a tool name is a known built-in tool */ const isBuiltInTool = (toolName) => @@ -571,6 +576,9 @@ async function loadToolDefinitionsWrapper({ req, res, agent, streamId = null, to if (tool === Tools.web_search) { return checkCapability(AgentCapabilities.web_search); } + if (tool === Tools.memory) { + return checkCapability(AgentCapabilities.memory); + } if (isActionTool(tool)) { return actionsEnabled; } @@ -1123,6 +1131,8 @@ async function loadAgentTools({ } else if (tool === Tools.web_search) { includesWebSearch = checkCapability(AgentCapabilities.web_search); return includesWebSearch; + } else if (tool === Tools.memory) { + return checkCapability(AgentCapabilities.memory); } else if (isActionTool(tool)) { return actionsEnabled; } else if (tool?.includes(Constants.mcp_delimiter)) { diff --git a/client/src/Providers/BadgeRowContext.tsx b/client/src/Providers/BadgeRowContext.tsx index 448af4339f..17e8f3b6e8 100644 --- a/client/src/Providers/BadgeRowContext.tsx +++ b/client/src/Providers/BadgeRowContext.tsx @@ -17,6 +17,7 @@ interface BadgeRowContextType { storageContextKey?: string; agentsConfig?: TAgentsEndpoint | null; skills: ReturnType; + memory: ReturnType; webSearch: ReturnType; artifacts: ReturnType; fileSearch: ReturnType; @@ -100,12 +101,14 @@ export default function BadgeRowProvider({ const fileSearchToggleKey = `${LocalStorageKeys.LAST_FILE_SEARCH_TOGGLE_}${storageSuffix}`; const artifactsToggleKey = `${LocalStorageKeys.LAST_ARTIFACTS_TOGGLE_}${storageSuffix}`; const skillsToggleKey = `${LocalStorageKeys.LAST_SKILLS_TOGGLE_}${storageSuffix}`; + const memoryToggleKey = `${LocalStorageKeys.LAST_MEMORY_TOGGLE_}${storageSuffix}`; const codeToggleValue = getTimestampedValue(codeToggleKey); const webSearchToggleValue = getTimestampedValue(webSearchToggleKey); const fileSearchToggleValue = getTimestampedValue(fileSearchToggleKey); const artifactsToggleValue = getTimestampedValue(artifactsToggleKey); const skillsToggleValue = getTimestampedValue(skillsToggleKey); + const memoryToggleValue = getTimestampedValue(memoryToggleKey); const initialValues: Record = {}; @@ -149,6 +152,14 @@ export default function BadgeRowProvider({ } } + if (memoryToggleValue !== null) { + try { + initialValues[Tools.memory] = JSON.parse(memoryToggleValue); + } catch (e) { + console.error('Failed to parse memory toggle value:', e); + } + } + const hasOverrides = Object.keys(initialValues).length > 0; /** Read persisted MCP values from localStorage */ @@ -250,10 +261,20 @@ export default function BadgeRowProvider({ isAuthenticated: true, }); + /** Memory hook - per-conversation toggle for the inline memory tools */ + const memory = useToolToggle({ + conversationId, + storageContextKey, + toolKey: Tools.memory, + localStorageKey: LocalStorageKeys.LAST_MEMORY_TOGGLE_, + isAuthenticated: true, + }); + const mcpServerManager = useMCPServerManager({ conversationId, storageContextKey }); const value: BadgeRowContextType = { skills, + memory, webSearch, artifacts, fileSearch, diff --git a/client/src/common/agents-types.ts b/client/src/common/agents-types.ts index 7313812ec5..91692efab3 100644 --- a/client/src/common/agents-types.ts +++ b/client/src/common/agents-types.ts @@ -24,6 +24,7 @@ export type TAgentCapabilities = { [AgentCapabilities.web_search]: boolean; [AgentCapabilities.file_search]: boolean; [AgentCapabilities.execute_code]: boolean; + [AgentCapabilities.memory]?: boolean; [AgentCapabilities.end_after_tools]?: boolean; [AgentCapabilities.hide_sequential_outputs]?: boolean; }; diff --git a/client/src/components/Chat/Input/BadgeRow.tsx b/client/src/components/Chat/Input/BadgeRow.tsx index 7b7140c9fb..0964926a23 100644 --- a/client/src/components/Chat/Input/BadgeRow.tsx +++ b/client/src/components/Chat/Input/BadgeRow.tsx @@ -11,16 +11,17 @@ import React, { import { Badge } from '@librechat/client'; import { useRecoilValue, useRecoilCallback } from 'recoil'; import type { LucideIcon } from 'lucide-react'; +import type { BadgeItem } from '~/common'; import CodeInterpreter from './CodeInterpreter'; import { BadgeRowProvider } from '~/Providers'; import ToolsDropdown from './ToolsDropdown'; -import type { BadgeItem } from '~/common'; import { useChatBadges } from '~/hooks'; import ToolDialogs from './ToolDialogs'; import FileSearch from './FileSearch'; import Artifacts from './Artifacts'; import MCPSelect from './MCPSelect'; import WebSearch from './WebSearch'; +import Memory from './Memory'; import Skills from './Skills'; import store from '~/store'; @@ -375,6 +376,7 @@ function BadgeRow({ + diff --git a/client/src/components/Chat/Input/Memory.tsx b/client/src/components/Chat/Input/Memory.tsx new file mode 100644 index 0000000000..d43610599b --- /dev/null +++ b/client/src/components/Chat/Input/Memory.tsx @@ -0,0 +1,40 @@ +import React, { memo } from 'react'; +import { Brain } from 'lucide-react'; +import { CheckboxButton } from '@librechat/client'; +import { defaultAgentCapabilities } from 'librechat-data-provider'; +import { useLocalize, useHasMemoryAccess, useAgentCapabilities, useAuthContext } from '~/hooks'; +import { useBadgeRowContext } from '~/Providers'; + +function Memory() { + const localize = useLocalize(); + const { user } = useAuthContext(); + const context = useBadgeRowContext(); + const { toggleState: memoryActive, debouncedChange, isPinned } = context?.memory ?? {}; + + const canUseMemory = useHasMemoryAccess(); + + const { memoryEnabled } = useAgentCapabilities( + context?.agentsConfig?.capabilities ?? defaultAgentCapabilities, + ); + + const hasOptedOut = user?.personalization?.memories === false; + + if (!canUseMemory || !memoryEnabled || hasOptedOut) { + return null; + } + + return ( + (memoryActive || isPinned) && ( +