From d5819becf240bb68da941213ae99b756940adaca Mon Sep 17 00:00:00 2001 From: Danny Avila Date: Thu, 30 Jul 2026 13:32:04 -0400 Subject: [PATCH] =?UTF-8?q?=F0=9F=8E=AF=20feat:=20Per-Tool=20Intent=20Labe?= =?UTF-8?q?ls=20for=20Model=20Specs=20(#14526)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * 🎯 feat: Per-Tool Intent Labels for Model Specs A model spec could only turn intent labels on for ALL of its eligible tools. Saved agents have had per-tool control since the capability landed (`tool_options[id].describe_intent`, with the builder toggle following in the UI slice), but a model spec is admin YAML that produces an ephemeral agent β€” there is no agent document to hold per-tool options, so `describeIntent: true` synthesized an entry for every eligible tool. `describeIntent` now accepts a string array alongside the boolean, matching the `skills` field already in the same schema: describeIntent: true # every eligible tool describeIntent: ['web_search', 'search_code_mcp_github'] This matters because the label costs schema tokens on every request, so an admin may want it on a handful of illegible calls rather than the whole toolset. - Named tools still pass eligibility, so an excluded tool cannot be forced on by listing it. - An empty array reads as disabled. - Names that are not eligible or not equipped on the spec are logged rather than silently skipped β€” a typo in a spec would otherwise be undiagnosable. - The ephemeral toggle stays boolean and stays global even when a spec list is present: it has no per-tool UI to drive it, so narrowing it would silently cover fewer tools than the user asked for. `runInBackground` has the same all-or-nothing limitation and could take the same shape; left alone here to keep this change reviewable. * 🎯 feat: Per-Tool Background Dispatch for Model Specs Gives `runInBackground` the same `boolean | string[]` shape as `describeIntent`, so the two per-tool capabilities are configured identically from a model spec: runInBackground: true # every eligible tool runInBackground: ['slow_report_mcp_analytics'] # only this one Selectivity matters more here than for intent labels. An intent label is inert β€” it costs tokens and nothing else. Backgrounding changes execution semantics: the model gets a synthetic handle and must poll. Letting an admin detach one slow MCP call without making every other tool in the spec detachable is the difference between a usable setting and an all-or-nothing one. Same guarantees as the intent equivalent: - Named tools still pass eligibility, so the exclusion list still holds β€” a list cannot force on web_search, file_search, image gen, the HITL tool, or anything whose attachments/artifact continuity would break. - Empty array reads as disabled. - Unmatched names are logged rather than silently skipped. - The ephemeral toggle stays boolean and global; it has no per-tool UI. Also fixes a latent no-op: this function did not skip the lazily-expanded `mcp_all` placeholder, so a spec with an overlay MCP server recorded an option under a name `applyBackgroundToolCalls` can never match. The intent equivalent already skipped it; now both do. * ♻️ refactor: One Definition of the mcp_all Placeholder Guard Fixing the background no-op left the placeholder prefix declared twice β€” once per capability synthesizer β€” which is the same duplicated-literal shape that made the intent label marker fragile: two copies that must agree, with drift producing a silent no-op rather than an error. `MCP_ALL_PLACEHOLDER_PREFIX` and `isMCPAllPlaceholder` now live beside `mcpToolPattern` in mcp/utils, so both synthesizers cannot disagree about which tool entries to ignore, and anything added later that keys per-tool config by exact name has an obvious guard to reach for. Audited the rest of the capability family while here: only background and intent synthesize per-tool options from a model spec. `defer_loading` and `allowed_callers` have no model-spec path at all, so neither can carry this bug. Both synthesizers now have an explicit regression test naming the placeholder. * 🧯 fix: Treat a describeIntent List as a Selection Policy, Not a Filter Two build/behavior defects from the previous commits. **Narrowing did not actually narrow.** Omitting a tool from the synthesized options is not the same as opting it out, because intent has two default-on paths background does not: `isIntentOptedIn` treats every NATIVE_INTENT_TOOL_NAMES member as enabled when it finds no entry, and `sanitizeIntentLabels` keeps an SDK-native label unless it sees an explicit `describe_intent: false`. So `describeIntent: ['web_search']` still labelled `set_memory`, and an empty list β€” the most explicit way to say "none" β€” disabled nothing at all. A list is now a selection policy: selected eligible tools get true, unselected get an explicit false. Ineligible tools still get no entry at all. Background needs no equivalent change: it opts in on `run_in_background === true` only, with no default-on set, so omission there genuinely means off. **Fixed the CI build break.** `MCP_ALL_PLACEHOLDER_PREFIX` was exported without a type annotation, which `tsc --noEmit` accepts but the package build rejects under `--isolatedDeclarations` (TS9010) β€” the same reason `mcpToolPattern` beside it is annotated `: RegExp`. Verified with an actual `npm run build` this time, not just a typecheck. * 🧯 fix: Propagate Intent Selection Through Capability Marker Expansion A model spec's `tools` carries capability MARKERS, not the definition names initialization actually registers, so an option recorded under a marker never matches the tool it becomes β€” the same silent no-op as an `mcp_all` placeholder entry. Harmless for an opt-IN (the tool keeps its default) but not for the opt-OUTs the previous commit introduced: `memory` becomes `set_memory` + `delete_memory`, both default-on natives, and `execute_code` becomes `bash_tool`, which carries an SDK-native label that survives unless sanitize sees an explicit false. So `describeIntent: ['web_search']` on a spec with `memory: true` still labelled both memory tools, and `[]` disabled neither. `expandIntentToolOptions` propagates a marker's value to the names it expands into, mirroring `expandCodeToolOptions` in background.ts which solves the same problem for the code marker. Applied at APPLY time rather than synthesis, so hand-edited saved agents that key options by marker benefit too, not just synthesized specs. An explicit per-tool entry always wins β€” expansion only fills names the caller did not already decide β€” and it runs in both `applyIntentLabels` and `sanitizeIntentLabels`, since the SDK-native strip reads the same options. Verified with an actual package build, not just a typecheck. * ♻️ refactor: Resolve Spec Tool Selections Against Final Definitions Three review rounds hit the same root cause from three directions: a per-tool option synthesized at load time is keyed by names that may not exist at injection time. Spec tools carry capability markers (execute_code, memory), skills never reach the tools array at all, and lazy MCP servers expand after synthesis - every mismatch was a silent no-op, and each fix added another hand-maintained marker map that the next case leaked past. This removes the name-space gap instead of bridging it per case: - Synthesis records the selection as a policy: a wildcard '*' entry carries the default (true for "every tool", false for "only the named ones") and listed names are recorded verbatim. No load-time enumeration, eligibility checks, or placeholder special-casing. - Resolution happens at injection time, per final definition: explicit name -> capability marker projection -> wildcard. The projection maps a marker onto the names its registration actually produced this run - the registrars report their own tool names and initializeAgent accumulates them - so the mapping cannot drift from what gets registered. - The unmatched-name diagnosis moves to the apply passes, the one place the real definitions are known, so a selection naming a marker whose runtime expansion is entirely ineligible (runInBackground: ['memory']) is now warned about instead of recorded as a dead success. Covers all four round-3 findings: code opt-outs now reach create_file/edit_file/read_file, skill definitions are governed by narrowing selections, the memory marker is rejected and diagnosed for backgrounding, and the mcp_all placeholder predicate is no longer consulted for selections at all (its one remaining consumer is the definitions loader that defines the convention). * 🧯 fix: Reject Dead Ask-Tool Selections and the Reserved Wildcard Two review findings on the selection policy, both fixed at the point where they are knowably invalid: - ask_user_question joins EXCLUDED_INTENT_TOOL_NAMES: createRun strips its provisional definition and rebuilds the graph tool from its own Zod schema, so definition-level injection never reaches the model. A describeIntent selection naming it now warns as ineligible instead of crediting a label that gets discarded. Real intent support for the ask tool lands with the HITL slice via the interrupt payload. - A literal '*' in a describeIntent/runInBackground list is dropped with a warning at synthesis: it would overwrite the wildcard opt-out default and silently enable the capability for every eligible tool instead of selecting one named tool. The wildcard is reserved for the internal policy; boolean true is the supported way to cover everything. --- .../api/src/agents/__tests__/load.spec.ts | 10 +- packages/api/src/agents/added.ts | 23 +- packages/api/src/agents/background.spec.ts | 173 ++++++++-- packages/api/src/agents/background.ts | 128 ++++--- packages/api/src/agents/initialize.ts | 32 ++ packages/api/src/agents/intent.spec.ts | 321 ++++++++++++++++-- packages/api/src/agents/intent.ts | 171 +++++----- packages/api/src/agents/load.ts | 13 +- packages/api/src/agents/memory.ts | 7 +- packages/api/src/agents/selection.ts | 214 ++++++++++++ packages/api/src/agents/skills.ts | 19 +- packages/api/src/agents/tools.ts | 16 +- packages/api/src/mcp/utils.ts | 17 + packages/api/src/tools/definitions.ts | 7 +- packages/data-provider/src/models.ts | 26 +- 15 files changed, 927 insertions(+), 250 deletions(-) create mode 100644 packages/api/src/agents/selection.ts diff --git a/packages/api/src/agents/__tests__/load.spec.ts b/packages/api/src/agents/__tests__/load.spec.ts index 15719fdb60..0767f388bb 100644 --- a/packages/api/src/agents/__tests__/load.spec.ts +++ b/packages/api/src/agents/__tests__/load.spec.ts @@ -413,10 +413,9 @@ describe('loadAgent', () => { deps, ); - // eligible tools opt in (MCP + code execution); excluded built-ins (web_search) do not - expect(result?.tool_options?.crm_lookup).toEqual({ run_in_background: true }); - expect(result?.tool_options?.web_search).toBeUndefined(); - expect(result?.tool_options?.execute_code).toEqual({ run_in_background: true }); + // recorded as a wildcard policy; eligibility (e.g. excluding web_search) + // is enforced against the final definitions in applyBackgroundToolCalls + expect(result?.tool_options).toEqual({ '*': { run_in_background: true } }); }); test('synthesizes background tool_options from a model spec (runInBackground: true), and not without it', async () => { @@ -456,8 +455,7 @@ describe('loadAgent', () => { }, deps, ); - expect(withFlag?.tool_options?.crm_lookup).toEqual({ run_in_background: true }); - expect(withFlag?.tool_options?.web_search).toBeUndefined(); + expect(withFlag?.tool_options).toEqual({ '*': { run_in_background: true } }); const withoutFlag = await loadAgent( { diff --git a/packages/api/src/agents/added.ts b/packages/api/src/agents/added.ts index c96a707d28..b2b80a437a 100644 --- a/packages/api/src/agents/added.ts +++ b/packages/api/src/agents/added.ts @@ -9,9 +9,10 @@ import { } from 'librechat-data-provider'; import type { Agent, AgentToolOptions, TConversation, TModelSpec } from 'librechat-data-provider'; import type { AppConfig } from '@librechat/data-schemas'; -import { synthesizeIntentToolOptions, mergeSynthesizedToolOptions } from '~/agents/intent'; import { ASK_USER_QUESTION_TOOL_NAME } from '~/agents/hitl/askUserQuestionTool'; import { synthesizeBackgroundToolOptions } from '~/agents/background'; +import { mergeSynthesizedToolOptions } from '~/agents/selection'; +import { synthesizeIntentToolOptions } from '~/agents/intent'; import { requiresEphemeralUserConnection } from '~/mcp/utils'; import { getCustomEndpointConfig } from '~/app/config'; @@ -161,14 +162,14 @@ export async function loadAddedAgent( applyModelSpecSkills(result, modelSpec); applyModelSpecSubagents(result, modelSpec); const primaryBackgroundToolOptions: AgentToolOptions | undefined = - synthesizeBackgroundToolOptions(result.tools as string[], { ephemeralAgent, modelSpec }); + synthesizeBackgroundToolOptions({ ephemeralAgent, modelSpec }); if (primaryBackgroundToolOptions) { result.tool_options = primaryBackgroundToolOptions; } - const primaryIntentToolOptions: AgentToolOptions | undefined = synthesizeIntentToolOptions( - result.tools as string[], - { ephemeralAgent, modelSpec }, - ); + const primaryIntentToolOptions: AgentToolOptions | undefined = synthesizeIntentToolOptions({ + ephemeralAgent, + modelSpec, + }); if (primaryIntentToolOptions) { result.tool_options = mergeSynthesizedToolOptions( result.tool_options as AgentToolOptions | undefined, @@ -286,14 +287,14 @@ export async function loadAddedAgent( applyModelSpecSubagents(result, modelSpec); applyModelSpecSkills(result, modelSpec); - const backgroundToolOptions: AgentToolOptions | undefined = synthesizeBackgroundToolOptions( - tools, - { ephemeralAgent, modelSpec }, - ); + const backgroundToolOptions: AgentToolOptions | undefined = synthesizeBackgroundToolOptions({ + ephemeralAgent, + modelSpec, + }); if (backgroundToolOptions) { result.tool_options = backgroundToolOptions; } - const intentToolOptions: AgentToolOptions | undefined = synthesizeIntentToolOptions(tools, { + const intentToolOptions: AgentToolOptions | undefined = synthesizeIntentToolOptions({ ephemeralAgent, modelSpec, }); diff --git a/packages/api/src/agents/background.spec.ts b/packages/api/src/agents/background.spec.ts index d0d0c42194..0e71028955 100644 --- a/packages/api/src/agents/background.spec.ts +++ b/packages/api/src/agents/background.spec.ts @@ -1,3 +1,4 @@ +import { logger } from '@librechat/data-schemas'; import type { LCTool, LCToolRegistry } from '@librechat/agents'; import { isBackgroundEligibleToolName, @@ -17,6 +18,7 @@ import { CHECK_BACKGROUND_TASK_NAME, RUN_IN_BACKGROUND_ARG, } from './background'; +import { TOOL_SELECTION_WILDCARD } from './selection'; import { toolOptionsSchema } from './validation'; const mcpDef = (name: string): LCTool => @@ -292,33 +294,159 @@ describe('registerBackgroundTaskTool', () => { describe('synthesizeBackgroundToolOptions', () => { it('returns undefined when neither the ephemeral toggle nor the model spec enables it', () => { - expect(synthesizeBackgroundToolOptions(['search_mcp_docs'], {})).toBeUndefined(); + expect(synthesizeBackgroundToolOptions({})).toBeUndefined(); expect( - synthesizeBackgroundToolOptions(['search_mcp_docs'], { + synthesizeBackgroundToolOptions({ ephemeralAgent: { run_in_background: false }, modelSpec: { runInBackground: false }, }), ).toBeUndefined(); }); - it('marks only eligible tools (excludes HITL/attachment built-ins; code tools are eligible)', () => { - const options = synthesizeBackgroundToolOptions( - ['search_mcp_docs', 'execute_code', 'ask_user_question', 'web_search', 'lookup_customer'], - { ephemeralAgent: { run_in_background: true } }, + it('records boolean/ephemeral modes as a wildcard opt-in (no name enumeration)', () => { + const expected = { [TOOL_SELECTION_WILDCARD]: { run_in_background: true } }; + expect(synthesizeBackgroundToolOptions({ modelSpec: { runInBackground: true } })).toEqual( + expected, ); - expect(options).toEqual({ - search_mcp_docs: { run_in_background: true }, + expect( + synthesizeBackgroundToolOptions({ ephemeralAgent: { run_in_background: true } }), + ).toEqual(expected); + }); + + it('records a list as a wildcard opt-out plus verbatim opt-ins', () => { + expect( + synthesizeBackgroundToolOptions({ + modelSpec: { runInBackground: ['slow_report_mcp_analytics', 'execute_code'] }, + }), + ).toEqual({ + [TOOL_SELECTION_WILDCARD]: { run_in_background: false }, + slow_report_mcp_analytics: { run_in_background: true }, execute_code: { run_in_background: true }, - lookup_customer: { run_in_background: true }, }); }); - it('returns undefined when nothing is eligible', () => { + it('treats an empty list as enabling nothing', () => { + expect(synthesizeBackgroundToolOptions({ modelSpec: { runInBackground: [] } })).toEqual({ + [TOOL_SELECTION_WILDCARD]: { run_in_background: false }, + }); + }); + + it('drops and warns about a literal wildcard in the list (reserved)', () => { + /** `runInBackground: ['*']` would otherwise overwrite the opt-out default + * and detach-enable every eligible tool instead of selecting one. */ + const warn = jest.spyOn(logger, 'warn').mockImplementation(() => logger); expect( - synthesizeBackgroundToolOptions(['read_file', 'skill'], { - modelSpec: { runInBackground: true }, + synthesizeBackgroundToolOptions({ + modelSpec: { runInBackground: [TOOL_SELECTION_WILDCARD, 'search_mcp_docs'] }, }), - ).toBeUndefined(); + ).toEqual({ + [TOOL_SELECTION_WILDCARD]: { run_in_background: false }, + search_mcp_docs: { run_in_background: true }, + }); + expect(warn).toHaveBeenCalledWith(expect.stringContaining('reserved')); + warn.mockRestore(); + }); + + it('the ephemeral toggle stays global even when the spec narrows', () => { + expect( + synthesizeBackgroundToolOptions({ + ephemeralAgent: { run_in_background: true }, + modelSpec: { runInBackground: ['search_mcp_docs'] }, + }), + ).toEqual({ [TOOL_SELECTION_WILDCARD]: { run_in_background: true } }); + }); +}); + +describe('selection policy at injection time', () => { + it('a wildcard opt-in reaches eligible definitions and skips excluded built-ins', () => { + const toolOptions = synthesizeBackgroundToolOptions({ modelSpec: { runInBackground: true } }); + const { backgroundToolNames } = applyBackgroundToolCalls({ + toolDefinitions: [ + mcpDef('search_mcp_overlay_server'), + mcpDef('web_search'), + mcpDef('ask_user_question'), + ], + toolRegistry: undefined, + toolOptions, + }); + expect(backgroundToolNames).toEqual(['search_mcp_overlay_server']); + }); + + it('rejects and diagnoses a marker whose runtime definitions are all excluded', () => { + /** `runInBackground: ['memory']` used to record a successful-looking + * option under the marker while set_memory/delete_memory β€” the + * definitions it expands into β€” are background-excluded; nothing + * consumed the entry and nothing warned. */ + const warn = jest.spyOn(logger, 'warn').mockImplementation(() => logger); + const toolOptions = synthesizeBackgroundToolOptions({ + modelSpec: { runInBackground: ['memory'] }, + }); + const { backgroundToolNames } = applyBackgroundToolCalls({ + toolDefinitions: [mcpDef('set_memory'), mcpDef('delete_memory')], + toolRegistry: undefined, + toolOptions, + capabilityToolNames: new Map([['memory', ['set_memory', 'delete_memory']]]), + }); + expect(backgroundToolNames).toEqual([]); + expect(warn).toHaveBeenCalledWith(expect.stringContaining('memory')); + warn.mockRestore(); + }); + + it('projects a saved-agent execute_code entry onto the bash_tool definition', () => { + const { backgroundToolNames } = applyBackgroundToolCalls({ + toolDefinitions: [mcpDef('bash_tool')], + toolRegistry: undefined, + toolOptions: { execute_code: { run_in_background: true } }, + capabilityToolNames: new Map([['execute_code', ['read_file', 'bash_tool']]]), + }); + expect(backgroundToolNames).toEqual(['bash_tool']); + }); + + it('still enforces eligibility for explicitly named tools, and diagnoses them', () => { + /** Backgrounding these would silently drop attachments/citations or break + * artifact continuity, so a list must not be able to force them on. */ + const warn = jest.spyOn(logger, 'warn').mockImplementation(() => logger); + const toolOptions = synthesizeBackgroundToolOptions({ + modelSpec: { runInBackground: ['search_mcp_docs', 'web_search', 'ask_user_question'] }, + }); + const { backgroundToolNames } = applyBackgroundToolCalls({ + toolDefinitions: [ + mcpDef('search_mcp_docs'), + mcpDef('web_search'), + mcpDef('ask_user_question'), + ], + toolRegistry: undefined, + toolOptions, + }); + expect(backgroundToolNames).toEqual(['search_mcp_docs']); + expect(warn).toHaveBeenCalledWith(expect.stringContaining('web_search')); + warn.mockRestore(); + }); + + it('warns about selection names the spec does not equip, rather than silently skipping', () => { + const warn = jest.spyOn(logger, 'warn').mockImplementation(() => logger); + const toolOptions = synthesizeBackgroundToolOptions({ + modelSpec: { runInBackground: ['search_mcp_docs', 'typo_tool_name'] }, + }); + const { backgroundToolNames } = applyBackgroundToolCalls({ + toolDefinitions: [mcpDef('search_mcp_docs')], + toolRegistry: undefined, + toolOptions, + }); + expect(backgroundToolNames).toEqual(['search_mcp_docs']); + expect(warn).toHaveBeenCalledWith(expect.stringContaining('typo_tool_name')); + warn.mockRestore(); + }); + + it('does not warn about saved-agent options with no narrowing policy', () => { + const warn = jest.spyOn(logger, 'warn').mockImplementation(() => logger); + applyBackgroundToolCalls({ + toolDefinitions: [mcpDef('search_mcp_docs')], + toolRegistry: undefined, + toolOptions: { stale_tool: { run_in_background: true } }, + }); + expect(warn).not.toHaveBeenCalled(); + warn.mockRestore(); }); }); @@ -762,25 +890,6 @@ describe('BackgroundTaskRegistryClass', () => { }); }); -describe('applyBackgroundToolCalls β€” code-pair expansion', () => { - it('an execute_code opt-in covers the runtime bash_tool definition', () => { - const defs = [mcpDef('bash_tool')]; - const registry: LCToolRegistry = new Map(defs.map((d) => [d.name, { ...d }])); - const result = applyBackgroundToolCalls({ - toolDefinitions: defs, - toolRegistry: registry, - toolOptions: { execute_code: { run_in_background: true } }, - }); - expect(result.backgroundToolNames).toEqual(['bash_tool']); - const bashDef = result.toolDefinitions.find((d) => d.name === 'bash_tool'); - expect( - (bashDef?.parameters as { properties: Record }).properties[ - RUN_IN_BACKGROUND_ARG - ], - ).toBeDefined(); - }); -}); - describe('getBackgroundCodeDelivery (singleton)', () => { it('exposes harvest state for a settled task and stays available across polls', () => { const created = backgroundTaskRegistry.create({ diff --git a/packages/api/src/agents/background.ts b/packages/api/src/agents/background.ts index 8dd2683496..6b7ded14a8 100644 --- a/packages/api/src/agents/background.ts +++ b/packages/api/src/agents/background.ts @@ -33,6 +33,13 @@ import { Constants as AgentConstants } from '@librechat/agents'; import { Tools, Constants, imageGenTools } from 'librechat-data-provider'; import type { LCTool, LCToolRegistry, JsonSchemaType } from '@librechat/agents'; import type { AgentToolOptions } from 'librechat-data-provider'; +import type { CapabilityToolNames } from './selection'; +import { + resolveToolOption, + getSelectionNames, + warnUnmatchedSelectionNames, + synthesizeSelectionToolOptions, +} from './selection'; import { SET_MEMORY_TOOL_NAME, DELETE_MEMORY_TOOL_NAME } from './memory'; import { ASK_USER_QUESTION_TOOL_NAME } from './hitl/askUserQuestionTool'; import { CREATE_FILE_TOOL_NAME, EDIT_FILE_TOOL_NAME } from './tools'; @@ -41,6 +48,9 @@ import { truncateMiddle } from '~/utils'; /** Argument the model sets on a tool call to dispatch it in the background. */ export const RUN_IN_BACKGROUND_ARG = 'run_in_background'; +/** Log prefix for selection diagnostics, phrased in the spec's own field name. */ +const BACKGROUND_SELECTION_LABEL = '[background] runInBackground'; + /** * `type` of the synthetic attachment emitted on a poll turn when a harvested * code task settles β€” the live "this backgrounded call finished" signal for @@ -94,37 +104,6 @@ const EXCLUDED_BACKGROUND_TOOL_NAMES: ReadonlySet = new Set([ 'image_edit_oai', ]); -/** - * The `execute_code` capability marker expands into the `bash_tool` definition - * at load time (there is one code-execution tool path end-to-end), so a code - * background opt-in keyed by EITHER name covers the pair. Synthesized - * ephemeral/model-spec options and hand-edited agents typically carry only the - * `execute_code` key; without this the actual runtime def (`bash_tool`) would - * silently never receive the injected param. - */ -function expandCodeToolOptions(toolOptions?: AgentToolOptions): AgentToolOptions | undefined { - if (!toolOptions) { - return toolOptions; - } - const codeOptIn = - toolOptions[AgentConstants.EXECUTE_CODE]?.run_in_background === true || - toolOptions[AgentConstants.BASH_TOOL]?.run_in_background === true; - if (!codeOptIn) { - return toolOptions; - } - return { - ...toolOptions, - [AgentConstants.EXECUTE_CODE]: { - ...toolOptions[AgentConstants.EXECUTE_CODE], - run_in_background: true, - }, - [AgentConstants.BASH_TOOL]: { - ...toolOptions[AgentConstants.BASH_TOOL], - run_in_background: true, - }, - }; -} - /** * Whether a tool may be dispatched in the background. Handoff tools * (`lc_transfer_to_*`) run through the direct path and are excluded by prefix. @@ -362,14 +341,21 @@ export function registerBackgroundTaskTool(params: { * Injects the `run_in_background` param into every opted-in, eligible tool and * registers the poll tool when at least one tool became backgroundable. * - * Opt-in is per tool via `tool_options[name].run_in_background`. Both saved - * agents and ephemeral/model-spec agents reach this with `tool_options` - * populated, so the logic is written once. + * Opt-in resolves per FINAL definition via {@link resolveToolOption} + * (explicit name β†’ capability marker projection β†’ wildcard), so a saved + * agent's `execute_code` entry reaches `bash_tool` and a spec selection + * reaches lazily-registered definitions. Both saved agents and + * ephemeral/model-spec agents reach this with `tool_options` populated, so + * the logic is written once. When a narrowing selection is present, names + * that never took effect β€” including markers whose every runtime definition + * is background-excluded, like `memory` β€” are warned about here. */ export function applyBackgroundToolCalls(params: { toolDefinitions: LCTool[] | undefined; toolRegistry: LCToolRegistry | undefined; toolOptions: AgentToolOptions | undefined; + /** Capability marker β†’ registered definition names, from `initializeAgent`. */ + capabilityToolNames?: CapabilityToolNames; /** * Extra host-context exclusion (e.g. tools of ephemeral request-scoped MCP * servers, whose connection dies at request end): a `true` return skips the @@ -378,17 +364,27 @@ export function applyBackgroundToolCalls(params: { */ excludeTool?: (toolName: string) => boolean; }): { toolDefinitions: LCTool[]; backgroundToolNames: string[] } { - const { toolRegistry, excludeTool } = params; - const toolOptions = expandCodeToolOptions(params.toolOptions); + const { toolRegistry, toolOptions, capabilityToolNames, excludeTool } = params; const defs = params.toolDefinitions ?? []; if (!toolOptions || !Object.values(toolOptions).some((o) => o?.run_in_background === true)) { return { toolDefinitions: defs, backgroundToolNames: [] }; } + const selectionNames = getSelectionNames(toolOptions, 'run_in_background'); + const effectiveSources = new Set(); const backgroundToolNames: string[] = []; const nextDefs = defs.map((def) => { - const optedIn = toolOptions[def.name]?.run_in_background === true; - if (!optedIn || !isBackgroundEligibleToolName(def.name) || excludeTool?.(def.name) === true) { + const resolved = resolveToolOption( + def.name, + 'run_in_background', + toolOptions, + capabilityToolNames, + ); + if ( + resolved?.value !== true || + !isBackgroundEligibleToolName(def.name) || + excludeTool?.(def.name) === true + ) { return def; } if (!canInjectRunInBackgroundParam(def)) { @@ -397,6 +393,7 @@ export function applyBackgroundToolCalls(params: { ); return def; } + effectiveSources.add(resolved.source); backgroundToolNames.push(def.name); const injected = injectRunInBackgroundParam(def); if (injected === def) { @@ -409,6 +406,8 @@ export function applyBackgroundToolCalls(params: { return injected; }); + warnUnmatchedSelectionNames(selectionNames, effectiveSources, BACKGROUND_SELECTION_LABEL); + if (backgroundToolNames.length === 0) { return { toolDefinitions: defs, backgroundToolNames: [] }; } @@ -418,36 +417,33 @@ export function applyBackgroundToolCalls(params: { } /** - * Builds `tool_options` marking each eligible tool as backgroundable. Ephemeral - * and model-spec agents carry no `tool_options`, so the blanket spec/ephemeral - * toggle is expanded per-tool here to reuse the same per-tool opt-in the saved - * agent path uses. Returns undefined when disabled or nothing is eligible. + * Records the background selection for ephemeral and model-spec agents, which + * carry no per-tool options of their own. Returns undefined when disabled. * - * Note: MCP servers that expand lazily (via the `mcp_all` placeholder for - * overlay/user-connection servers) are not known by name at this point, so - * their tools are not marked; standard cached MCP servers push real names and - * are covered. + * A model spec's `runInBackground` selects the scope: `true` opts in every + * eligible tool, while a string array opts in ONLY the named ones. Selecting + * per tool matters more here than for intent labels β€” backgrounding changes + * execution semantics, so an admin may want it on one slow MCP call without + * letting the model detach every other tool in the spec. The ephemeral toggle + * stays boolean and never narrows; it has no per-tool UI to drive it. + * + * The selection is recorded as policy (wildcard default + verbatim names) + * and resolved against the FINAL definition set in + * `applyBackgroundToolCalls`, so capability markers and lazily-expanded MCP + * servers are governed, and names that never take effect β€” a typo, or a + * marker like `memory` whose runtime definitions are all + * background-excluded β€” are diagnosed where the real definitions are known. */ -export function synthesizeBackgroundToolOptions( - tools: string[], - sources: { - ephemeralAgent?: { run_in_background?: boolean } | null; - modelSpec?: { runInBackground?: boolean } | null; - }, -): AgentToolOptions | undefined { - const enabled = - sources.ephemeralAgent?.run_in_background === true || - sources.modelSpec?.runInBackground === true; - if (!enabled) { - return undefined; - } - const toolOptions: AgentToolOptions = {}; - for (const name of tools) { - if (isBackgroundEligibleToolName(name)) { - toolOptions[name] = { run_in_background: true }; - } - } - return Object.keys(toolOptions).length > 0 ? toolOptions : undefined; +export function synthesizeBackgroundToolOptions(sources: { + ephemeralAgent?: { run_in_background?: boolean } | null; + modelSpec?: { runInBackground?: boolean | string[] } | null; +}): AgentToolOptions | undefined { + return synthesizeSelectionToolOptions( + 'run_in_background', + sources.modelSpec?.runInBackground, + sources.ephemeralAgent?.run_in_background === true, + BACKGROUND_SELECTION_LABEL, + ); } export type BackgroundTaskStatus = 'running' | 'completed' | 'error'; diff --git a/packages/api/src/agents/initialize.ts b/packages/api/src/agents/initialize.ts index d9118f23c6..f70843b994 100644 --- a/packages/api/src/agents/initialize.ts +++ b/packages/api/src/agents/initialize.ts @@ -8,6 +8,7 @@ import { EToolResources, paramEndpoints, isAgentsEndpoint, + AgentCapabilities, replaceSpecialVars, providerEndpointMap, } from 'librechat-data-provider'; @@ -1112,6 +1113,22 @@ export async function initializeAgent( */ const agentRequestsCodeExec = (agent.tools ?? []).includes(Tools.execute_code); const effectiveCodeEnvAvailable = params.codeEnvAvailable === true && agentRequestsCodeExec; + /** + * Capability marker β†’ definition names its registration produced this run, + * reported by the registrars themselves. `tool_options` entries keyed by a + * marker (`execute_code`, `memory`, `skills`) β€” from a model-spec selection + * or a hand-edited saved agent β€” resolve onto exactly these names in the + * background/intent passes, so the projection cannot drift from what + * actually got registered. + */ + const capabilityToolNames = new Map(); + const recordCapabilityToolNames = (capability: string, toolNames: readonly string[]): void => { + if (toolNames.length === 0) { + return; + } + const existing = capabilityToolNames.get(capability); + capabilityToolNames.set(capability, existing ? [...existing, ...toolNames] : toolNames); + }; /** Per-agent stateful-session truth: the admin capability AND the agent's * own builder opt-in AND a working code env. Resolved once here so the * registered bash description, the tool factories, and `createRun`'s @@ -1130,6 +1147,7 @@ export async function initializeAgent( statefulSessions: effectiveStatefulSessions, }); toolDefinitions = codeExecResult.toolDefinitions; + recordCapabilityToolNames(AgentCapabilities.execute_code, codeExecResult.toolNames); } else if (agentRequestsCodeExec) { /** * Agent asked for `execute_code` but the admin-level gate is off β€” @@ -1160,6 +1178,7 @@ export async function initializeAgent( validKeys: req.config?.memory?.validKeys, }); toolDefinitions = memoryResult.toolDefinitions; + recordCapabilityToolNames(AgentCapabilities.memory, memoryResult.toolNames); appendAdditionalInstructions(agent, memoryToolUsageGuard); } else if (agentRequestsMemory) { logger.debug( @@ -1176,6 +1195,7 @@ export async function initializeAgent( enableToolOutputReferences: effectiveCodeEnvAvailable, }); toolDefinitions = skillReadResult.toolDefinitions; + recordCapabilityToolNames(AgentCapabilities.skills, skillReadResult.toolNames); } if (effectiveCodeEnvAvailable || skillAuthoringAvailable) { @@ -1185,6 +1205,14 @@ export async function initializeAgent( includeSkillFileInstructions: skillAuthoringAvailable, }); toolDefinitions = fileAuthoringResult.toolDefinitions; + /** File authoring is owned by whichever capability switched it on β€” + * both, when both are active, so either marker's selection governs. */ + if (effectiveCodeEnvAvailable) { + recordCapabilityToolNames(AgentCapabilities.execute_code, fileAuthoringResult.toolNames); + } + if (skillAuthoringAvailable) { + recordCapabilityToolNames(AgentCapabilities.skills, fileAuthoringResult.toolNames); + } } let intentToolNames: string[] | undefined; @@ -1214,6 +1242,7 @@ export async function initializeAgent( toolDefinitions, toolRegistry, toolOptions: agent.tool_options, + capabilityToolNames, /** Tools of ephemeral request-scoped MCP servers (runtime body * placeholders) never get the param: their connection dies at request * end, so the executor would only downgrade the call to foreground. @@ -1324,6 +1353,7 @@ export async function initializeAgent( skillCount = skillResult.skillCount; executableSkillIds = skillResult.activeSkillIds; activeSkillNames = skillResult.activeSkillNames; + recordCapabilityToolNames(AgentCapabilities.skills, skillResult.toolNames); } /** @@ -1343,6 +1373,7 @@ export async function initializeAgent( toolDefinitions, toolRegistry, toolOptions: agent.tool_options, + capabilityToolNames, }); toolDefinitions = intentResult.toolDefinitions; if (intentResult.intentToolNames.length > 0) { @@ -1354,6 +1385,7 @@ export async function initializeAgent( toolRegistry, toolOptions: agent.tool_options, capabilityEnabled: params.toolIntentsAvailable === true, + capabilityToolNames, }); toolDefinitions = intentSanitized.toolDefinitions; diff --git a/packages/api/src/agents/intent.spec.ts b/packages/api/src/agents/intent.spec.ts index 5c894c56af..28f3e4af7d 100644 --- a/packages/api/src/agents/intent.spec.ts +++ b/packages/api/src/agents/intent.spec.ts @@ -1,4 +1,4 @@ -import { Constants } from 'librechat-data-provider'; +import { logger } from '@librechat/data-schemas'; import type { LCTool, LCToolRegistry } from '@librechat/agents'; import { INTENT_ARG, @@ -14,9 +14,9 @@ import { applyIntentLabels, sanitizeIntentLabels, synthesizeIntentToolOptions, - mergeSynthesizedToolOptions, } from './intent'; import { applyBackgroundToolCalls, CHECK_BACKGROUND_TASK_NAME } from './background'; +import { mergeSynthesizedToolOptions, TOOL_SELECTION_WILDCARD } from './selection'; import { toolOptionsSchema } from './validation'; const mcpDef = (name: string): LCTool => @@ -45,9 +45,13 @@ const sdkNativeDef = (name: string): LCTool => }) as unknown as LCTool; describe('isIntentEligibleToolName', () => { - it('excludes only the poll tool and handoff tools', () => { + it('excludes the poll tool, handoff tools, and the rebuilt ask tool', () => { expect(isIntentEligibleToolName(CHECK_BACKGROUND_TASK_NAME)).toBe(false); expect(isIntentEligibleToolName('lc_transfer_to_researcher')).toBe(false); + /** `createRun` strips this definition and rebuilds the graph tool from + * its own Zod schema, so definition-level injection never reaches the + * model; eligibility must say so or a selection credits a dead label. */ + expect(isIntentEligibleToolName('ask_user_question')).toBe(false); }); it('allows MCP, native, and code-execution tools (labels are inert)', () => { @@ -58,7 +62,6 @@ describe('isIntentEligibleToolName', () => { 'edit_file', 'set_memory', 'delete_memory', - 'ask_user_question', 'execute_code', 'bash_tool', 'file_search', @@ -394,6 +397,108 @@ describe('stripIntentFromToolDefinitions / stripIntentFromToolRegistry', () => { }); }); +describe('capability marker projection', () => { + const memoryDefs = (): LCTool[] => [mcpDef('set_memory'), mcpDef('delete_memory')]; + /** What `initializeAgent` records from the registrars' own reports. */ + const MEMORY_MAP = new Map([['memory', ['set_memory', 'delete_memory']]]); + const CODE_MAP = new Map([ + ['execute_code', ['read_file', 'bash_tool', 'create_file', 'edit_file']], + ]); + + it('projects a marker OPT-OUT onto the names its capability registered', () => { + /** A spec's tools carry `memory`, but initialization registers + * set_memory/delete_memory β€” both default-on natives, so without + * projection a marker opt-out would leave them labelled. */ + const { intentToolNames } = applyIntentLabels({ + toolDefinitions: memoryDefs(), + toolRegistry: undefined, + toolOptions: { memory: { describe_intent: false } }, + capabilityToolNames: MEMORY_MAP, + }); + expect(intentToolNames).toEqual([]); + }); + + it('projects a marker OPT-IN onto the names its capability registered', () => { + const { intentToolNames } = applyIntentLabels({ + toolDefinitions: [mcpDef('bash_tool')], + toolRegistry: undefined, + toolOptions: { execute_code: { describe_intent: true } }, + capabilityToolNames: CODE_MAP, + }); + expect(intentToolNames).toEqual(['bash_tool']); + }); + + it('covers file-authoring tools registered by the code capability', () => { + /** create_file/edit_file are default-on natives the code capability + * registers, so a code opt-out must reach them, not just bash_tool. */ + const { intentToolNames } = applyIntentLabels({ + toolDefinitions: [mcpDef('create_file'), mcpDef('edit_file')], + toolRegistry: undefined, + toolOptions: { execute_code: { describe_intent: false } }, + capabilityToolNames: CODE_MAP, + }); + expect(intentToolNames).toEqual([]); + }); + + it('lets an explicit per-tool entry win over the marker', () => { + const { intentToolNames } = applyIntentLabels({ + toolDefinitions: memoryDefs(), + toolRegistry: undefined, + toolOptions: { + memory: { describe_intent: false }, + set_memory: { describe_intent: true }, + }, + capabilityToolNames: MEMORY_MAP, + }); + expect(intentToolNames).toEqual(['set_memory']); + }); + + it('lets an opting-in marker win over an opting-out one for a shared tool', () => { + /** read_file registers under both code and skills; opting into skills + * must not be vetoed by a code opt-out. */ + const shared = new Map([ + ['execute_code', ['read_file', 'bash_tool']], + ['skills', ['skill', 'read_file']], + ]); + const { intentToolNames } = applyIntentLabels({ + toolDefinitions: [mcpDef('read_file')], + toolRegistry: undefined, + toolOptions: { + execute_code: { describe_intent: false }, + skills: { describe_intent: true }, + }, + capabilityToolNames: shared, + }); + expect(intentToolNames).toEqual(['read_file']); + }); + + it('carries a marker opt-out through the capability-on sanitize pass', () => { + /** SDK-native labels persist unless sanitize resolves an opt-out. */ + const nativeBash = sdkNativeDef('bash_tool'); + const { toolDefinitions } = sanitizeIntentLabels({ + toolDefinitions: [nativeBash], + toolRegistry: undefined, + toolOptions: { execute_code: { describe_intent: false } }, + capabilityEnabled: true, + capabilityToolNames: CODE_MAP, + }); + expect(INTENT_ARG in (toolDefinitions[0].parameters as { properties: object }).properties).toBe( + false, + ); + }); + + it('leaves options untouched when no marker is present', () => { + const options = { web_search: { describe_intent: true } }; + const { intentToolNames } = applyIntentLabels({ + toolDefinitions: [mcpDef('web_search')], + toolRegistry: undefined, + toolOptions: options, + }); + expect(intentToolNames).toEqual(['web_search']); + expect(options).toEqual({ web_search: { describe_intent: true } }); + }); +}); + describe('sanitizeIntentLabels', () => { it('strips every SDK-native label when the capability is disabled (kill switch)', () => { const skill = sdkNativeDef('skill'); @@ -471,37 +576,207 @@ describe('sanitizeIntentLabels', () => { describe('synthesizeIntentToolOptions', () => { it('returns undefined when neither the ephemeral toggle nor the model spec enables it', () => { - expect(synthesizeIntentToolOptions(['web_search'], {})).toBeUndefined(); + expect(synthesizeIntentToolOptions({})).toBeUndefined(); expect( - synthesizeIntentToolOptions(['web_search'], { + synthesizeIntentToolOptions({ ephemeralAgent: { describe_intent: false }, modelSpec: { describeIntent: false }, }), ).toBeUndefined(); }); - it('marks only eligible tools', () => { - const options = synthesizeIntentToolOptions( - ['web_search', CHECK_BACKGROUND_TASK_NAME, 'lc_transfer_to_researcher'], - { ephemeralAgent: { describe_intent: true } }, + it('records boolean/ephemeral modes as a wildcard opt-in (no name enumeration)', () => { + const expected = { [TOOL_SELECTION_WILDCARD]: { describe_intent: true } }; + expect(synthesizeIntentToolOptions({ modelSpec: { describeIntent: true } })).toEqual(expected); + expect(synthesizeIntentToolOptions({ ephemeralAgent: { describe_intent: true } })).toEqual( + expected, ); - expect(options).toEqual({ web_search: { describe_intent: true } }); }); - it('skips lazily-expanded mcp_all placeholders (exact-name matching would never apply)', () => { - const placeholder = `${Constants.mcp_all}${Constants.mcp_delimiter}overlay_server`; - const options = synthesizeIntentToolOptions([placeholder, 'web_search'], { - ephemeralAgent: { describe_intent: true }, - }); - expect(options).toEqual({ web_search: { describe_intent: true } }); - }); - - it('returns undefined when nothing is eligible', () => { + it('records a list as a wildcard opt-out plus verbatim opt-ins', () => { + /** Names are recorded verbatim β€” markers, late-registered definitions, + * and lazily-expanded MCP names all resolve at injection time, where the + * final definitions exist. */ expect( - synthesizeIntentToolOptions([CHECK_BACKGROUND_TASK_NAME], { - modelSpec: { describeIntent: true }, + synthesizeIntentToolOptions({ + modelSpec: { describeIntent: ['web_search', 'execute_code'] }, }), - ).toBeUndefined(); + ).toEqual({ + [TOOL_SELECTION_WILDCARD]: { describe_intent: false }, + web_search: { describe_intent: true }, + execute_code: { describe_intent: true }, + }); + }); + + it('treats an empty list as an explicit none', () => { + expect(synthesizeIntentToolOptions({ modelSpec: { describeIntent: [] } })).toEqual({ + [TOOL_SELECTION_WILDCARD]: { describe_intent: false }, + }); + }); + + it('drops and warns about a literal wildcard in the list (reserved)', () => { + /** A verbatim `*` entry would overwrite the opt-out default and silently + * enable every tool instead of the named selection. */ + const warn = jest.spyOn(logger, 'warn').mockImplementation(() => logger); + expect( + synthesizeIntentToolOptions({ + modelSpec: { describeIntent: [TOOL_SELECTION_WILDCARD, 'web_search'] }, + }), + ).toEqual({ + [TOOL_SELECTION_WILDCARD]: { describe_intent: false }, + web_search: { describe_intent: true }, + }); + expect(warn).toHaveBeenCalledWith(expect.stringContaining('reserved')); + warn.mockRestore(); + }); + + it('the ephemeral toggle stays global even when the spec narrows', () => { + /** The ephemeral switch has no per-tool UI, so it must not be narrowed by + * a co-present spec list β€” otherwise enabling it would silently cover + * fewer tools than the user asked for. */ + expect( + synthesizeIntentToolOptions({ + ephemeralAgent: { describe_intent: true }, + modelSpec: { describeIntent: ['web_search'] }, + }), + ).toEqual({ [TOOL_SELECTION_WILDCARD]: { describe_intent: true } }); + }); +}); + +describe('selection policy at injection time', () => { + it('a narrowing selection opts out defaults and SDK-native labels alike', () => { + /** `describeIntent: []` with code enabled must reach EVERY definition the + * capability registered β€” bash_tool/read_file (SDK-native labels) and + * create_file/edit_file (default-on natives) β€” not just a legacy pair. */ + const toolOptions = synthesizeIntentToolOptions({ modelSpec: { describeIntent: [] } }); + const capabilityToolNames = new Map([ + ['execute_code', ['read_file', 'bash_tool', 'create_file', 'edit_file']], + ]); + const applied = applyIntentLabels({ + toolDefinitions: [ + sdkNativeDef('bash_tool'), + sdkNativeDef('read_file'), + mcpDef('create_file'), + mcpDef('edit_file'), + ], + toolRegistry: undefined, + toolOptions, + capabilityToolNames, + }); + expect(applied.intentToolNames).toEqual([]); + const { toolDefinitions } = sanitizeIntentLabels({ + toolDefinitions: applied.toolDefinitions, + toolRegistry: undefined, + toolOptions, + capabilityEnabled: true, + capabilityToolNames, + }); + for (const def of toolDefinitions) { + expect(INTENT_ARG in (def.parameters as { properties: object }).properties).toBe(false); + } + }); + + it('a narrowing selection governs the late-registered skill definition', () => { + /** A spec's `skills` never reaches the `tools` array, so no load-time + * enumeration could see it; the wildcard opt-out reaches the definition + * in the post-catalog sanitize pass. */ + const toolOptions = synthesizeIntentToolOptions({ + modelSpec: { describeIntent: ['web_search'] }, + }); + const { toolDefinitions } = sanitizeIntentLabels({ + toolDefinitions: [sdkNativeDef('skill')], + toolRegistry: undefined, + toolOptions, + capabilityEnabled: true, + capabilityToolNames: new Map([['skills', ['skill', 'read_file']]]), + }); + expect(INTENT_ARG in (toolDefinitions[0].parameters as { properties: object }).properties).toBe( + false, + ); + }); + + it('naming `skills` in the selection keeps the skill label', () => { + const toolOptions = synthesizeIntentToolOptions({ modelSpec: { describeIntent: ['skills'] } }); + const { toolDefinitions } = sanitizeIntentLabels({ + toolDefinitions: [sdkNativeDef('skill')], + toolRegistry: undefined, + toolOptions, + capabilityEnabled: true, + capabilityToolNames: new Map([['skills', ['skill', 'read_file']]]), + }); + expect(INTENT_ARG in (toolDefinitions[0].parameters as { properties: object }).properties).toBe( + true, + ); + }); + + it('a wildcard opt-in covers tools unknown at load time (lazy MCP expansion)', () => { + const toolOptions = synthesizeIntentToolOptions({ modelSpec: { describeIntent: true } }); + const { intentToolNames } = applyIntentLabels({ + toolDefinitions: [mcpDef('search_mcp_overlay_server')], + toolRegistry: undefined, + toolOptions, + }); + expect(intentToolNames).toEqual(['search_mcp_overlay_server']); + }); + + it('diagnoses a selection naming the rebuilt ask tool instead of crediting a dead label', () => { + /** The provisional `ask_user_question` definition is discarded by + * `createRun` and rebuilt without an intent field, so a selection + * naming it must warn rather than inject into a definition the model + * never sees. */ + const warn = jest.spyOn(logger, 'warn').mockImplementation(() => logger); + const toolOptions = synthesizeIntentToolOptions({ + modelSpec: { describeIntent: ['ask_user_question'] }, + }); + const { intentToolNames } = applyIntentLabels({ + toolDefinitions: [mcpDef('ask_user_question')], + toolRegistry: undefined, + toolOptions, + }); + expect(intentToolNames).toEqual([]); + expect(warn).toHaveBeenCalledWith(expect.stringContaining('ask_user_question')); + warn.mockRestore(); + }); + + it('still enforces eligibility for explicitly named tools, and diagnoses them', () => { + const warn = jest.spyOn(logger, 'warn').mockImplementation(() => logger); + const toolOptions = synthesizeIntentToolOptions({ + modelSpec: { describeIntent: ['web_search', CHECK_BACKGROUND_TASK_NAME] }, + }); + const { intentToolNames } = applyIntentLabels({ + toolDefinitions: [mcpDef('web_search'), mcpDef(CHECK_BACKGROUND_TASK_NAME)], + toolRegistry: undefined, + toolOptions, + }); + expect(intentToolNames).toEqual(['web_search']); + expect(warn).toHaveBeenCalledWith(expect.stringContaining(CHECK_BACKGROUND_TASK_NAME)); + warn.mockRestore(); + }); + + it('warns about selection names that never took effect, rather than silently skipping', () => { + const warn = jest.spyOn(logger, 'warn').mockImplementation(() => logger); + const toolOptions = synthesizeIntentToolOptions({ + modelSpec: { describeIntent: ['web_search', 'typo_tool_name'] }, + }); + const { intentToolNames } = applyIntentLabels({ + toolDefinitions: [mcpDef('web_search')], + toolRegistry: undefined, + toolOptions, + }); + expect(intentToolNames).toEqual(['web_search']); + expect(warn).toHaveBeenCalledWith(expect.stringContaining('typo_tool_name')); + warn.mockRestore(); + }); + + it('does not warn about saved-agent options with no narrowing policy', () => { + const warn = jest.spyOn(logger, 'warn').mockImplementation(() => logger); + applyIntentLabels({ + toolDefinitions: [mcpDef('web_search')], + toolRegistry: undefined, + toolOptions: { stale_tool: { describe_intent: true } }, + }); + expect(warn).not.toHaveBeenCalled(); + warn.mockRestore(); }); }); diff --git a/packages/api/src/agents/intent.ts b/packages/api/src/agents/intent.ts index f84f6400d0..5989ce95c5 100644 --- a/packages/api/src/agents/intent.ts +++ b/packages/api/src/agents/intent.ts @@ -32,23 +32,28 @@ import { } from '@librechat/agents'; import type { LCTool, LCToolRegistry, JsonSchemaType } from '@librechat/agents'; import type { AgentToolOptions } from 'librechat-data-provider'; +import type { CapabilityToolNames } from './selection'; +import { + resolveToolOption, + getSelectionNames, + warnUnmatchedSelectionNames, + synthesizeSelectionToolOptions, +} from './selection'; import { SET_MEMORY_TOOL_NAME, DELETE_MEMORY_TOOL_NAME } from './memory'; +import { ASK_USER_QUESTION_TOOL_NAME } from './hitl/askUserQuestionTool'; import { CREATE_FILE_TOOL_NAME, EDIT_FILE_TOOL_NAME } from './tools'; /** Argument carrying the model-authored label for a tool call. */ export const INTENT_ARG = 'intent'; +/** Log prefix for selection diagnostics, phrased in the spec's own field name. */ +const INTENT_SELECTION_LABEL = '[intent] describeIntent'; + /** * Host-native tools that default INTO intent labels while the capability is * enabled (an explicit `describe_intent: false` opts one out). These are the * least legible calls in the UI today, and the convention only becomes a * convention if our own tools model it. - * - * `ask_user_question` is deliberately absent: its graph tool is rebuilt in - * `run.ts` from its own Zod schema (which is also the HITL card's wire - * shape), so definition-level injection never reaches the model. Its intent - * support lands with the HITL slice, which threads the label into the - * interrupt payload on purpose. */ export const NATIVE_INTENT_TOOL_NAMES: ReadonlySet = new Set([ Tools.web_search, @@ -61,11 +66,20 @@ export const NATIVE_INTENT_TOOL_NAMES: ReadonlySet = new Set([ /** * Tools that never get the injected param: the background poll tool is host * machinery, and handoff tools run through the direct path where no card - * renders a label. Intent labels are otherwise inert, so β€” unlike - * background's correctness-driven list β€” nothing else is excluded. + * renders a label. + * + * `ask_user_question` is excluded because injection into its definition can + * never reach the model: `createRun` strips the definition and rebuilds the + * graph tool from its own Zod schema (also the HITL card's wire shape). + * Excluding it makes an explicit selection warn as ineligible instead of + * crediting a label that will be discarded. Its intent support lands with + * the HITL slice, which threads the label into the interrupt payload on + * purpose. Intent labels are otherwise inert, so β€” unlike background's + * correctness-driven list β€” nothing else is excluded. */ const EXCLUDED_INTENT_TOOL_NAMES: ReadonlySet = new Set([ String(Constants.CHECK_BACKGROUND_TASK), + ASK_USER_QUESTION_TOOL_NAME, ]); /** Whether a tool may carry an intent label. */ @@ -290,40 +304,37 @@ export function stripIntentFromToolRegistry( return next; } -/** - * Whether a tool is opted into intent labels: an explicit per-tool - * `describe_intent` wins; native host tools default on. - */ -function isIntentOptedIn(name: string, toolOptions?: AgentToolOptions): boolean { - const explicit = toolOptions?.[name]?.describe_intent; - if (explicit != null) { - return explicit === true; - } - return NATIVE_INTENT_TOOL_NAMES.has(name); -} - /** * Injects the `intent` param into every opted-in, eligible tool definition, * mirroring the injection into the registry entry so a deferred tool * discovered later (tool_search reads the registry) arrives with the same * schema. Definitions that already declare `intent` (SDK-native tools) are * left alone and NOT counted as host-injected β€” their schema is their own β€” - * unless the tool is explicitly opted OUT (`describe_intent: false`), in - * which case the property is removed so the opt-out actually disables the - * arg's token cost (the SDK tool bodies tolerate its absence). + * unless the tool is opted OUT, in which case the property is removed so the + * opt-out actually disables the arg's token cost (the SDK tool bodies + * tolerate its absence). * - * Both saved agents and ephemeral/model-spec agents reach this with - * `tool_options` populated, so the logic is written once. + * Opt-in resolves per FINAL definition via {@link resolveToolOption} + * (explicit name β†’ capability marker projection β†’ wildcard), with native + * host tools defaulting on when no policy speaks. Both saved agents and + * ephemeral/model-spec agents reach this with `tool_options` populated, so + * the logic is written once. When a narrowing selection is present, names + * that never took effect on any definition are warned about here β€” the one + * place the final definition set is known. */ export function applyIntentLabels(params: { toolDefinitions: LCTool[] | undefined; toolRegistry: LCToolRegistry | undefined; toolOptions: AgentToolOptions | undefined; + /** Capability marker β†’ registered definition names, from `initializeAgent`. */ + capabilityToolNames?: CapabilityToolNames; /** Extra host-context exclusion, mirroring `applyBackgroundToolCalls`. */ excludeTool?: (toolName: string) => boolean; }): { toolDefinitions: LCTool[]; intentToolNames: string[] } { - const { toolRegistry, toolOptions, excludeTool } = params; + const { toolRegistry, toolOptions, capabilityToolNames, excludeTool } = params; const defs = params.toolDefinitions ?? []; + const selectionNames = getSelectionNames(toolOptions, 'describe_intent'); + const effectiveSources = new Set(); let changed = false; const intentToolNames: string[] = []; @@ -334,7 +345,13 @@ export function applyIntentLabels(params: { } }; const nextDefs = defs.map((def) => { - if (toolOptions?.[def.name]?.describe_intent === false) { + const resolved = resolveToolOption( + def.name, + 'describe_intent', + toolOptions, + capabilityToolNames, + ); + if (resolved?.value === false) { const stripped = removeIntentParam(def); if (stripped !== def) { changed = true; @@ -342,7 +359,7 @@ export function applyIntentLabels(params: { } return stripped; } - if (!isIntentOptedIn(def.name, toolOptions)) { + if (resolved == null && !NATIVE_INTENT_TOOL_NAMES.has(def.name)) { return def; } if (!isIntentEligibleToolName(def.name) || excludeTool?.(def.name) === true) { @@ -357,6 +374,11 @@ export function applyIntentLabels(params: { return def; } const injected = injectIntentParam(def); + /** The selection took effect whether the label was injected here or the + * definition already carries its own (SDK-native) one. */ + if (resolved != null) { + effectiveSources.add(resolved.source); + } if (injected === def) { return def; } @@ -366,14 +388,14 @@ export function applyIntentLabels(params: { return injected; }); + warnUnmatchedSelectionNames(selectionNames, effectiveSources, INTENT_SELECTION_LABEL); + if (!changed) { return { toolDefinitions: defs, intentToolNames }; } return { toolDefinitions: nextDefs, intentToolNames }; } -const MCP_ALL_PLACEHOLDER_PREFIX = `${Constants.mcp_all}${Constants.mcp_delimiter}`; - /** * Post-registration sanitize pass, run AFTER every tool registration step β€” * including the skill catalog, which appends its definition after the @@ -388,11 +410,16 @@ export function sanitizeIntentLabels(params: { toolRegistry: LCToolRegistry | undefined; toolOptions: AgentToolOptions | undefined; capabilityEnabled: boolean; + /** Capability marker β†’ registered definition names, from `initializeAgent`. */ + capabilityToolNames?: CapabilityToolNames; }): { toolDefinitions: LCTool[] } { - const { toolRegistry, toolOptions, capabilityEnabled } = params; + const { toolRegistry, toolOptions, capabilityEnabled, capabilityToolNames } = params; const defs = params.toolDefinitions ?? []; const shouldStrip = (name: string): boolean => - capabilityEnabled ? toolOptions?.[name]?.describe_intent === false : true; + capabilityEnabled + ? resolveToolOption(name, 'describe_intent', toolOptions, capabilityToolNames)?.value === + false + : true; let changed = false; const nextDefs = defs.map((def) => { @@ -424,59 +451,33 @@ export function sanitizeIntentLabels(params: { } /** - * Builds `tool_options` marking each eligible tool as intent-describing for - * ephemeral and model-spec agents, which carry no per-tool options of their - * own. Returns undefined when disabled or nothing is eligible. + * Records the intent selection for ephemeral and model-spec agents, which + * carry no per-tool options of their own. Returns undefined when disabled. * - * Note: MCP servers that expand lazily (via the `mcp_all` placeholder for - * overlay/user-connection servers) are not known by name at this point β€” - * `applyIntentLabels` matches expanded tool names exactly, so an option - * recorded under the placeholder would silently never apply. Those entries - * are skipped rather than synthesized dead; standard cached MCP servers push - * real names and are covered. Mirrors `synthesizeBackgroundToolOptions`. + * A model spec's `describeIntent` selects the scope: `true` opts in every + * eligible tool, while a string array opts in ONLY the named ones β€” a list + * is a SELECTION POLICY, not an additive filter, so everything unselected is + * opted out (natives and SDK-native labels alike), and an empty list is an + * explicit "none". Selecting per tool matters because the label costs schema + * tokens on every request, so an admin may want it on a handful of illegible + * calls rather than the whole toolset. The ephemeral toggle stays boolean + * and never narrows; it has no per-tool UI to drive a selection. + * + * The selection is recorded as policy (wildcard default + verbatim names) + * and resolved against the FINAL definition set in `applyIntentLabels` / + * `sanitizeIntentLabels`, so capability markers (`execute_code`, `memory`), + * spec fields that never reach `tools` (`skills`), and lazily-expanded MCP + * servers are all governed β€” and misspelled or unsupported names are + * diagnosed where the real definitions are known. */ -export function synthesizeIntentToolOptions( - tools: string[], - sources: { - ephemeralAgent?: { describe_intent?: boolean } | null; - modelSpec?: { describeIntent?: boolean } | null; - }, -): AgentToolOptions | undefined { - const enabled = - sources.ephemeralAgent?.describe_intent === true || sources.modelSpec?.describeIntent === true; - if (!enabled) { - return undefined; - } - const toolOptions: AgentToolOptions = {}; - for (const name of tools) { - if (name.startsWith(MCP_ALL_PLACEHOLDER_PREFIX)) { - continue; - } - if (isIntentEligibleToolName(name)) { - toolOptions[name] = { describe_intent: true }; - } - } - return Object.keys(toolOptions).length > 0 ? toolOptions : undefined; -} - -/** - * Deep-merges two synthesized `tool_options` maps per tool key, so the - * ephemeral background and intent toggles compose instead of overwriting - * each other's per-tool entries. - */ -export function mergeSynthesizedToolOptions( - base: AgentToolOptions | undefined, - extra: AgentToolOptions | undefined, -): AgentToolOptions | undefined { - if (!extra) { - return base; - } - if (!base) { - return extra; - } - const merged: AgentToolOptions = { ...base }; - for (const [name, options] of Object.entries(extra)) { - merged[name] = { ...merged[name], ...options }; - } - return merged; +export function synthesizeIntentToolOptions(sources: { + ephemeralAgent?: { describe_intent?: boolean } | null; + modelSpec?: { describeIntent?: boolean | string[] } | null; +}): AgentToolOptions | undefined { + return synthesizeSelectionToolOptions( + 'describe_intent', + sources.modelSpec?.describeIntent, + sources.ephemeralAgent?.describe_intent === true, + INTENT_SELECTION_LABEL, + ); } diff --git a/packages/api/src/agents/load.ts b/packages/api/src/agents/load.ts index 8ba1d4b2a1..b4a73a8f54 100644 --- a/packages/api/src/agents/load.ts +++ b/packages/api/src/agents/load.ts @@ -14,9 +14,10 @@ import type { Agent, } from 'librechat-data-provider'; import type { AppConfig } from '@librechat/data-schemas'; -import { synthesizeIntentToolOptions, mergeSynthesizedToolOptions } from '~/agents/intent'; import { ASK_USER_QUESTION_TOOL_NAME } from '~/agents/hitl/askUserQuestionTool'; import { synthesizeBackgroundToolOptions } from '~/agents/background'; +import { mergeSynthesizedToolOptions } from '~/agents/selection'; +import { synthesizeIntentToolOptions } from '~/agents/intent'; import { requiresEphemeralUserConnection } from '~/mcp/utils'; import { getCustomEndpointConfig } from '~/app/config'; @@ -152,14 +153,14 @@ export async function loadEphemeralAgent( tools, }; - const backgroundToolOptions: AgentToolOptions | undefined = synthesizeBackgroundToolOptions( - tools, - { ephemeralAgent, modelSpec }, - ); + const backgroundToolOptions: AgentToolOptions | undefined = synthesizeBackgroundToolOptions({ + ephemeralAgent, + modelSpec, + }); if (backgroundToolOptions) { result.tool_options = backgroundToolOptions; } - const intentToolOptions: AgentToolOptions | undefined = synthesizeIntentToolOptions(tools, { + const intentToolOptions: AgentToolOptions | undefined = synthesizeIntentToolOptions({ ephemeralAgent, modelSpec, }); diff --git a/packages/api/src/agents/memory.ts b/packages/api/src/agents/memory.ts index b7922ec7dd..0d70928848 100644 --- a/packages/api/src/agents/memory.ts +++ b/packages/api/src/agents/memory.ts @@ -415,8 +415,9 @@ export function registerMemoryTools({ toolRegistry?: LCToolRegistry; toolDefinitions?: LCTool[]; validKeys?: string[]; -}): { toolDefinitions: LCTool[]; registered: string[] } { +}): { toolDefinitions: LCTool[]; registered: string[]; toolNames: string[] } { const memoryToolDefinitions = getMemoryToolDefinitions(validKeys); + const toolNames = memoryToolDefinitions.map((def) => def.name); const inputDefinitions = toolDefinitions ?? []; const newDefs: LCTool[] = []; const registered: string[] = []; @@ -433,9 +434,9 @@ export function registerMemoryTools({ } if (newDefs.length === 0) { - return { toolDefinitions: inputDefinitions, registered }; + return { toolDefinitions: inputDefinitions, registered, toolNames }; } - return { toolDefinitions: [...inputDefinitions, ...newDefs], registered }; + return { toolDefinitions: [...inputDefinitions, ...newDefs], registered, toolNames }; } type GetRoleByName = ( diff --git a/packages/api/src/agents/selection.ts b/packages/api/src/agents/selection.ts new file mode 100644 index 0000000000..d0232a3cdc --- /dev/null +++ b/packages/api/src/agents/selection.ts @@ -0,0 +1,214 @@ +/** + * @fileoverview Tool-selection policy for capability-injected per-tool options. + * + * A model spec / ephemeral agent expresses per-tool capability choices + * (`describeIntent`, `runInBackground`) against its LOAD-TIME tool entries: + * capability markers (`execute_code`, `memory`), lazy `mcp_all` placeholders, + * and fields that never reach the `tools` array at all (`skills`). The + * definitions those choices must govern only exist at INITIALIZATION time, + * after markers expand and late registrations (file authoring, the skill + * catalog) run β€” so any option pre-keyed by a load-time name risks being a + * silent no-op against the final definition set. + * + * This module removes that name-space gap instead of bridging it per case: + * + * - Synthesis records the selection as a POLICY, not per-name entries: a + * wildcard `*` entry carries the default (`true` for "every tool", `false` + * for "only the named ones"), and listed names are recorded verbatim as + * explicit opt-ins. + * - Resolution happens at injection time, per final definition: + * explicit name β†’ capability projection β†’ wildcard β†’ caller's default. + * The capability projection maps a marker entry onto the definition names + * its registration ACTUALLY produced this run (reported by the registrars + * themselves in `initializeAgent`), so the mapping cannot drift from what + * gets registered. + * + * Saved agents are untouched by the wildcard (their `tool_options` come from + * the builder, keyed by real names or markers) but gain the same projection, + * replacing the hand-maintained markerβ†’names constants that kept leaking + * (create_file/edit_file, the skill catalog, memory-in-background). + * + * @module packages/api/src/agents/selection + */ + +import { logger } from '@librechat/data-schemas'; +import type { AgentToolOptions } from 'librechat-data-provider'; + +/** + * Reserved `tool_options` key holding a synthesized selection's default for + * every tool without a more specific entry. Provider tool-name grammars + * (`^[a-zA-Z0-9_-]+$`) cannot produce it, and every other consumer of + * `tool_options` reads by exact definition name, so the entry is inert + * outside the resolution below. Never persisted β€” it only appears on + * synthesized ephemeral/model-spec options. + */ +export const TOOL_SELECTION_WILDCARD = '*'; + +/** + * Capability marker β†’ tool definition names its registration produced for + * this run. Built by `initializeAgent` from the registrars' own reports, so + * an option keyed by a marker (a spec selection or a hand-edited saved + * agent) projects onto exactly what got registered. + */ +export type CapabilityToolNames = ReadonlyMap; + +type SelectionField = 'describe_intent' | 'run_in_background'; + +export interface ResolvedToolOption { + value: boolean; + /** The `tool_options` key that decided the value: the definition's own + * name, a capability marker that registered it, or the wildcard. */ + source: string; +} + +/** + * Resolves one selection field for a definition name. Precedence: an + * explicit entry under the definition's own name, then capability markers + * whose registrations include the name (an opting-in marker wins over an + * opting-out one when two capabilities register the same tool, e.g. + * `read_file` under both code and skills), then the wildcard default. + * Returns undefined when no policy speaks, leaving the caller's default + * (native intent tools, background's opt-in-only) in force. + */ +export function resolveToolOption( + name: string, + field: SelectionField, + toolOptions: AgentToolOptions | undefined, + capabilityToolNames?: CapabilityToolNames, +): ResolvedToolOption | undefined { + if (!toolOptions) { + return undefined; + } + const explicit = toolOptions[name]?.[field]; + if (explicit != null) { + return { value: explicit === true, source: name }; + } + let markerOptOut: ResolvedToolOption | undefined; + if (capabilityToolNames) { + for (const [marker, names] of capabilityToolNames) { + const markerValue = toolOptions[marker]?.[field]; + if (markerValue == null || !names.includes(name)) { + continue; + } + if (markerValue === true) { + return { value: true, source: marker }; + } + markerOptOut ??= { value: false, source: marker }; + } + } + if (markerOptOut) { + return markerOptOut; + } + const wildcard = toolOptions[TOOL_SELECTION_WILDCARD]?.[field]; + if (wildcard != null) { + return { value: wildcard === true, source: TOOL_SELECTION_WILDCARD }; + } + return undefined; +} + +/** + * Records a spec/ephemeral selection as policy entries. `true` (or the + * ephemeral toggle, which has no per-tool UI and therefore never narrows) + * becomes a wildcard opt-in; a list becomes a wildcard opt-out plus verbatim + * opt-ins for the named entries β€” names are NOT validated here, because the + * definitions they must match only exist at injection time, where + * {@link resolveToolOption} consumes them and the apply pass diagnoses + * selections that never took effect. The one exception is the reserved + * wildcard itself: a literal `*` list entry would overwrite the opt-out + * default and silently enable the capability for EVERY eligible tool, so it + * is dropped and warned about here, where it is knowably invalid. + */ +export function synthesizeSelectionToolOptions( + field: SelectionField, + selection: boolean | string[] | undefined, + ephemeralEnabled: boolean, + label: string, +): AgentToolOptions | undefined { + const selectedNames = Array.isArray(selection) ? selection : undefined; + if (!ephemeralEnabled && selection !== true && selectedNames == null) { + return undefined; + } + if (ephemeralEnabled || selection === true) { + return { [TOOL_SELECTION_WILDCARD]: { [field]: true } }; + } + const toolOptions: AgentToolOptions = { [TOOL_SELECTION_WILDCARD]: { [field]: false } }; + for (const name of selectedNames ?? []) { + if (name === TOOL_SELECTION_WILDCARD) { + logger.warn( + `${label} contains the reserved wildcard "${TOOL_SELECTION_WILDCARD}"; ignoring it. Set the option to true to cover every eligible tool.`, + ); + continue; + } + toolOptions[name] = { [field]: true }; + } + return toolOptions; +} + +/** + * The names a narrowing selection opted in, or undefined when no narrowing + * policy is present (boolean modes, saved agents). Presence of a wildcard + * opt-out is the discriminator: only synthesized list selections carry one, + * so saved agents with stale hand-edited entries are never warned about. + */ +export function getSelectionNames( + toolOptions: AgentToolOptions | undefined, + field: SelectionField, +): Set | undefined { + if (toolOptions?.[TOOL_SELECTION_WILDCARD]?.[field] !== false) { + return undefined; + } + const names = new Set(); + for (const [name, options] of Object.entries(toolOptions)) { + if (name !== TOOL_SELECTION_WILDCARD && options?.[field] === true) { + names.add(name); + } + } + return names; +} + +/** + * Warns about selection names that never took effect on any definition β€” a + * typo, a tool the spec doesn't equip, or a name whose every definition is + * ineligible (e.g. `runInBackground: ['memory']`, whose expansion is + * entirely background-excluded). A silent no-op would leave the misspelled + * or unsupported entry undiagnosable, since the symptom is simply "nothing + * happened". + */ +export function warnUnmatchedSelectionNames( + selectionNames: ReadonlySet | undefined, + effectiveSources: ReadonlySet, + label: string, +): void { + if (selectionNames == null || selectionNames.size === 0) { + return; + } + const unmatched = [...selectionNames].filter((name) => !effectiveSources.has(name)); + if (unmatched.length === 0) { + return; + } + logger.warn( + `${label} named ${unmatched.length} tool(s) that are not eligible or not equipped on this spec: ${unmatched.join(', ')}`, + ); +} + +/** + * Deep-merges two synthesized `tool_options` maps per tool key, so the + * ephemeral background and intent toggles compose instead of overwriting + * each other's per-tool entries. + */ +export function mergeSynthesizedToolOptions( + base: AgentToolOptions | undefined, + extra: AgentToolOptions | undefined, +): AgentToolOptions | undefined { + if (!extra) { + return base; + } + if (!base) { + return extra; + } + const merged: AgentToolOptions = { ...base }; + for (const [name, options] of Object.entries(extra)) { + merged[name] = { ...merged[name], ...options }; + } + return merged; +} diff --git a/packages/api/src/agents/skills.ts b/packages/api/src/agents/skills.ts index 377e3dfc22..bed7747687 100644 --- a/packages/api/src/agents/skills.ts +++ b/packages/api/src/agents/skills.ts @@ -1,7 +1,7 @@ import { logger } from '@librechat/data-schemas'; import { isEphemeralAgentId } from 'librechat-data-provider'; import { HumanMessage } from '@librechat/agents/langchain/messages'; -import { formatSkillCatalog, SkillToolDefinition } from '@librechat/agents'; +import { formatSkillCatalog, SkillToolDefinition, ReadFileToolDefinition } from '@librechat/agents'; import type { LCToolRegistry, LCTool, InjectedMessage } from '@librechat/agents'; import type { BaseMessage } from '@librechat/agents/langchain/messages'; import type { Agent } from 'librechat-data-provider'; @@ -349,6 +349,15 @@ export interface InjectSkillCatalogParams { export interface InjectSkillCatalogResult { toolDefinitions: LCTool[] | undefined; skillCount: number; + /** + * Tool names the skills capability manages this run: the `skill` tool (when + * anything is model-invocable) and `read_file` (always, for primed skill + * references). `bash_tool` is excluded even when this call registers it β€” + * it belongs to the `execute_code` capability, which reports it itself. + * `initializeAgent` records these under the `skills` marker so spec + * selections naming `skills` govern exactly these definitions. + */ + toolNames: string[]; /** * IDs of skills the runtime is authorized to resolve via `getSkillByName`. * Includes `disable-model-invocation: true` skills even though they're @@ -404,6 +413,7 @@ export async function injectSkillCatalog( return { toolDefinitions: inputDefs, skillCount: 0, + toolNames: [], activeSkillIds: [], activeSkillNames: new Set(), }; @@ -469,6 +479,7 @@ export async function injectSkillCatalog( return { toolDefinitions: inputDefs, skillCount: 0, + toolNames: [], activeSkillIds: [], activeSkillNames: new Set(), }; @@ -590,9 +601,15 @@ export async function injectSkillCatalog( }); workingDefs = codeExecResult.toolDefinitions; + const toolNames = + catalogVisibleSkills.length > 0 + ? [skillToolDef.name, ReadFileToolDefinition.name] + : [ReadFileToolDefinition.name]; + return { toolDefinitions: workingDefs, skillCount: catalogVisibleSkills.length, + toolNames, activeSkillIds: executableSkills.map((s) => s._id), activeSkillNames: new Set(executableSkills.map((s) => s.name)), }; diff --git a/packages/api/src/agents/tools.ts b/packages/api/src/agents/tools.ts index e1865eb273..3f789d9a1a 100644 --- a/packages/api/src/agents/tools.ts +++ b/packages/api/src/agents/tools.ts @@ -107,6 +107,14 @@ export interface RegisterCodeExecutionToolsResult { toolDefinitions: LCTool[]; /** Tool names newly registered (skipped names that already existed). */ registered: string[]; + /** + * Every tool name this registration manages, whether newly registered or + * already present. `initializeAgent` records these under the capability + * marker that triggered the call, so a `tool_options` entry keyed by the + * marker projects onto exactly the definitions the capability produced β€” + * the registrar itself is the source of truth, not a hand-maintained map. + */ + toolNames: string[]; } export type RegisterFileAuthoringToolsResult = RegisterCodeExecutionToolsResult; @@ -443,6 +451,7 @@ export function registerCodeExecutionTools( const candidates: LCTool[] = includeBash ? [readFileDef, buildBashToolDef({ enableToolOutputReferences, statefulSessions })] : [readFileDef]; + const toolNames = candidates.map((def) => def.name); const inputDefinitions = toolDefinitions ?? []; let workingDefinitions = inputDefinitions; @@ -485,11 +494,12 @@ export function registerCodeExecutionTools( * code-only `read_file` definition was upgraded above. */ if (newDefs.length === 0) { - return { toolDefinitions: workingDefinitions, registered }; + return { toolDefinitions: workingDefinitions, registered, toolNames }; } return { toolDefinitions: [...workingDefinitions, ...newDefs], registered, + toolNames, }; } @@ -499,6 +509,7 @@ export function registerFileAuthoringTools( const { toolRegistry, toolDefinitions, includeSkillFileInstructions = true } = params; const candidates = buildFileAuthoringDefs(includeSkillFileInstructions); + const toolNames = candidates.map((def) => def.name); const inputDefinitions = toolDefinitions ?? []; let workingDefinitions = inputDefinitions; @@ -536,10 +547,11 @@ export function registerFileAuthoringTools( } if (newDefs.length === 0) { - return { toolDefinitions: workingDefinitions, registered }; + return { toolDefinitions: workingDefinitions, registered, toolNames }; } return { toolDefinitions: [...workingDefinitions, ...newDefs], registered, + toolNames, }; } diff --git a/packages/api/src/mcp/utils.ts b/packages/api/src/mcp/utils.ts index e72ad08b2c..24ab69a5cc 100644 --- a/packages/api/src/mcp/utils.ts +++ b/packages/api/src/mcp/utils.ts @@ -4,6 +4,23 @@ import type { RequestBody } from '~/types'; export const mcpToolPattern: RegExp = new RegExp(`^.+${Constants.mcp_delimiter}.+$`); +/** + * Prefix of the lazily-expanded MCP placeholder `mcp_all`, + * pushed into an agent's `tools` for overlay/user-connection servers whose + * tool names are not known until the definitions loader expands them. + * + * The name is reserved by that convention: the definitions loader treats ANY + * matching entry as "expand every tool on this server", so a remote tool + * literally named `mcp_all` cannot be addressed individually anywhere in the + * pipeline. Kept here as the one definition of the prefix. + */ +export const MCP_ALL_PLACEHOLDER_PREFIX: string = `${Constants.mcp_all}${Constants.mcp_delimiter}`; + +/** Whether a tool entry is the lazily-expanded `mcp_all` placeholder. */ +export function isMCPAllPlaceholder(toolName: string): boolean { + return toolName.startsWith(MCP_ALL_PLACEHOLDER_PREFIX); +} + const RUNTIME_CONTEXT_PLACEHOLDER_PATTERN = /\{\{LIBRECHAT_(?:USER|OPENID|GRAPH|BODY)_[^}]+\}\}/; const RUNTIME_BODY_PLACEHOLDER_PATTERN = /\{\{LIBRECHAT_BODY_[^}]+\}\}/; const RUNTIME_BODY_PLACEHOLDER_CAPTURE_PATTERN = /\{\{LIBRECHAT_BODY_([^}]+)\}\}/g; diff --git a/packages/api/src/tools/definitions.ts b/packages/api/src/tools/definitions.ts index b6e6340f78..de31b08dc3 100644 --- a/packages/api/src/tools/definitions.ts +++ b/packages/api/src/tools/definitions.ts @@ -6,7 +6,7 @@ */ import { Providers } from '@librechat/agents'; -import { Constants, isActionTool, splitMCPToolKey } from 'librechat-data-provider'; +import { isActionTool, splitMCPToolKey } from 'librechat-data-provider'; import type { LCToolRegistry, JsonSchemaType, LCTool, GenericTool } from '@librechat/agents'; import type { AgentToolOptions } from 'librechat-data-provider'; import type { ToolDefinition } from './classification'; @@ -14,6 +14,7 @@ import { resolveJsonSchemaRefs, normalizeJsonSchema, sanitizeGeminiSchema } from import { buildToolClassification } from './classification'; import { getToolDefinition } from './registry/definitions'; import { toolkitExpansion } from './toolkits/mapping'; +import { isMCPAllPlaceholder } from '~/mcp/utils'; export interface MCPServerTool { function?: { @@ -120,8 +121,6 @@ export async function loadToolDefinitions( let actionToolDefs: ToolDefinition[] = []; const actionToolNames: string[] = []; - const mcpAllPattern = `${Constants.mcp_all}${Constants.mcp_delimiter}`; - for (const toolName of tools) { if (isActionTool(toolName)) { actionToolNames.push(toolName); @@ -171,7 +170,7 @@ export async function loadToolDefinitions( continue; } - if (toolName.startsWith(mcpAllPattern)) { + if (isMCPAllPlaceholder(toolName)) { for (const [actualToolName, toolDef] of Object.entries(serverTools)) { if (toolDef?.function) { mcpToolDefs.push({ diff --git a/packages/data-provider/src/models.ts b/packages/data-provider/src/models.ts index ca07cc1129..67fad1b41a 100644 --- a/packages/data-provider/src/models.ts +++ b/packages/data-provider/src/models.ts @@ -56,17 +56,21 @@ export type TModelSpec = { /** Equip the spec's ephemeral agent with the `ask_user_question` HITL tool. */ askUserQuestion?: boolean; /** - * Let the model dispatch this spec's eligible tool calls in the background - * (poll results via `check_background_task`). Requires the `run_in_background` + * Let the model dispatch tool calls in the background (poll results via + * `check_background_task`). `true` opts in every eligible tool; a string + * array opts in only the named tools, matched against the spec's resolved + * tool ids (e.g. `['slow_report_mcp_analytics']`). Requires the + * `run_in_background` agent capability to be enabled by the admin. + */ + runInBackground?: boolean | string[]; + /** + * Inject the `intent` label param so each call streams a live status label. + * `true` opts in every eligible tool; a string array opts in only the named + * tools, matched against the spec's resolved tool ids (e.g. + * `['web_search', 'search_code_mcp_github']`). Requires the `tool_intents` * agent capability to be enabled by the admin. */ - runInBackground?: boolean; - /** - * Inject the `intent` label param into this spec's eligible tools so each - * call streams a live status label. Requires the `tool_intents` agent - * capability to be enabled by the admin. - */ - describeIntent?: boolean; + describeIntent?: boolean | string[]; artifacts?: string | boolean; mcpServers?: string[]; skills?: boolean | string[]; @@ -102,8 +106,8 @@ export const tModelSpecSchema = z.object({ executeCode: z.boolean().optional(), memory: z.boolean().optional(), askUserQuestion: z.boolean().optional(), - runInBackground: z.boolean().optional(), - describeIntent: z.boolean().optional(), + runInBackground: z.union([z.boolean(), z.array(z.string())]).optional(), + describeIntent: z.union([z.boolean(), z.array(z.string())]).optional(), artifacts: z.union([z.string(), z.boolean()]).optional(), mcpServers: z.array(z.string()).optional(), skills: z.union([z.boolean(), z.array(z.string())]).optional(),