🪢 fix: Prune Dangling Skill IDs from Agent Allowlists (#13702)

* 🧹 fix: Prune Dangling Skill IDs from Agent Allowlists

Deleted skills left their ids behind in every agent's `skills` allowlist:
nothing removed them on skill deletion, the builder rendered no chip for
unresolvable ids (so users could neither see nor remove them), and at
runtime the non-empty allowlist intersected with accessible skills to an
empty set — silently disabling the entire skills catalog for the agent
even though the panel looked like "no skills selected."

- deleteSkill / deleteUserSkills now $pull deleted ids from all agent
  allowlists (no versioning, timestamps untouched)
- createAgent / updateAgent prune allowlist ids whose skill doc no longer
  exists (existence-only check, never ACL), so poisoned agents self-heal
  on the next save — including duplicates and sync paths
- the builder renders unresolvable allowlist entries as removable
  "Unavailable skill" chips once the catalog query resolves

* 🪞 fix: Keep Skill Queries and Authoring Labels Truthful After Chat Edits

Skills authored mid-chat via create_file/edit_file never reached the
Skills panel or builder without a manual refresh, and a create_file that
overwrote an existing file still announced "Created" in the tool card.

- invalidate all skill query caches (refetchType: 'all', since the skill
  hooks opt out of refetchOnMount) when a completed create_file/edit_file
  call targets a skills/ path
- label create_file completions from the host-authored output summary:
  overwrites now read "Updated <file>" with the edit icon

* ♻️ refactor: Inject Skill Authoring Callback Instead of Query Client

useStepHandler took useQueryClient directly, forcing a QueryClientProvider
wrapper onto all 54 renderHook calls in its spec. Its only consumer,
useEventHandlers, already holds the query client and does this exact
invalidation pattern for project/MCP keys — so pass an optional
onSkillAuthoringComplete callback instead. Detection stays in the
completion handler; the side effect lives with the client. Spec diff
collapses to pure additions.

* 🩹 fix: Resolve Codex Review Findings on Allowlist Pruning

- normalize allowlist candidates to lowercase in filterExistingSkillIds:
  isValidObjectIdString accepts uppercase hex, but _id.toString() is
  lowercase, so a casing mismatch silently emptied a valid allowlist
  (widening scope to the full catalog)
- prune agent allowlists immediately after the Skill row deletion in
  deleteSkill: a SkillFile cleanup failure previously skipped the prune
  forever, since retries exit early on deletedCount === 0
- filter version-snapshot skills through filterExistingSkillIds in
  revertAgentVersion so reverting to a pre-delete version cannot
  resurrect dangling ids
- resolve allowlist ids missing from the builder's first catalog page
  individually via getSkill before labeling them unavailable — a cache
  miss on a >100-skill catalog no longer invites removing a valid skill

* 🚪 fix: Fail Closed When Pruning Empties a Skill Allowlist

Codex round 2: an automated prune that empties an enabled allowlist
would silently widen the agent to the full accessible catalog (empty +
enabled = full per the #13526 semantics). Hygiene must only ever narrow.

- deleteSkill/deleteUserSkills: agents whose entire allowlist is being
  deleted get skills disabled instead of an emptied-but-enabled list;
  ids are lowercased before the $pull so an uppercase-but-valid id
  cannot leave the dangling entry behind
- createAgent/updateAgent/revertAgentVersion: pruning a non-empty
  allowlist to zero survivors disables skills; an explicit user-sent
  skills: [] keeps the full-catalog semantics
- builder: a per-id skill lookup only renders the removable
  "Unavailable skill" chip on a confirmed 404/403 — transient and
  server errors keep the chip hidden rather than inviting removal
This commit is contained in:
Danny Avila 2026-06-13 11:16:14 -04:00 • committed by GitHub
parent dea71c8396
commit 5ceabad5f3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
11 changed files with 615 additions and 19 deletions

View file

@ -123,6 +123,10 @@ export default function FileAuthoringCall({
}) {
const localize = useLocalize();
const isCreate = toolName === 'create_file';
/** `create_file` can overwrite an existing file (sandbox `overwrite: true`,
* or skill SKILL.md updates). The host-authored summary always opens with
* `Created`/`Updated`, so key the finished label off it for truthfulness. */
const overwrote = isCreate && output.startsWith('Updated ');
const filePath = useMemo(() => parseJsonField(args, 'file_path'), [args]);
const authoredContent = useMemo(() => parseJsonField(args, 'content'), [args]);
const editArgsPreview = useMemo(() => buildEditArgsPreview(args), [args]);
@ -145,7 +149,12 @@ export default function FileAuthoringCall({
useToolCallState(initialProgress, isSubmitting, output, !!filePath || !!preview, onExpand);
const highlighted = useLazyHighlight(preview || undefined, previewLang);
const Icon = isCreate ? FilePlus2 : FilePenLine;
const Icon = isCreate && !overwrote ? FilePlus2 : FilePenLine;
let finishedKey: 'com_ui_created_file' | 'com_ui_updated_file' | 'com_ui_edited_file' =
'com_ui_edited_file';
if (isCreate) {
finishedKey = overwrote ? 'com_ui_updated_file' : 'com_ui_created_file';
}
return (
<>
@ -157,9 +166,7 @@ export default function FileAuthoringCall({
0: fileName,
})}
finishedText={
cancelled
? localize('com_ui_cancelled')
: localize(isCreate ? 'com_ui_created_file' : 'com_ui_edited_file', { 0: fileName })
cancelled ? localize('com_ui_cancelled') : localize(finishedKey, { 0: fileName })
}
errorSuffix={hasError && !cancelled ? localize('com_ui_tool_failed') : undefined}
icon={

View file

@ -9,6 +9,7 @@ jest.mock('~/hooks', () => ({
const translations: Record<string, string> = {
com_ui_created_file: 'Created {{0}}',
com_ui_creating_file: 'Creating {{0}}',
com_ui_updated_file: 'Updated {{0}}',
com_ui_edited_file: 'Edited {{0}}',
com_ui_editing_file: 'Editing {{0}}',
com_ui_cancelled: 'Cancelled',
@ -104,6 +105,23 @@ describe('FileAuthoringCall', () => {
expect(screen.getByText('Created skills/demo/SKILL.md (4096 chars).')).toBeInTheDocument();
});
it('labels a create_file overwrite as Updated when the output summary says so', () => {
render(
<FileAuthoringCall
toolName="create_file"
initialProgress={1}
isSubmitting={false}
args={{
file_path: 'skills/demo/SKILL.md',
content: '# Demo\n\nRevised body.',
}}
output="Updated skills/demo/SKILL.md (8302 chars)."
/>,
);
expect(screen.getByTestId('progress-text')).toHaveTextContent('Updated SKILL.md');
});
it('prefers the output diff over the args preview after edit_file completes', () => {
const output = [
'Edited skills/demo/SKILL.md (exact match).',

View file

@ -1,11 +1,14 @@
import React, { useState, useMemo, useCallback } from 'react';
import { X } from 'lucide-react';
import { useQueries } from '@tanstack/react-query';
import { Switch, useToastContext } from '@librechat/client';
import { Controller, useWatch, useFormContext } from 'react-hook-form';
import {
EModelEndpoint,
PermissionTypes,
Permissions,
QueryKeys,
dataService,
getEndpointField,
} from 'librechat-data-provider';
import type { AgentForm, IconComponentTypes } from '~/common';
@ -18,14 +21,14 @@ import {
cn,
} from '~/utils';
import { ToolSelectDialog, MCPToolSelectDialog } from '~/components/Tools';
import { SkillSelectDialog } from '~/components/Skills/dialogs';
import useAgentCapabilities from '~/hooks/Agents/useAgentCapabilities';
import { useListSkillsQuery, useGetAgentFiles } from '~/data-provider';
import { useFileMapContext, useAgentPanelContext } from '~/Providers';
import { useLocalize, useVisibleTools, useHasAccess } from '~/hooks';
import { SkillSelectDialog } from '~/components/Skills/dialogs';
import AgentCategorySelector from './AgentCategorySelector';
import Action from '~/components/SidePanel/Builder/Action';
import { useLocalize, useVisibleTools, useHasAccess } from '~/hooks';
import { Panel, isEphemeralAgent } from '~/common';
import { useListSkillsQuery, useGetAgentFiles } from '~/data-provider';
import { icons } from '~/hooks/Endpoint/Icons';
import Instructions from './Instructions';
import AgentAvatar from './AgentAvatar';
@ -37,6 +40,14 @@ import AgentTool from './AgentTool';
import CodeForm from './Code/Form';
import MCPTools from './MCPTools';
/** A skill lookup only counts as a confirmed miss on 404/403 — deleted or no
* longer shared. Transient/network/server errors must not present a valid
* configured skill as removable. */
const isConfirmedSkillMiss = (error: unknown): boolean => {
const status = (error as { response?: { status?: number } } | null)?.response?.status;
return status === 404 || status === 403;
};
const labelClass = 'mb-2 text-token-text-primary block text-sm font-medium';
const inputClass = cn(
defaultTextProps,
@ -111,6 +122,36 @@ export default function AgentConfig() {
return map;
}, [skillsData?.skills]);
/** Allowlist ids missing from the first catalog page (`limit: 100`) are
* resolved individually — a cache miss alone must never present a valid
* configured skill as unavailable and invite its removal. */
const unresolvedSkillIds = useMemo(
() => (skillsData === undefined ? [] : (skills ?? []).filter((id) => !skillsMap.has(id))),
[skills, skillsMap, skillsData],
);
const unresolvedSkillQueries = useQueries({
queries: unresolvedSkillIds.map((skillId) => ({
queryKey: [QueryKeys.skill, skillId],
queryFn: () => dataService.getSkill(skillId),
retry: false,
refetchOnMount: false,
refetchOnReconnect: false,
refetchOnWindowFocus: false,
})),
});
const unresolvedSkills = useMemo(() => {
const map = new Map<string, { name?: string; missing: boolean }>();
unresolvedSkillIds.forEach((skillId, index) => {
const query = unresolvedSkillQueries[index];
if (query?.isError === true && isConfirmedSkillMiss(query.error)) {
map.set(skillId, { missing: true });
} else if (query?.data?.name) {
map.set(skillId, { name: query.data.name, missing: false });
}
});
return map;
}, [unresolvedSkillIds, unresolvedSkillQueries]);
const { data: agentFiles = [] } = useGetAgentFiles(agent_id);
const mergedFileMap = useMemo(() => {
@ -380,16 +421,31 @@ export default function AgentConfig() {
>
<div className="mb-1">
{(skills ?? []).map((skillId) => {
const skillName = skillsMap.get(skillId);
if (!skillName) {
const skillName = skillsMap.get(skillId) ?? unresolvedSkills.get(skillId)?.name;
/** Hide chips while the catalog page or per-id lookup is in
* flight. Once the backend confirms a miss (deleted or no
* longer shared), the id must stay visible and removable —
* otherwise the allowlist silently scopes the agent to
* zero skills with no way to fix it in the UI. */
if (!skillName && unresolvedSkills.get(skillId)?.missing !== true) {
return null;
}
const isUnavailable = !skillName;
return (
<div
key={skillId}
className="mb-1 flex items-center justify-between rounded-md border border-border-light px-3 py-2 text-sm"
>
<span className="truncate text-text-primary">{skillName}</span>
<span
className={
isUnavailable
? 'truncate italic text-text-secondary'
: 'truncate text-text-primary'
}
title={isUnavailable ? skillId : undefined}
>
{skillName ?? localize('com_ui_skill_unavailable')}
</span>
<button
type="button"
onClick={() => {
@ -401,7 +457,9 @@ export default function AgentConfig() {
);
}}
className="ml-2 flex-shrink-0 text-text-secondary transition-colors hover:text-text-primary"
aria-label={localize('com_ui_remove_skill_var', { 0: skillName })}
aria-label={localize('com_ui_remove_skill_var', {
0: skillName ?? skillId,
})}
disabled={skillsActive !== true}
>
<X className="h-4 w-4" aria-hidden="true" />

View file

@ -49,6 +49,7 @@ describe('useStepHandler', () => {
const mockSetMessages = jest.fn();
const mockGetMessages = jest.fn();
const mockAnnouncePolite = jest.fn();
const mockOnSkillAuthoringComplete = jest.fn();
const mockLastAnnouncementTimeRef = { current: 0 };
const createHookParams = () => ({
@ -56,6 +57,7 @@ describe('useStepHandler', () => {
getMessages: mockGetMessages,
announcePolite: mockAnnouncePolite,
lastAnnouncementTimeRef: mockLastAnnouncementTimeRef,
onSkillAuthoringComplete: mockOnSkillAuthoringComplete,
});
const createUserMessage = (overrides: Partial<TMessage> = {}): TMessage => ({
@ -1445,6 +1447,86 @@ describe('useStepHandler', () => {
expect(toolCallContent?.tool_call?.progress).toBe(1);
});
it('signals skill authoring when a completed create_file call targets a skill path', () => {
const responseMessage = createResponseMessage();
mockGetMessages.mockReturnValue([responseMessage]);
const { result } = renderHook(() => useStepHandler(createHookParams()));
const runStep = createToolCallRunStep();
const submission = createSubmission();
act(() => {
result.current.stepHandler({ event: StepEvents.ON_RUN_STEP, data: runStep }, submission);
});
const completedEvent = {
result: {
id: 'step-tool-1',
index: 0,
tool_call: {
id: 'tool-call-1',
name: 'create_file',
args: JSON.stringify({ file_path: 'skills/demo/SKILL.md', content: '# Demo' }),
output: 'Updated skills/demo/SKILL.md (6 chars).',
type: ToolCallTypes.TOOL_CALL,
},
},
};
act(() => {
result.current.stepHandler(
{
event: StepEvents.ON_RUN_STEP_COMPLETED,
data: completedEvent as { result: Agents.ToolEndEvent },
},
submission,
);
});
expect(mockOnSkillAuthoringComplete).toHaveBeenCalledTimes(1);
});
it('does not signal skill authoring for non-skill file authoring calls', () => {
const responseMessage = createResponseMessage();
mockGetMessages.mockReturnValue([responseMessage]);
const { result } = renderHook(() => useStepHandler(createHookParams()));
const runStep = createToolCallRunStep();
const submission = createSubmission();
act(() => {
result.current.stepHandler({ event: StepEvents.ON_RUN_STEP, data: runStep }, submission);
});
const completedEvent = {
result: {
id: 'step-tool-1',
index: 0,
tool_call: {
id: 'tool-call-1',
name: 'create_file',
args: JSON.stringify({ file_path: '/mnt/data/report.md', content: '# Report' }),
output: 'Created /mnt/data/report.md (8 chars).',
type: ToolCallTypes.TOOL_CALL,
},
},
};
act(() => {
result.current.stepHandler(
{
event: StepEvents.ON_RUN_STEP_COMPLETED,
data: completedEvent as { result: Agents.ToolEndEvent },
},
submission,
);
});
expect(mockOnSkillAuthoringComplete).not.toHaveBeenCalled();
});
it('should warn when step not found for completed event', () => {
const consoleSpy = jest.spyOn(console, 'warn').mockImplementation();

View file

@ -68,6 +68,16 @@ type TTitleEvent = {
const hasRealTitle = (title?: string | null): title is string =>
title != null && title !== '' && title !== 'New Chat';
/** Skill caches refreshed when a chat turn authors a skill via `create_file`/`edit_file`. */
const SKILL_QUERY_KEYS = [
QueryKeys.skills,
QueryKeys.skill,
QueryKeys.skillFiles,
QueryKeys.skillFileContent,
QueryKeys.skillTree,
QueryKeys.skillNodeContent,
] as const;
export const buildCreatedInitialResponse = ({
initialResponse,
userMessage,
@ -277,12 +287,21 @@ export default function useEventHandlers({
const { token } = useAuthContext();
const { contentHandler, resetContentHandler } = useContentHandler({ setMessages, getMessages });
/** `refetchType: 'all'` so cached-but-unmounted skill queries refresh too —
* they opt out of `refetchOnMount`, so a plain invalidation would leave
* the Skills panel stale until a manual refresh. */
const onSkillAuthoringComplete = useCallback(() => {
for (const key of SKILL_QUERY_KEYS) {
queryClient.invalidateQueries({ queryKey: [key], refetchType: 'all' });
}
}, [queryClient]);
const { stepHandler, clearStepMaps, resetSubagentAtoms, syncStepMessage } = useStepHandler({
setMessages,
getMessages,
announcePolite,
setIsSubmitting,
lastAnnouncementTimeRef,
onSkillAuthoringComplete,
});
const attachmentHandler = useAttachmentHandler(queryClient);

View file

@ -36,6 +36,12 @@ type TUseStepHandler = {
/** @deprecated - isSubmitting should be derived from submission state */
setIsSubmitting?: SetterOrUpdater<boolean>;
lastAnnouncementTimeRef: React.MutableRefObject<number>;
/**
* Fired when a completed `create_file`/`edit_file` call targeted a
* `skills/...` path. The caller owns the side effect (skill query cache
* invalidation) so this hook stays free of query-client coupling.
*/
onSkillAuthoringComplete?: () => void;
};
type TStepEvent =
@ -63,6 +69,34 @@ type AllContentTypes =
| ContentTypes.SUMMARY
| ContentTypes.ERROR;
/** Mirrors `SKILL_FILE_PREFIX` in `@librechat/api` file-authoring handlers. */
const SKILL_FILE_PREFIX = 'skills/';
const FILE_AUTHORING_TOOLS = new Set(['create_file', 'edit_file']);
/**
* True when a completed tool call authored a skill file (`create_file` /
* `edit_file` targeting a `skills/...` path). Skills created or edited
* mid-chat must invalidate the cached skill queries, or the Skills panel
* and builder keep showing the pre-authoring catalog.
*/
function isSkillAuthoringToolCall(toolCall?: Agents.ToolCall): boolean {
if (!toolCall?.name || !FILE_AUTHORING_TOOLS.has(toolCall.name)) {
return false;
}
const { args } = toolCall;
let filePath: unknown;
if (typeof args === 'object' && args !== null) {
filePath = (args as { file_path?: unknown }).file_path;
} else if (typeof args === 'string') {
try {
filePath = (JSON.parse(args) as { file_path?: unknown }).file_path;
} catch {
return false;
}
}
return typeof filePath === 'string' && filePath.startsWith(SKILL_FILE_PREFIX);
}
const isOAuthToolCallName = (name?: string) =>
typeof name === 'string' && name.startsWith(`oauth${Constants.mcp_delimiter}`);
@ -80,6 +114,7 @@ export default function useStepHandler({
getMessages,
announcePolite,
lastAnnouncementTimeRef,
onSkillAuthoringComplete,
}: TUseStepHandler) {
const toolCallIdMap = useRef(new Map<string, string | undefined>());
const messageMap = useRef(new Map<string, TMessage>());
@ -898,6 +933,10 @@ export default function useStepHandler({
return;
}
if (isSkillAuthoringToolCall(result.tool_call)) {
onSkillAuthoringComplete?.();
}
const response = messageMap.current.get(responseMessageId);
if (response) {
let updatedResponse = { ...response };
@ -1029,6 +1068,7 @@ export default function useStepHandler({
calculateContentIndex,
getCurrentMessages,
applySubagentUpdate,
onSkillAuthoringComplete,
],
);

View file

@ -1575,6 +1575,7 @@
"com_ui_skill_sr_public": "Public skill",
"com_ui_skill_states_limit": "You've reached the limit of {{0}} active/inactive skill overrides. Remove some overrides to toggle new skills.",
"com_ui_skill_toggle_active": "Toggle skill active state",
"com_ui_skill_unavailable": "Unavailable skill",
"com_ui_skill_update_conflict": "Another edit was saved before yours. Reloading the latest version.",
"com_ui_skill_update_error": "Failed to save skill",
"com_ui_skill_updated": "Skill saved",
@ -1703,6 +1704,7 @@
"com_ui_untitled": "Untitled",
"com_ui_update": "Update",
"com_ui_update_mcp_server": "Update MCP server",
"com_ui_updated_file": "Updated {{0}}",
"com_ui_updating": "Updating...",
"com_ui_upload": "Upload",
"com_ui_upload_agent_avatar": "Successfully updated agent avatar",