🧬 feat: Add GitHub Skill Sync (#13293)
Some checks failed
Docker Dev Branch Images Build / build (Dockerfile, lc-dev, node) (push) Waiting to run
Docker Dev Branch Images Build / build (Dockerfile.multi, lc-dev-api, api-build) (push) Waiting to run
GitNexus Index / index (push) Waiting to run
GitNexus Index / post-index (push) Blocked by required conditions
Docker Dev Images Build / build (Dockerfile, librechat-dev, node) (push) Waiting to run
Docker Dev Images Build / build (Dockerfile.multi, librechat-dev-api, api-build) (push) Waiting to run
Sync Locize Translations & Create Translation PR / Sync Translation Keys with Locize (push) Waiting to run
Sync Locize Translations & Create Translation PR / Create Translation PR on Version Published (push) Blocked by required conditions
Sync Helm Chart Tags / Ignore non-main push (push) Waiting to run
Sync Helm Chart Tags / Sync chart tags (push) Waiting to run
Publish `librechat-data-provider` to NPM / pack (push) Has been cancelled
Publish `librechat-data-provider` to NPM / publish-npm (push) Has been cancelled

* feat: Add GitHub skill sync

* fix: Address GitHub skill sync CI

* fix: Harden GitHub skill sync review paths

* fix: Prevent overlapping skill sync runs

* fix: Address GitHub skill sync review findings

* fix: Satisfy Git ref lint rule

* fix: Address GitHub sync review follow-ups

* fix: Match skill frontmatter closing fence

* fix: Address GitHub sync review cycle

* fix: Address GitHub sync review follow-ups

* fix: Harden GitHub skill sync worker

* fix: Format GitHub sync rollback log

* fix: Address GitHub sync review feedback

* fix: Format skill import parse handling

* fix: Coerce scalar skill frontmatter and correct scheduler timer clear

- parse: coerce numeric/boolean name and description scalars to strings instead of dropping them to empty (restores pre-refactor behavior; preserves absent-vs-empty distinction for the when-to-use fallback)
- scheduler: clear the setTimeout handle with clearTimeout rather than clearInterval
- test: cover non-string scalar frontmatter coercion

* fix: Tolerate trailing whitespace after SKILL.md opening frontmatter fence

extractFrontmatterBlock required the opening fence to be exactly '---\n', so an opener with trailing spaces/tabs (e.g. '---   \n') silently dropped all frontmatter even though the closing-fence regex already tolerates it. Match the opener with /^---[ \t]*\n/ for symmetry. Addresses Codex P3 (parse.ts:24).

* feat: Run GitHub skill sync under a per-source tenant context

Under TENANT_ISOLATION_STRICT, the sync ran with no async tenant context, so the tenant-isolation mongoose hooks threw on every Skill/SkillFile/AclEntry operation; in non-strict mode synced skills were written tenant-less and never matched tenant-scoped reads. Add an optional per-source tenantId to the skillSync config; when set, each source sync runs inside tenantStorage.run({ tenantId }) so skills, files, and public ACL grants are created and listed within that tenant, and the skill row is stamped with the tenantId for correct dedup. Sources without tenantId keep the prior single-tenant behavior. Avoids runAsSystem. Addresses Codex P2 (sync.js:70).

Lock/status/credential bookkeeping stays outside the tenant context (those collections are intentionally global).

* test: Restore dropped tenant-context coverage for GitHub skill sync

The prior commit shipped the getTenantId import in github.spec.ts without the tenant tests that use it (lost in an interrupted edit), which failed the eslint --max-warnings=0 CI job on an unused import. Restore both github.spec.ts tenant tests (tenant-scoped run stamps tenantId and executes inside the tenant ALS context; no-tenant run stays ambient) and the two config-schemas tenant tests (accepts tenantId, rejects __SYSTEM__).

* test: Restore dropped github.spec tenant-context tests

The previous commit's github.spec.ts edit did not apply (anchor mismatch), so the getTenantId import remained unused and failed eslint --max-warnings=0. Add the two tenant tests that use it: a tenant-scoped run stamps tenantId and executes inside the tenant ALS context, and a no-tenant run stays ambient.

* feat: Scope synced skill author to tenant and harden tenant-context sync

Addresses the latest Codex review on the per-source tenant change:
- makeSourceAuthorId now folds tenantId into the synthetic author hash so the
  same source mirrored into different tenants gets distinct author ids (clearer
  audits, no cross-tenant author collisions). Single-tenant author ids stay
  stable (suffix omitted when tenantId is absent).
- syncSourceInTenantContext uses an async callback per the tenant-context
  contract so the ALS store propagates across awaited Mongoose calls.
- Tests: same-source/different-tenant yields distinct authors; mirror cleanup
  is scoped to the source and deletes only its absent-upstream skills.

* fix: Repair tsc error and guard external edits in github skill sync

- Fix TS2352 in github.spec mirror-cleanup test: build the existing-skill mock via makeSkill with authorName instead of an under-typed 'as CreateSkillInput' cast (this was the failing TypeScript CI check on f00ce3c5a).
- 808: commitExistingRemoteSkillAfterFileSync re-reads to clear our own file-sync version bumps, but now compares refreshed content against the pre-sync snapshot (body/name/description/always-apply) and throws SKILL_CONFLICT on a concurrent external edit instead of overwriting it.

* docs: Note skillSync source tenantId is effectively immutable

Changing/adding/removing a source's tenantId orphans previously mirrored skills in the old tenant (a tenant-scoped sync cannot clean another tenant's data without runAsSystem, which is intentionally avoided).

* fix: Key GitHub skill upstream identity on source id and path only

Addresses Codex finding (github.ts:217): makeUpstreamId previously included owner/repo, so repointing a source to a renamed or replacement repository (same source id) changed the upstreamId, made findSkillBySourceIdentity miss the existing mirror, and then collided on the (name, author, tenantId) uniqueness constraint — leaving the source stuck failing. Identity now keys on the stable source id + root path only. The feature is unreleased, so there is no stored-id migration. Updated spec upstreamId fixtures to the new format; the existing ref-independent identity test now also covers repo moves.

* fix: Scope GitHub skill mirror deletion to the source tenant

Addresses Codex P1 (github.ts:1047/1057): an ambient source (no tenantId) runs listSkillsBySource without tenant context, which under non-strict isolation returns github-synced skills across all tenants. The mirror-deletion pass then treated other tenants' skills as absent-upstream and could delete them. Filter existingSyncedSkills to rows whose tenantId matches the source's configured tenantId (absent = its own ambient bucket) before deleting, so a sync never removes another tenant's mirrored skills. Covered by a test where an ambient run leaves a tenant-b-owned skill untouched.

* fix: Apply tenant-scoped mirror deletion implementation

The prior commit (75ccfa3fc) added the test but the source change to github.ts was lost in an interrupted edit, leaving a failing test with no implementation. This adds the actual guard: the mirror-deletion pass skips skills whose tenantId does not match the source's configured tenantId (absent = ambient bucket), so an ambient source whose listSkillsBySource returns cross-tenant rows under non-strict isolation cannot delete another tenant's mirrored skills.

* fix: Resolve global access role outside tenant context for synced skill grants

Addresses Codex P2 (github.ts:1166): default access roles (incl. skill_viewer) are seeded globally with no tenantId under runAsSystem, but a tenant-scoped sync wraps ensurePublicViewer in the source's tenant context. The PermissionService grantPermission resolved the role via a tenant-isolated AccessRole query, so the global role did not match and tenant-scoped syncs failed with 'Role skill_viewer not found'. The sync adapter now resolves the role inside runAsSystem (matching the global seed) and writes the ACL entry in the active tenant context, so the AclEntry is tenant-scoped (visible to tenant users) while the role lookup still succeeds. Covered by service tests for the resolve-vs-write split and the missing-role failure.

* fix: Strip placeholder frontmatter booleans and check skill conflict before file sync

- 1083 (github.ts:759): toCleanFrontmatter now drops a non-boolean always-apply (e.g. the 'always-apply:' / 'always-apply: # TODO' placeholder, which js-yaml yields as null). The boolean is already captured in the dedicated alwaysApply field; persisting null left ambiguous frontmatter on the synced skill.
- 1080 (github.ts:1057): for an existing mirrored skill, check for an external content edit (via getSkillById + hasExternalSkillEdit) BEFORE syncSkillFiles mutates the bundled files, so a concurrently edited skill fails fast with SKILL_CONFLICT without partial file rewrites. The post-file-sync check still guards edits that land during the file sync window.
Tests: placeholder always-apply is dropped from synced frontmatter; concurrent-edit conflict leaves files unmutated (no upsert/delete).

* fix: Harden GitHub skill sync review paths

* fix: Reuse moved GitHub skill mirrors

* fix: Scope GitHub sync identity conflicts

* test: Fix GitHub sync conflict mock typing

* fix: Support nested env-backed skill sync

* fix: Keep skill sync config base-only

* fix: Scope GitHub skill identity lookup by tenant

* fix: Harden GitHub skill sync admin gates

* fix: Guard existing skill sync permission grants

* feat: Trigger skill sync from resolved config

* fix: Scope resolved skill sync by tenant

* test: Allow manual skill sync status tenant scoping

* refactor: Extract skill sync trigger orchestrator

* test: Complete orchestrator status fixture

* chore: Bump data provider version

* fix: Restrict skill sync server credentials

* test: Complete admin skill sync status fixtures

* fix: tighten skill sync trigger safeguards

* fix: preserve alwaysApply skill sync alias

* chore: sort skill sync imports

* fix: preserve skill sync request scope

* fix: harden skill sync review edges

* refactor: move skill sync admin access to api package

* fix: add skill sync declaration return types

* fix: satisfy skill sync type checks

* fix: resolve codex skill sync review findings

* fix: harden skill sync review edges

* fix: resolve codex skill sync edge findings

* fix: satisfy API declaration build after rebase
This commit is contained in:
Danny Avila 2026-06-10 21:05:54 -04:00 • committed by GitHub
parent 470be2395f
commit 197a1dc4e2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
58 changed files with 8805 additions and 162 deletions

View file

@ -194,6 +194,29 @@ describe('createAdminConfigHandlers', () => {
expect(savedOverrides.interface).toEqual({ modelSelect: false });
});
it('preserves skillSync sections in admin overrides', async () => {
const { handlers, deps } = createHandlers({
upsertConfig: jest.fn().mockResolvedValue({ _id: 'c1', configVersion: 1 }),
});
const req = mockReq({
params: { principalType: 'role', principalId: 'admin' },
body: {
overrides: {
skillSync: { github: { enabled: true } },
interface: { modelSelect: false },
},
},
});
const res = mockRes();
await handlers.upsertConfigOverrides(req, res);
expect(res.statusCode).toBe(201);
const savedOverrides = deps.upsertConfig.mock.calls[0][3];
expect(savedOverrides.skillSync).toEqual({ github: { enabled: true } });
expect(savedOverrides.interface).toEqual({ modelSelect: false });
});
it('preserves UI sub-keys in composite permission fields like mcpServers', async () => {
const { handlers, deps } = createHandlers({
upsertConfig: jest.fn().mockResolvedValue({ _id: 'c1', configVersion: 1 }),
@ -338,6 +361,24 @@ describe('createAdminConfigHandlers', () => {
expect(deps.unsetConfigField).not.toHaveBeenCalled();
});
it('allows deleting skillSync field paths', async () => {
const { handlers, deps } = createHandlers();
const req = mockReq({
params: { principalType: 'role', principalId: 'admin' },
query: { fieldPath: 'skillSync.github.enabled' },
});
const res = mockRes();
await handlers.deleteConfigField(req, res);
expect(res.statusCode).toBe(200);
expect(deps.unsetConfigField).toHaveBeenCalledWith(
'role',
'admin',
'skillSync.github.enabled',
);
});
it('allows deleting interface UI field paths', async () => {
const { handlers, deps } = createHandlers();
const req = mockReq({
@ -489,6 +530,27 @@ describe('createAdminConfigHandlers', () => {
expect(patchedFields['interface.prompts']).toBeUndefined();
});
it('preserves skillSync field entries in patches', async () => {
const { handlers, deps } = createHandlers();
const req = mockReq({
params: { principalType: 'role', principalId: 'admin' },
body: {
entries: [
{ fieldPath: 'skillSync.github.enabled', value: true },
{ fieldPath: 'interface.modelSelect', value: false },
],
},
});
const res = mockRes();
await handlers.patchConfigField(req, res);
expect(res.statusCode).toBe(200);
const patchedFields = deps.patchConfigFields.mock.calls[0][3];
expect(patchedFields['skillSync.github.enabled']).toBe(true);
expect(patchedFields['interface.modelSelect']).toBe(false);
});
it('blocks peoplePicker permission sub-key paths', async () => {
const { handlers, deps } = createHandlers();
const req = mockReq({

View file

@ -1,5 +1,6 @@
import { logger } from '@librechat/data-schemas';
import {
BASE_ONLY_CONFIG_SECTIONS,
PrincipalType,
PrincipalModel,
INTERFACE_PERMISSION_FIELDS,
@ -15,6 +16,7 @@ import type { ServerRequest } from '~/types/http';
const UNSAFE_SEGMENTS = /(?:^|\.)(__[\w]*|constructor|prototype)(?:\.|$)/;
const MAX_PATCH_ENTRIES = 100;
const DEFAULT_PRIORITY = 10;
const BASE_ONLY_OVERRIDE_SECTIONS = new Set<string>(BASE_ONLY_CONFIG_SECTIONS);
export function isValidFieldPath(path: string): boolean {
return (
@ -31,6 +33,10 @@ export function getTopLevelSection(fieldPath: string): string {
return fieldPath.split('.')[0];
}
function isBaseOnlyFieldPath(fieldPath: string): boolean {
return BASE_ONLY_OVERRIDE_SECTIONS.has(getTopLevelSection(fieldPath));
}
/**
* Returns true if `fieldPath` targets an interface permission field or permission sub-key.
*
@ -316,7 +322,17 @@ export function createAdminConfigHandlers(deps: AdminConfigDeps): {
return res.status(403).json({ error: 'Insufficient permissions' });
}
let filteredOverrides = overrides;
const filteredOverrides = {
...(overrides as Record<string, unknown>),
} as Partial<TCustomConfig>;
for (const section of BASE_ONLY_OVERRIDE_SECTIONS) {
if (section in filteredOverrides) {
delete (filteredOverrides as Record<string, unknown>)[section];
logger.warn(
`[adminConfig] Stripping base-only config section "${section}" - configure it in librechat.yaml instead`,
);
}
}
const iface = (overrides as Record<string, unknown>).interface;
if (iface != null && typeof iface === 'object' && !Array.isArray(iface)) {
const filteredIface: Record<string, unknown> = {};
@ -345,7 +361,6 @@ export function createAdminConfigHandlers(deps: AdminConfigDeps): {
);
}
}
filteredOverrides = { ...(overrides as Record<string, unknown>) } as Partial<TCustomConfig>;
if (Object.keys(filteredIface).length > 0) {
(filteredOverrides as Record<string, unknown>).interface = filteredIface;
} else {
@ -436,6 +451,12 @@ export function createAdminConfigHandlers(deps: AdminConfigDeps): {
}
const validEntries = entries.filter((entry) => {
if (isBaseOnlyFieldPath(entry.fieldPath)) {
logger.warn(
`[adminConfig] Stripping base-only config field "${entry.fieldPath}" - configure it in librechat.yaml instead`,
);
return false;
}
if (isInterfacePermissionPath(entry.fieldPath)) {
logger.warn(
`[adminConfig] Stripping interface permission field "${entry.fieldPath}" — use role permissions instead`,
@ -608,6 +629,13 @@ export function createAdminConfigHandlers(deps: AdminConfigDeps): {
});
}
if (isBaseOnlyFieldPath(fieldPath)) {
logger.warn(
`[adminConfig] Ignoring delete for base-only config field "${fieldPath}" - configure it in librechat.yaml instead`,
);
return res.status(200).json({ message: 'No actionable field path provided' });
}
if (isInterfacePermissionPath(fieldPath)) {
logger.warn(
`[adminConfig] Ignoring delete for interface permission field "${fieldPath}" — use role permissions instead`,

View file

@ -2,9 +2,11 @@ export { createAdminConfigHandlers } from './config';
export { createAdminGrantsHandlers } from './grants';
export { createAdminGroupsHandlers } from './groups';
export { createAdminRolesHandlers } from './roles';
export { createAdminSkillsSyncAccess, createAdminSkillsSyncHandlers } from './skills';
export { createAdminUsersHandlers } from './users';
export type { AdminConfigDeps } from './config';
export type { AdminGrantsDeps, GrantPrincipalType } from './grants';
export type { AdminGroupsDeps } from './groups';
export type { AdminRolesDeps } from './roles';
export type { AdminSkillSyncAccessDeps, AdminSkillSyncDeps } from './skills';
export type { AdminUsersDeps } from './users';

View file

@ -0,0 +1,363 @@
import { SystemCapabilities } from '@librechat/data-schemas';
import type { NextFunction, Response } from 'express';
import { createAdminSkillsSyncAccess, createAdminSkillsSyncHandlers } from './skills';
function createResponse() {
const res = {
status: jest.fn().mockReturnThis(),
json: jest.fn().mockReturnThis(),
};
return res as unknown as Response & {
status: jest.Mock;
json: jest.Mock;
};
}
function createNext(): NextFunction & jest.Mock {
return jest.fn() as NextFunction & jest.Mock;
}
function createHandlers({
statusErrorCode,
statusErrorMessage,
}: { statusErrorCode?: string; statusErrorMessage?: string } = {}) {
const runner = {
getStatus: jest.fn(async () => ({
enabled: true,
intervalMinutes: 60,
runOnStartup: false,
sources: [
{
provider: 'github' as const,
sourceId: 'tenant-skills',
tenantId: 'tenant-a',
status: 'idle' as const,
credentialKey: 'github-skills-prod',
credentialPresent: true,
owner: 'LibreChat',
repo: 'skills',
ref: 'main',
paths: ['skills'],
syncedSkillCount: 0,
syncedFileCount: 0,
deletedSkillCount: 0,
deletedFileCount: 0,
errorCode: statusErrorCode,
errorMessage: statusErrorMessage,
startedAt: undefined,
finishedAt: undefined,
lastSuccessAt: undefined,
lastFailureAt: undefined,
createdAt: undefined,
updatedAt: undefined,
},
],
credentials: [
{
provider: 'github' as const,
credentialKey: 'github-skills-prod',
credentialPresent: true,
tokenFingerprint: 'abc123',
},
],
fineGrainedTokenRecommendation: 'Use a fine-grained token.',
})),
runOnce: jest.fn(async () => ({
status: 'completed' as const,
sources: [
{
provider: 'github' as const,
sourceId: 'tenant-skills',
tenantId: 'tenant-a',
status: 'succeeded' as const,
credentialKey: 'github-skills-prod',
credentialPresent: true,
syncedSkillCount: 1,
syncedFileCount: 2,
deletedSkillCount: 0,
deletedFileCount: 0,
errorCode: statusErrorCode,
errorMessage: statusErrorMessage,
startedAt: undefined,
finishedAt: undefined,
lastSuccessAt: undefined,
lastFailureAt: undefined,
createdAt: undefined,
updatedAt: undefined,
},
],
})),
};
const handlers = createAdminSkillsSyncHandlers({
runner,
upsertCredential: jest.fn(),
deleteCredential: jest.fn(),
});
return { handlers, runner };
}
describe('createAdminSkillsSyncHandlers', () => {
it('omits credential summaries and source credential metadata for tenant-scoped status reads', async () => {
const { handlers } = createHandlers();
const res = createResponse();
await handlers.getSyncStatus({ skillSyncCanReadCredentials: false } as never, res);
expect(res.status).toHaveBeenCalledWith(200);
expect(res.json).toHaveBeenCalledWith(
expect.objectContaining({
credentials: [],
sources: [
expect.objectContaining({
credentialKey: undefined,
credentialPresent: false,
owner: undefined,
repo: undefined,
ref: undefined,
paths: undefined,
}),
],
}),
);
});
it('redacts credential-related errors from tenant-scoped status reads', async () => {
const { handlers } = createHandlers({
statusErrorCode: 'MISSING_CREDENTIAL',
statusErrorMessage: 'Missing GitHub token environment variable "GITHUB_SKILLS_TOKEN"',
});
const res = createResponse();
await handlers.getSyncStatus({ skillSyncCanReadCredentials: false } as never, res);
expect(res.json).toHaveBeenCalledWith(
expect.objectContaining({
sources: [
expect.objectContaining({
errorCode: 'MISSING_CREDENTIAL',
errorMessage: 'GitHub skill sync credentials are not available',
}),
],
}),
);
});
it('includes credential summaries and source credential metadata for platform status reads', async () => {
const { handlers } = createHandlers({
statusErrorCode: 'MISSING_CREDENTIAL',
statusErrorMessage: 'Missing GitHub credential "github-skills-prod"',
});
const res = createResponse();
await handlers.getSyncStatus({ skillSyncCanReadCredentials: true } as never, res);
expect(res.json).toHaveBeenCalledWith(
expect.objectContaining({
credentials: [expect.objectContaining({ credentialKey: 'github-skills-prod' })],
sources: [
expect.objectContaining({
credentialKey: 'github-skills-prod',
credentialPresent: true,
owner: 'LibreChat',
repo: 'skills',
ref: 'main',
paths: ['skills'],
errorMessage: 'Missing GitHub credential "github-skills-prod"',
}),
],
}),
);
});
it('omits source credential metadata from tenant-scoped manual run responses', async () => {
const { handlers } = createHandlers({
statusErrorCode: 'MISSING_CREDENTIAL',
statusErrorMessage: 'Missing GitHub credential "github-skills-prod"',
});
const res = createResponse();
await handlers.runSync(
{
skillSyncAllowServerCredentials: true,
skillSyncCanReadCredentials: false,
} as never,
res,
);
expect(res.json).toHaveBeenCalledWith(
expect.objectContaining({
sources: [
expect.objectContaining({
credentialKey: undefined,
credentialPresent: false,
errorMessage: 'GitHub skill sync credentials are not available',
}),
],
}),
);
});
});
describe('createAdminSkillsSyncAccess', () => {
const baseSkillSync = {
github: {
enabled: true,
intervalMinutes: 60,
runOnStartup: false,
sources: [
{
id: 'base-skills',
owner: 'LibreChat',
repo: 'skills',
ref: 'main',
paths: ['skills'],
token: '${GITHUB_SKILLS_TOKEN}',
},
],
},
};
function createAccess({
hasCapability = jest.fn().mockResolvedValue(true),
getAppConfig = jest.fn().mockResolvedValue({ skillSync: undefined }),
}: {
hasCapability?: jest.Mock;
getAppConfig?: jest.Mock;
} = {}) {
return {
access: createAdminSkillsSyncAccess({ getAppConfig, hasCapability }),
getAppConfig,
hasCapability,
};
}
it('attaches the base skill sync config for override comparison', async () => {
const getAppConfig = jest.fn().mockResolvedValue({ skillSync: baseSkillSync });
const { access } = createAccess({ getAppConfig });
const req = { config: { skillSync: undefined, config: { endpoints: {} } } };
const res = createResponse();
const next = createNext();
await access.attachBaseSkillSyncConfig(req as never, res, next);
expect(getAppConfig).toHaveBeenCalledWith({ baseOnly: true });
expect(req.config.config).toEqual({ endpoints: {}, skillSync: baseSkillSync });
expect(next).toHaveBeenCalledTimes(1);
});
it('marks credential metadata hidden for tenant-scoped status reads', async () => {
const hasCapability = jest.fn(
async (user: { tenantId?: string }, capability: string): Promise<boolean> => {
if (capability === SystemCapabilities.READ_SKILLS) {
return Boolean(user.tenantId);
}
return true;
},
);
const { access } = createAccess({ hasCapability });
const req = { user: { id: 'user-1', role: 'ADMIN', tenantId: 'tenant-a' } };
const res = createResponse();
const next = createNext();
await access.requireReadSkills(req as never, res, next);
await access.attachCredentialReadAccess(req as never, res, next);
expect(req).toMatchObject({
skillSyncCanReadCredentials: false,
skillSyncAllowServerCredentials: false,
});
expect(hasCapability).toHaveBeenCalledWith(
{ id: 'user-1', role: 'ADMIN', tenantId: 'tenant-a' },
SystemCapabilities.READ_SKILLS,
);
expect(hasCapability).toHaveBeenCalledWith(
{ id: 'user-1', role: 'ADMIN' },
SystemCapabilities.READ_SKILLS,
);
});
it('prevents tenant admins from running overrides that require server credentials', async () => {
const tenantSkillSync = {
github: {
enabled: true,
intervalMinutes: 60,
runOnStartup: false,
sources: [
{
id: 'tenant-skills',
owner: 'LibreChat',
repo: 'skills',
ref: 'main',
paths: ['skills'],
token: '${GITHUB_SKILLS_TOKEN}',
},
],
},
};
const hasCapability = jest.fn(
async (user: { tenantId?: string }, capability: string): Promise<boolean> => {
if (capability === SystemCapabilities.MANAGE_SKILLS) {
return Boolean(user.tenantId);
}
return true;
},
);
const { access } = createAccess({ hasCapability });
const req = {
user: { id: 'user-1', role: 'ADMIN', tenantId: 'tenant-a' },
config: { skillSync: tenantSkillSync, config: {} },
};
const res = createResponse();
const next = createNext();
await access.requireSyncRunCapability(req as never, res, next);
expect(res.status).toHaveBeenCalledWith(403);
expect(res.json).toHaveBeenCalledWith({
message: 'Tenant-scoped manual skill sync requires platform credential access',
});
expect(next).not.toHaveBeenCalled();
});
it('prevents tenant admins from manually running base skill sync config', async () => {
const hasCapability = jest.fn(
async (user: { tenantId?: string }, capability: string): Promise<boolean> => {
if (capability === SystemCapabilities.MANAGE_SKILLS) {
return Boolean(user.tenantId);
}
return true;
},
);
const { access } = createAccess({ hasCapability });
const req = {
user: { id: 'user-1', role: 'ADMIN', tenantId: 'tenant-a' },
config: { skillSync: baseSkillSync, config: { skillSync: baseSkillSync } },
};
const res = createResponse();
const next = createNext();
await access.requireSyncRunCapability(req as never, res, next);
expect(res.status).toHaveBeenCalledWith(403);
expect(res.json).toHaveBeenCalledWith({ message: 'Forbidden' });
expect(next).not.toHaveBeenCalled();
});
it('allows platform admins to manually run base skill sync config with server credentials', async () => {
const { access } = createAccess();
const req = {
user: { id: 'user-1', role: 'ADMIN', tenantId: 'tenant-a' },
config: { skillSync: baseSkillSync, config: { skillSync: baseSkillSync } },
};
const res = createResponse();
const next = createNext();
await access.requireSyncRunCapability(req as never, res, next);
expect(req).toMatchObject({
skillSyncAllowServerCredentials: true,
skillSyncCanReadCredentials: true,
});
expect(next).toHaveBeenCalledTimes(1);
});
});

View file

@ -0,0 +1,390 @@
import { SystemCapabilities } from '@librechat/data-schemas';
import { skillSyncConfigSchema } from 'librechat-data-provider';
import type {
TGitHubSkillSyncStatusResponse,
TGitHubSkillSyncSourceStatus,
TGitHubSkillSyncCredentialSummary,
TGitHubSkillSyncManualRunResponse,
SkillSyncConfig,
} from 'librechat-data-provider';
import type {
ISkillSyncStatus,
SkillSyncProvider,
SkillSyncCredentialSummary,
UpsertSkillSyncCredentialInput,
SystemCapability,
} from '@librechat/data-schemas';
import type { NextFunction, Request, RequestHandler, Response } from 'express';
import type { Types } from 'mongoose';
import type { GitHubSkillSyncRunner } from '~/skills/sync';
export type AdminSkillsRequest = Request & {
user?: {
_id?: Types.ObjectId;
id?: string;
};
skillSyncAllowServerCredentials?: boolean;
skillSyncCanReadCredentials?: boolean;
};
type SkillSyncConfigContainer = {
skillSync?: unknown;
config?: {
skillSync?: unknown;
} & Record<string, unknown>;
} & Record<string, unknown>;
type AdminSkillSyncAccessRequest = Request & {
user?: {
_id?: Types.ObjectId | { toString(): string };
id?: string;
role?: string;
tenantId?: string;
};
config?: SkillSyncConfigContainer;
skillSyncAllowServerCredentials?: boolean;
skillSyncCanReadCredentials?: boolean;
};
type SkillSyncCapabilityUser = {
id: string;
role: string;
tenantId?: string;
};
export type AdminSkillSyncDeps = {
runner?: GitHubSkillSyncRunner;
getRunner?: (req: Request) => GitHubSkillSyncRunner;
upsertCredential: (input: UpsertSkillSyncCredentialInput) => Promise<SkillSyncCredentialSummary>;
deleteCredential: (
provider: SkillSyncProvider,
credentialKey: string,
) => Promise<{ deleted: boolean }>;
};
export type AdminSkillSyncAccessDeps = {
getAppConfig: (options: { baseOnly: true }) => Promise<{ skillSync?: unknown } | undefined>;
hasCapability: (user: SkillSyncCapabilityUser, capability: SystemCapability) => Promise<boolean>;
};
type AdminSkillsSyncHandler = (req: AdminSkillsRequest, res: Response) => Promise<Response>;
export type AdminSkillsSyncHandlers = {
getSyncStatus: AdminSkillsSyncHandler;
runSync: AdminSkillsSyncHandler;
setCredential: AdminSkillsSyncHandler;
deleteCredential: (req: Request, res: Response) => Promise<Response>;
};
export type AdminSkillsSyncAccess = {
attachBaseSkillSyncConfig: RequestHandler;
attachCredentialReadAccess: RequestHandler;
requireReadSkills: RequestHandler;
requirePlatformManageSkills: RequestHandler;
requireSyncRunCapability: RequestHandler;
};
const CREDENTIAL_KEY_PATTERN = /^[a-zA-Z0-9][a-zA-Z0-9_-]{0,63}$/;
function toIso(date: Date | undefined): string | undefined {
return date ? date.toISOString() : undefined;
}
function serializeCredential(
credential: SkillSyncCredentialSummary,
): TGitHubSkillSyncCredentialSummary {
return {
provider: credential.provider,
credentialKey: credential.credentialKey,
credentialPresent: credential.credentialPresent,
tokenFingerprint: credential.tokenFingerprint,
createdAt: toIso(credential.createdAt),
updatedAt: toIso(credential.updatedAt),
};
}
function isCredentialError(status: ISkillSyncStatus): boolean {
if (status.errorCode === 'MISSING_CREDENTIAL') {
return true;
}
return /credential|token environment variable|server github credentials/i.test(
status.errorMessage ?? '',
);
}
function serializeErrorMessage(
status: ISkillSyncStatus,
{ includeCredentialMetadata }: { includeCredentialMetadata: boolean },
): string | undefined {
if (includeCredentialMetadata || !isCredentialError(status)) {
return status.errorMessage;
}
return 'GitHub skill sync credentials are not available';
}
function serializeSourceStatus(
status: ISkillSyncStatus & { credentialPresent?: boolean },
{ includeCredentialMetadata = true }: { includeCredentialMetadata?: boolean } = {},
): TGitHubSkillSyncSourceStatus {
const includePrivateSourceMetadata = includeCredentialMetadata;
return {
provider: status.provider,
sourceId: status.sourceId,
tenantId: status.tenantId,
status: status.status,
credentialKey: includeCredentialMetadata ? status.credentialKey : undefined,
credentialPresent: includeCredentialMetadata ? (status.credentialPresent ?? false) : false,
owner: includePrivateSourceMetadata ? status.owner : undefined,
repo: includePrivateSourceMetadata ? status.repo : undefined,
ref: includePrivateSourceMetadata ? status.ref : undefined,
paths: includePrivateSourceMetadata ? status.paths : undefined,
startedAt: toIso(status.startedAt),
finishedAt: toIso(status.finishedAt),
lastSuccessAt: toIso(status.lastSuccessAt),
lastFailureAt: toIso(status.lastFailureAt),
errorCode: status.errorCode,
errorMessage: serializeErrorMessage(status, { includeCredentialMetadata }),
syncedSkillCount: status.syncedSkillCount,
syncedFileCount: status.syncedFileCount,
deletedSkillCount: status.deletedSkillCount,
deletedFileCount: status.deletedFileCount,
createdAt: toIso(status.createdAt),
updatedAt: toIso(status.updatedAt),
};
}
function isCredentialKey(value: unknown): value is string {
return typeof value === 'string' && CREDENTIAL_KEY_PATTERN.test(value);
}
function getUserObjectId(req: AdminSkillsRequest): Types.ObjectId | undefined {
return req.user?._id;
}
function getCapabilityUser(
req: AdminSkillSyncAccessRequest,
{ platformOnly = false }: { platformOnly?: boolean } = {},
): SkillSyncCapabilityUser | null {
const id = req.user?.id ?? req.user?._id?.toString?.();
if (!id) {
return null;
}
return {
id,
role: req.user?.role ?? '',
...(platformOnly ? {} : { tenantId: req.user?.tenantId }),
};
}
function parseSkillSyncConfig(raw: unknown): SkillSyncConfig | undefined {
if (!raw || typeof raw !== 'object') {
return undefined;
}
const parsed = skillSyncConfigSchema.safeParse(raw);
return parsed.success ? parsed.data : undefined;
}
function isSameSkillSyncConfig(left: SkillSyncConfig, right: SkillSyncConfig): boolean {
return JSON.stringify(left ?? null) === JSON.stringify(right ?? null);
}
function hasResolvedSkillSyncOverride(req: AdminSkillSyncAccessRequest): boolean {
const resolved = parseSkillSyncConfig(req.config?.skillSync);
const base = parseSkillSyncConfig(req.config?.config?.skillSync);
return Boolean(resolved?.github && !isSameSkillSyncConfig(resolved, base));
}
function sendInternalServerError(res: Response): void {
res.status(500).json({ message: 'Internal Server Error' });
}
export function createAdminSkillsSyncAccess(deps: AdminSkillSyncAccessDeps): AdminSkillsSyncAccess {
async function hasSkillCapability(
req: AdminSkillSyncAccessRequest,
capability: SystemCapability,
{ platformOnly = false }: { platformOnly?: boolean } = {},
): Promise<boolean> {
const user = getCapabilityUser(req, { platformOnly });
if (!user) {
return false;
}
return deps.hasCapability(user, capability);
}
function requireSkillCapability(
capability: SystemCapability,
{ platformOnly = false }: { platformOnly?: boolean } = {},
): RequestHandler {
return async (
req: AdminSkillSyncAccessRequest,
res: Response,
next: NextFunction,
): Promise<void> => {
try {
const user = getCapabilityUser(req, { platformOnly });
if (!user) {
res.status(401).json({ message: 'Authentication required' });
return;
}
if (await deps.hasCapability(user, capability)) {
next();
return;
}
res.status(403).json({ message: 'Forbidden' });
} catch {
sendInternalServerError(res);
}
};
}
const attachBaseSkillSyncConfig: RequestHandler = async (
req: AdminSkillSyncAccessRequest,
res: Response,
next: NextFunction,
): Promise<void> => {
try {
const baseConfig = await deps.getAppConfig({ baseOnly: true });
const existingConfig = req.config ?? {};
req.config = {
...existingConfig,
config: {
...(existingConfig.config ?? {}),
skillSync: baseConfig?.skillSync,
},
};
next();
} catch {
sendInternalServerError(res);
}
};
const attachCredentialReadAccess: RequestHandler = async (
req: AdminSkillSyncAccessRequest,
res: Response,
next: NextFunction,
): Promise<void> => {
try {
const canReadCredentials = await hasSkillCapability(req, SystemCapabilities.READ_SKILLS, {
platformOnly: true,
});
req.skillSyncCanReadCredentials = canReadCredentials;
req.skillSyncAllowServerCredentials = canReadCredentials;
next();
} catch {
sendInternalServerError(res);
}
};
const requireSyncRunCapability: RequestHandler = async (
req: AdminSkillSyncAccessRequest,
res: Response,
next: NextFunction,
): Promise<void> => {
try {
const canManagePlatform = await hasSkillCapability(req, SystemCapabilities.MANAGE_SKILLS, {
platformOnly: true,
});
if (canManagePlatform) {
req.skillSyncAllowServerCredentials = true;
req.skillSyncCanReadCredentials = true;
next();
return;
}
if (
hasResolvedSkillSyncOverride(req) &&
(await hasSkillCapability(req, SystemCapabilities.MANAGE_SKILLS))
) {
res.status(403).json({
message: 'Tenant-scoped manual skill sync requires platform credential access',
});
return;
}
res.status(403).json({ message: 'Forbidden' });
} catch {
sendInternalServerError(res);
}
};
return {
attachBaseSkillSyncConfig,
attachCredentialReadAccess,
requireReadSkills: requireSkillCapability(SystemCapabilities.READ_SKILLS),
requirePlatformManageSkills: requireSkillCapability(SystemCapabilities.MANAGE_SKILLS, {
platformOnly: true,
}),
requireSyncRunCapability,
};
}
export function createAdminSkillsSyncHandlers(deps: AdminSkillSyncDeps): AdminSkillsSyncHandlers {
function getRunner(req: Request): GitHubSkillSyncRunner {
const runner = deps.getRunner?.(req) ?? deps.runner;
if (!runner) {
throw new Error('GitHub skill sync runner is not configured');
}
return runner;
}
async function getSyncStatus(req: AdminSkillsRequest, res: Response) {
const includeCredentialMetadata = req.skillSyncCanReadCredentials !== false;
const status = await getRunner(req).getStatus();
const response: TGitHubSkillSyncStatusResponse = {
enabled: status.enabled,
intervalMinutes: status.intervalMinutes,
runOnStartup: status.runOnStartup,
sources: status.sources.map((source) =>
serializeSourceStatus(source, { includeCredentialMetadata }),
),
credentials: includeCredentialMetadata ? status.credentials.map(serializeCredential) : [],
fineGrainedTokenRecommendation: status.fineGrainedTokenRecommendation,
};
return res.status(200).json(response);
}
async function runSync(req: AdminSkillsRequest, res: Response) {
const includeCredentialMetadata = req.skillSyncCanReadCredentials === true;
const result = await getRunner(req).runOnce();
const response: TGitHubSkillSyncManualRunResponse = {
status: result.status,
message: result.message,
sources: result.sources.map((source) =>
serializeSourceStatus(source, { includeCredentialMetadata }),
),
};
return res.status(result.status === 'skipped' ? 202 : 200).json(response);
}
async function setCredential(req: AdminSkillsRequest, res: Response) {
const { credentialKey } = req.params;
if (!isCredentialKey(credentialKey)) {
return res.status(400).json({ error: 'Invalid credential key' });
}
const token = (req.body as { token?: unknown } | undefined)?.token;
if (typeof token !== 'string' || token.trim().length === 0) {
return res.status(400).json({ error: 'GitHub token is required' });
}
const credential = await deps.upsertCredential({
provider: 'github',
credentialKey,
token: token.trim(),
userId: getUserObjectId(req),
});
return res.status(200).json(serializeCredential(credential));
}
async function deleteCredential(req: Request, res: Response) {
const { credentialKey } = req.params;
if (!isCredentialKey(credentialKey)) {
return res.status(400).json({ error: 'Invalid credential key' });
}
const result = await deps.deleteCredential('github', credentialKey);
return res.status(200).json({ credentialKey, deleted: result.deleted });
}
return {
getSyncStatus,
runSync,
setCredential,
deleteCredential,
};
}

View file

@ -67,6 +67,20 @@ function mockZipRequest(buffer: Buffer): ImportRequest {
} as unknown as ImportRequest;
}
function mockMarkdownRequest(content: string, originalname = 'bad-frontmatter.md'): ImportRequest {
return {
user: {
id: 'user-1',
_id: new Types.ObjectId(),
username: 'tester',
},
file: {
originalname,
buffer: Buffer.from(content),
},
} as unknown as ImportRequest;
}
function importSummary(body: unknown): ImportSummary {
return (body as { _importSummary: ImportSummary })._importSummary;
}
@ -90,6 +104,12 @@ async function zipWithAdditionalFiles(fileCount: number, fileBytes: number): Pro
return zip.generateAsync({ type: 'nodebuffer', compression: 'DEFLATE' });
}
async function zipWithSkillMarkdown(skillMarkdown: string): Promise<Buffer> {
const zip = new JSZip();
zip.file('SKILL.md', skillMarkdown);
return zip.generateAsync({ type: 'nodebuffer', compression: 'DEFLATE' });
}
describe('parseFrontmatter', () => {
it('extracts name + description from a minimal frontmatter block', () => {
const raw = `---\nname: demo\ndescription: A demo skill.\n---\n\n# Body`;
@ -101,6 +121,16 @@ describe('parseFrontmatter', () => {
});
});
it('coerces non-string scalar name and description to strings', () => {
const raw = `---\nname: 123\ndescription: 2024\n---\n\n# Body`;
expect(parseFrontmatter(raw)).toEqual({
name: '123',
description: '2024',
alwaysApply: undefined,
invalidBooleans: [],
});
});
it('extracts always-apply: true', () => {
const raw = `---\nname: legal\ndescription: Legal rules.\nalways-apply: true\n---\n\n# Legal body`;
expect(parseFrontmatter(raw)).toEqual({
@ -166,6 +196,20 @@ describe('parseFrontmatter', () => {
expect(result.invalidBooleans).toEqual(['alwaysApply']);
});
it('flags legacy YAML boolean aliases as invalid', () => {
const raw = `---\nname: n\ndescription: d\nalways-apply: on\n---\n\nbody`;
const result = parseFrontmatter(raw);
expect(result.alwaysApply).toBeUndefined();
expect(result.invalidBooleans).toEqual(['always-apply']);
});
it.each(['null', '~'])('flags always-apply: %s as invalid', (value) => {
const raw = `---\nname: n\ndescription: d\nalways-apply: ${value}\n---\n\nbody`;
const result = parseFrontmatter(raw);
expect(result.alwaysApply).toBeUndefined();
expect(result.invalidBooleans).toEqual(['always-apply']);
});
it('does not flag always-apply when the key is absent', () => {
const raw = `---\nname: n\ndescription: d\n---\n\nbody`;
expect(parseFrontmatter(raw).invalidBooleans).toEqual([]);
@ -202,6 +246,26 @@ describe('parseFrontmatter', () => {
});
});
it('extracts frontmatter after a BOM and leading blank lines', () => {
const raw = `\uFEFF\n\n---\nname: prologue\ndescription: Has leading whitespace.\n---\n\nbody`;
expect(parseFrontmatter(raw)).toEqual({
name: 'prologue',
description: 'Has leading whitespace.',
alwaysApply: undefined,
invalidBooleans: [],
});
});
it('does not treat frontmatter scalar lines that start with --- text as closing fences', () => {
const raw = `---\nname: marker\ndescription: 'first\n---not a closing fence\nlast'\nalways-apply: false\n---\n\nbody`;
expect(parseFrontmatter(raw)).toEqual({
name: 'marker',
description: 'first ---not a closing fence last',
alwaysApply: false,
invalidBooleans: [],
});
});
it('returns empty fields when frontmatter is unterminated', () => {
const raw = `---\nname: incomplete\n`;
expect(parseFrontmatter(raw)).toEqual({
@ -211,6 +275,18 @@ describe('parseFrontmatter', () => {
});
});
it('returns empty fields when frontmatter YAML is malformed', () => {
const raw = `---\nname: [\n---\n\nbody`;
expect(parseFrontmatter(raw)).toEqual(
expect.objectContaining({
name: '',
description: '',
invalidBooleans: [],
parseError: expect.any(String),
}),
);
});
it('ignores always-apply appearing outside the frontmatter block', () => {
const raw = `---\nname: n\ndescription: d\n---\n\nalways-apply: true (but this is in the body)`;
const result = parseFrontmatter(raw);
@ -294,4 +370,49 @@ describe('createImportHandler', () => {
expect(summary.filesFailed).toBe(3);
expect(summary.errors).toHaveLength(3);
});
it('rejects malformed YAML frontmatter in markdown imports', async () => {
const deps = mockImportDeps();
const handler = createImportHandler(deps);
const res = mockResponse();
await handler(mockMarkdownRequest('---\nname: [\n---\n\nbody'), res);
expect(res.status).toHaveBeenCalledWith(400);
expect(res.body).toEqual(
expect.objectContaining({
error: 'Validation failed',
issues: expect.arrayContaining([
expect.objectContaining({
field: 'frontmatter',
code: 'INVALID_YAML',
}),
]),
}),
);
expect(deps.createSkill).not.toHaveBeenCalled();
});
it('rejects malformed YAML frontmatter in archive imports', async () => {
const deps = mockImportDeps();
const handler = createImportHandler(deps);
const res = mockResponse();
const buffer = await zipWithSkillMarkdown('---\nname: [\n---\n\nbody');
await handler(mockZipRequest(buffer), res);
expect(res.status).toHaveBeenCalledWith(400);
expect(res.body).toEqual(
expect.objectContaining({
error: 'Validation failed',
issues: expect.arrayContaining([
expect.objectContaining({
field: 'frontmatter',
code: 'INVALID_YAML',
}),
]),
}),
);
expect(deps.createSkill).not.toHaveBeenCalled();
});
});

View file

@ -206,6 +206,7 @@ function serializeSkillFile(file: ISkillFile & { _id: Types.ObjectId }): TSkillF
storageKey: file.storageKey,
storageRegion: file.storageRegion,
source: file.source as TSkillFile['source'],
sourceMetadata: file.sourceMetadata as TSkillFile['sourceMetadata'],
mimeType: file.mimeType,
bytes: file.bytes,
category: file.category,

View file

@ -1,7 +1,7 @@
import path from 'path';
import JSZip from 'jszip';
import crypto from 'crypto';
import { logger, stripYamlTrailingComment } from '@librechat/data-schemas';
import { logger } from '@librechat/data-schemas';
import { ResourceType, AccessRoleIds, PrincipalType } from 'librechat-data-provider';
import type {
ISkill,
@ -12,50 +12,14 @@ import type {
} from '@librechat/data-schemas';
import type { Request, Response } from 'express';
import type { Types } from 'mongoose';
import type { ImportLimits } from './limits';
import { resolveRequestTenantId } from '~/middleware/tenant';
import { DEFAULT_SKILL_IMPORT_LIMITS } from './limits';
import { parseSkillMarkdown } from './parse';
/** Security limits for zip processing. */
const MAX_ZIP_BYTES = 50 * 1024 * 1024; // 50 MB compressed
const MAX_DECOMPRESSED_BYTES = 500 * 1024 * 1024; // 500 MB total decompressed
const MAX_ENTRIES = 500;
const MAX_SINGLE_FILE_BYTES = 10 * 1024 * 1024; // 10 MB per file
const SKILL_MD = 'SKILL.md';
export interface ImportLimits {
maxZipBytes: number;
maxDecompressedBytes: number;
maxEntries: number;
maxSingleFileBytes: number;
}
/** Strip surrounding YAML quotes (single or double) from a scalar value. */
function unquoteYaml(value: string): string {
if (
value.length >= 2 &&
((value[0] === '"' && value[value.length - 1] === '"') ||
(value[0] === "'" && value[value.length - 1] === "'"))
) {
return value.slice(1, -1);
}
return value;
}
/**
* Parse a YAML scalar as a strict boolean. Returns `undefined` when the
* value is neither `true` nor `false`. Callers should pre-strip inline
* comments with `stripYamlTrailingComment` when needed; this helper only
* normalizes case / whitespace so the call site stays one-purpose.
*/
function parseBooleanScalar(value: string): boolean | undefined {
const lowered = value.trim().toLowerCase();
if (lowered === 'true') {
return true;
}
if (lowered === 'false') {
return false;
}
return undefined;
}
export type { ImportLimits } from './limits';
/**
* YAML frontmatter parser — extracts the first-class fields LibreChat
@ -81,71 +45,40 @@ export function parseFrontmatter(raw: string): {
alwaysApply?: boolean;
/** Keys that carried non-boolean values for fields that must be boolean. */
invalidBooleans: string[];
parseError?: string;
} {
const trimmed = raw.trim();
if (!trimmed.startsWith('---')) {
return { name: '', description: '', invalidBooleans: [] };
const parsed = parseSkillMarkdown(raw);
const result: {
name: string;
description: string;
alwaysApply?: boolean;
invalidBooleans: string[];
parseError?: string;
} = {
name: parsed.name,
description: parsed.description,
invalidBooleans: parsed.invalidBooleans,
};
if (parsed.parseError) {
result.parseError = parsed.parseError;
}
const after = trimmed.slice(3);
const closingIdx = after.indexOf('\n---');
if (closingIdx === -1) {
return { name: '', description: '', invalidBooleans: [] };
if ('alwaysApply' in parsed) {
result.alwaysApply = parsed.alwaysApply;
}
const block = after.slice(0, closingIdx);
let name = '';
let description = '';
let alwaysApply: boolean | undefined;
let hasValidCanonicalAlwaysApply = false;
const invalidCanonicalAlwaysApply: string[] = [];
const invalidAliasAlwaysApply: string[] = [];
for (const line of block.split('\n')) {
const colon = line.indexOf(':');
if (colon === -1) {
continue;
}
const rawKey = line.slice(0, colon).trim();
const key = rawKey.toLowerCase();
const rawValue = line.slice(colon + 1).trim();
if (key === 'name') {
name = unquoteYaml(rawValue);
} else if (key === 'description') {
description = unquoteYaml(rawValue);
} else if (key === 'always-apply' || key === 'alwaysapply') {
const isCanonical = key === 'always-apply';
// Operate on the raw post-colon text (no outer `unquoteYaml`): a
// line like `always-apply: "true" # note` must have its comment
// stripped BEFORE unquoting. Running `unquoteYaml` on the whole
// line first would miss the quoted branch (the line doesn't end
// in a quote once the comment is attached), and unquoting twice
// would be fragile if `unquoteYaml` ever gains richer YAML-escape
// handling.
const stripped = stripYamlTrailingComment(rawValue).trim();
if (stripped === '') {
// Empty value or comment-only (`always-apply: # TBD`) — treat as
// absent so mid-edit placeholder states don't reject the save.
continue;
}
const parsed = parseBooleanScalar(unquoteYaml(stripped));
if (parsed === undefined) {
if (isCanonical) {
invalidCanonicalAlwaysApply.push(rawKey);
} else {
invalidAliasAlwaysApply.push(rawKey);
}
} else {
if (isCanonical) {
hasValidCanonicalAlwaysApply = true;
alwaysApply = parsed;
} else if (!hasValidCanonicalAlwaysApply && invalidCanonicalAlwaysApply.length === 0) {
alwaysApply = parsed;
}
}
}
}
const invalidBooleans = hasValidCanonicalAlwaysApply
? invalidCanonicalAlwaysApply
: [...invalidCanonicalAlwaysApply, ...invalidAliasAlwaysApply];
return { name, description, alwaysApply, invalidBooleans };
return result;
}
function sendFrontmatterParseError(res: Response, parseError: string) {
return res.status(400).json({
error: 'Validation failed',
issues: [
{
field: 'frontmatter',
code: 'INVALID_YAML',
message: `Invalid YAML frontmatter: ${parseError}`,
},
],
});
}
/** Validates a relative path is safe (no traversal, no absolute paths). */
@ -267,10 +200,12 @@ export function createImportHandler(deps: ImportSkillDeps) {
function getImportLimits(limits?: Partial<ImportLimits>): ImportLimits {
return {
maxZipBytes: limits?.maxZipBytes ?? MAX_ZIP_BYTES,
maxDecompressedBytes: limits?.maxDecompressedBytes ?? MAX_DECOMPRESSED_BYTES,
maxEntries: limits?.maxEntries ?? MAX_ENTRIES,
maxSingleFileBytes: limits?.maxSingleFileBytes ?? MAX_SINGLE_FILE_BYTES,
maxZipBytes: limits?.maxZipBytes ?? DEFAULT_SKILL_IMPORT_LIMITS.maxZipBytes,
maxDecompressedBytes:
limits?.maxDecompressedBytes ?? DEFAULT_SKILL_IMPORT_LIMITS.maxDecompressedBytes,
maxEntries: limits?.maxEntries ?? DEFAULT_SKILL_IMPORT_LIMITS.maxEntries,
maxSingleFileBytes:
limits?.maxSingleFileBytes ?? DEFAULT_SKILL_IMPORT_LIMITS.maxSingleFileBytes,
};
}
@ -325,7 +260,10 @@ async function handleMarkdown(
) {
const content = file.buffer.toString('utf-8');
const { name, description, alwaysApply, invalidBooleans } = parseFrontmatter(content);
const { name, description, alwaysApply, invalidBooleans, parseError } = parseFrontmatter(content);
if (parseError) {
return sendFrontmatterParseError(res, parseError);
}
if (invalidBooleans.length > 0) {
return res.status(400).json({
error: 'Validation failed',
@ -436,7 +374,11 @@ async function handleZip(
return res.status(400).json({ error: 'SKILL.md exceeds maximum file size' });
}
const { name, description, alwaysApply, invalidBooleans } = parseFrontmatter(skillMdContent);
const { name, description, alwaysApply, invalidBooleans, parseError } =
parseFrontmatter(skillMdContent);
if (parseError) {
return sendFrontmatterParseError(res, parseError);
}
if (invalidBooleans.length > 0) {
return res.status(400).json({
error: 'Validation failed',

View file

@ -1,5 +1,8 @@
export * from './binary';
export * from './handlers';
export * from './import';
export * from './limits';
export * from './parse';
export * from './skillStates';
export * from './deployment';
export * from './sync';

View file

@ -0,0 +1,13 @@
export type ImportLimits = {
maxZipBytes: number;
maxDecompressedBytes: number;
maxEntries: number;
maxSingleFileBytes: number;
};
export const DEFAULT_SKILL_IMPORT_LIMITS: ImportLimits = {
maxZipBytes: 50 * 1024 * 1024,
maxDecompressedBytes: 500 * 1024 * 1024,
maxEntries: 500,
maxSingleFileBytes: 10 * 1024 * 1024,
};

View file

@ -0,0 +1,160 @@
import yaml from 'js-yaml';
export type ParsedSkillMarkdown = {
name: string;
description: string;
alwaysApply?: boolean;
frontmatter?: Record<string, unknown>;
invalidBooleans: string[];
parseError?: string;
};
function isPlainObject(value: unknown): value is Record<string, unknown> {
return typeof value === 'object' && value !== null && !Array.isArray(value);
}
function extractFrontmatterBlock(raw: string): string | null {
const normalized = raw.replace(/^\uFEFF/, '').replace(/\r\n/g, '\n');
const firstContentIndex = normalized.search(/\S/);
if (firstContentIndex === -1) {
return null;
}
const content = normalized.slice(firstContentIndex);
const opening = /^---[ \t]*\n/.exec(content);
if (!opening) {
return null;
}
const body = content.slice(opening[0].length);
const closingFence = /(?:^|\n)---[ \t]*(?:\n|$)/.exec(body);
if (!closingFence) {
return null;
}
return body.slice(0, closingFence.index);
}
function getCaseInsensitive(frontmatter: Record<string, unknown>, key: string): unknown {
const entry = Object.entries(frontmatter).find(([candidate]) => candidate.toLowerCase() === key);
return entry?.[1];
}
function hasCaseInsensitive(frontmatter: Record<string, unknown>, key: string): boolean {
return Object.keys(frontmatter).some((candidate) => candidate.toLowerCase() === key);
}
function getRawFrontmatterValue(block: string, key: string): string | undefined {
const escapedKey = key.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
const pattern = new RegExp(`^\\s*${escapedKey}\\s*:\\s*(.*)$`, 'i');
const line = block.split('\n').find((candidate) => pattern.test(candidate));
const match = line?.match(pattern);
return match?.[1];
}
function stripInlineComment(value: string): string {
let quote: '"' | "'" | null = null;
for (let i = 0; i < value.length; i++) {
const char = value[i];
if ((char === '"' || char === "'") && (!quote || quote === char)) {
quote = quote ? null : char;
continue;
}
if (char === '#' && !quote) {
return value.slice(0, i).trim();
}
}
return value.trim();
}
function normalizeFrontmatterKeys(frontmatter: Record<string, unknown>): Record<string, unknown> {
return Object.entries(frontmatter).reduce<Record<string, unknown>>((acc, [key, value]) => {
const normalizedKey = key.toLowerCase();
acc[normalizedKey === 'alwaysapply' ? 'alwaysApply' : normalizedKey] = value;
return acc;
}, {});
}
function parseBoolean(value: unknown, rawValue?: string): boolean | undefined {
const raw = rawValue === undefined ? undefined : stripInlineComment(rawValue).toLowerCase();
if (typeof value === 'boolean') {
return raw === 'true' || raw === 'false' ? value : undefined;
}
if (typeof value !== 'string') {
return undefined;
}
const lowered = value.trim().toLowerCase();
if (lowered === 'true') {
return true;
}
if (lowered === 'false') {
return false;
}
return undefined;
}
function hasBooleanPlaceholder(rawValue?: string): boolean {
return rawValue !== undefined && stripInlineComment(rawValue).length === 0;
}
function toScalarString(value: unknown): string {
if (typeof value === 'string') {
return value;
}
if (typeof value === 'number' || typeof value === 'boolean') {
return String(value);
}
return '';
}
export function parseSkillMarkdown(raw: string): ParsedSkillMarkdown {
const block = extractFrontmatterBlock(raw);
if (!block) {
return { name: '', description: '', invalidBooleans: [] };
}
let parsed: unknown;
try {
parsed = yaml.load(block);
} catch (error) {
return {
name: '',
description: '',
invalidBooleans: [],
parseError: error instanceof Error ? error.message : 'Invalid YAML frontmatter',
};
}
const frontmatter = isPlainObject(parsed) ? normalizeFrontmatterKeys(parsed) : {};
const nameValue = getCaseInsensitive(frontmatter, 'name');
const descriptionValue = getCaseInsensitive(frontmatter, 'description');
const whenToUseValue = getCaseInsensitive(frontmatter, 'when-to-use');
const hasCanonicalAlwaysApply = hasCaseInsensitive(frontmatter, 'always-apply');
const hasAliasAlwaysApply = hasCaseInsensitive(frontmatter, 'alwaysapply');
const canonicalAlwaysApplyValue = getCaseInsensitive(frontmatter, 'always-apply');
const aliasAlwaysApplyValue = getCaseInsensitive(frontmatter, 'alwaysapply');
const rawCanonicalAlwaysApplyValue = getRawFrontmatterValue(block, 'always-apply');
const rawAliasAlwaysApplyValue = getRawFrontmatterValue(block, 'alwaysApply');
const name = toScalarString(nameValue);
let description = '';
if (descriptionValue !== undefined) {
description = toScalarString(descriptionValue);
} else if (whenToUseValue !== undefined) {
description = toScalarString(whenToUseValue);
}
let alwaysApply: boolean | undefined;
const invalidBooleans: string[] = [];
if (hasCanonicalAlwaysApply) {
alwaysApply = parseBoolean(canonicalAlwaysApplyValue, rawCanonicalAlwaysApplyValue);
if (alwaysApply === undefined && !hasBooleanPlaceholder(rawCanonicalAlwaysApplyValue)) {
invalidBooleans.push('always-apply');
}
} else if (hasAliasAlwaysApply) {
alwaysApply = parseBoolean(aliasAlwaysApplyValue, rawAliasAlwaysApplyValue);
if (alwaysApply === undefined && !hasBooleanPlaceholder(rawAliasAlwaysApplyValue)) {
invalidBooleans.push('alwaysApply');
}
}
return {
name,
description,
alwaysApply,
frontmatter,
invalidBooleans,
};
}

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,3 @@
export * from './github';
export * from './orchestrator';
export * from './scheduler';

View file

@ -0,0 +1,339 @@
import type { SkillSyncConfig } from 'librechat-data-provider';
import type { SkillSyncTriggerRunnerFactoryInput } from './orchestrator';
import type { GitHubSkillSyncRunner } from './github';
import { createSkillSyncTriggerOrchestrator } from './orchestrator';
type RunnerStatus = Awaited<ReturnType<GitHubSkillSyncRunner['getStatus']>>;
type RunnerRunResult = Awaited<ReturnType<GitHubSkillSyncRunner['runOnce']>>;
const source = {
id: 'tenant-skills',
owner: 'LibreChat',
repo: 'skills',
ref: 'main',
paths: ['skills'],
token: '${GITHUB_SKILLS_TOKEN}',
tenantId: 'other-tenant',
};
function skillSync(
overrides: Partial<NonNullable<SkillSyncConfig>['github']> = {},
): SkillSyncConfig {
return {
github: {
enabled: true,
intervalMinutes: 60,
runOnStartup: true,
sources: [source],
...overrides,
},
};
}
function statusFromConfig(
config: SkillSyncConfig | undefined,
{ allowServerCredentials = true }: { allowServerCredentials?: boolean } = {},
): RunnerStatus {
const github = config?.github;
return {
enabled: github?.enabled ?? false,
intervalMinutes: github?.intervalMinutes ?? 60,
runOnStartup: github?.runOnStartup ?? false,
sources:
github?.sources.map((configuredSource) => ({
provider: 'github',
sourceId: configuredSource.id,
tenantId: configuredSource.tenantId,
status: 'idle',
credentialKey: configuredSource.credentialKey,
credentialPresent:
allowServerCredentials &&
Boolean(configuredSource.credentialKey || configuredSource.token),
owner: configuredSource.owner,
repo: configuredSource.repo,
ref: configuredSource.ref,
paths: configuredSource.paths,
syncedSkillCount: 0,
syncedFileCount: 0,
deletedSkillCount: 0,
deletedFileCount: 0,
errorCode: undefined,
errorMessage: undefined,
startedAt: undefined,
finishedAt: undefined,
lastSuccessAt: undefined,
lastFailureAt: undefined,
createdAt: undefined,
updatedAt: undefined,
})) ?? [],
credentials: [],
fineGrainedTokenRecommendation: 'Use a GitHub fine-grained personal access token.',
};
}
function withRunnableCredentials(status: RunnerStatus): RunnerStatus {
return {
...status,
sources: status.sources.map((configuredSource) => ({
...configuredSource,
credentialPresent: true,
})),
};
}
function completedRun(): RunnerRunResult {
return { status: 'completed', sources: [] };
}
function deferred<T>() {
let resolve!: (value: T) => void;
const promise = new Promise<T>((res) => {
resolve = res;
});
return { promise, resolve };
}
async function flushPromises() {
await Promise.resolve();
await Promise.resolve();
}
function createHarness(
options: {
status?: RunnerStatus;
runOnce?: () => Promise<RunnerRunResult>;
} = {},
) {
const runners: Array<{
input: SkillSyncTriggerRunnerFactoryInput;
runner: GitHubSkillSyncRunner;
}> = [];
const logger = {
warn: jest.fn(),
error: jest.fn(),
};
const createRunner = jest.fn((input: SkillSyncTriggerRunnerFactoryInput) => {
const runner: GitHubSkillSyncRunner = {
getStatus: jest.fn(
async () =>
options.status ??
statusFromConfig(await input.getConfig(), {
allowServerCredentials: input.allowServerCredentials !== false,
}),
),
runOnce: jest.fn(options.runOnce ?? (async () => completedRun())),
};
runners.push({ input, runner });
return runner;
});
const orchestrator = createSkillSyncTriggerOrchestrator({
createRunner,
logger,
});
return { createRunner, logger, orchestrator, runners };
}
describe('createSkillSyncTriggerOrchestrator', () => {
it('starts request sync from resolved admin skillSync config and derives tenant from the request', async () => {
const config = skillSync();
const { orchestrator, runners } = createHarness({
status: withRunnableCredentials(statusFromConfig(config)),
});
const started = await orchestrator.maybeRunForRequest({
config: { skillSync: config, config: {} },
user: { tenantId: 'tenant-a' },
});
const requestConfig = await runners[0].input.getConfig();
expect(started).toBe(true);
expect(runners[0].input.allowServerCredentials).toBe(false);
expect(runners[0].runner.runOnce).toHaveBeenCalledTimes(1);
expect(requestConfig?.github?.runOnStartup).toBe(false);
expect(requestConfig?.github?.sources[0]).toEqual(
expect.objectContaining({ id: 'tenant-skills', tenantId: 'tenant-a' }),
);
});
it('does not auto-start request sync when only server credentials are configured', async () => {
const config = skillSync();
const { orchestrator, runners } = createHarness();
const started = await orchestrator.maybeRunForRequest({
config: { skillSync: config, config: {} },
user: { tenantId: 'tenant-a' },
});
expect(started).toBe(false);
expect(runners[0].input.allowServerCredentials).toBe(false);
expect(runners[0].runner.runOnce).not.toHaveBeenCalled();
});
it('starts request sync with server credentials when the caller opts in', async () => {
const config = skillSync();
const { orchestrator, runners } = createHarness();
const started = await orchestrator.maybeRunForRequest({
config: { skillSync: config, config: {} },
user: { tenantId: 'tenant-a' },
skillSyncAllowServerCredentials: true,
});
expect(started).toBe(true);
expect(runners[0].input.allowServerCredentials).toBe(true);
expect(runners[0].runner.runOnce).toHaveBeenCalledTimes(1);
});
it('does not start request sync for base YAML skillSync config', async () => {
const config = skillSync();
const { createRunner, orchestrator } = createHarness();
const started = await orchestrator.maybeRunForRequest({
config: { skillSync: config, config: { skillSync: config } },
user: { tenantId: 'tenant-a' },
});
expect(started).toBe(false);
expect(createRunner).not.toHaveBeenCalled();
});
it('creates an admin request runner from resolved config without disabling startup runs', async () => {
const config = skillSync();
const { orchestrator, runners } = createHarness();
const runner = orchestrator.getRunnerForAdminRequest({
config: { skillSync: config, config: {} },
user: { tenantId: 'tenant-a' },
skillSyncAllowServerCredentials: true,
});
const runnerConfig = await runners[0].input.getConfig();
expect(runner).toBe(runners[0].runner);
expect(runners[0].input.allowServerCredentials).toBe(true);
expect(runnerConfig?.github?.runOnStartup).toBe(true);
expect(runnerConfig?.github?.sources[0]).toEqual(
expect.objectContaining({ id: 'tenant-skills', tenantId: 'tenant-a' }),
);
});
it('preserves configured tenant scope for platform admin override runners', async () => {
const config = skillSync();
const { orchestrator, runners } = createHarness();
orchestrator.getRunnerForAdminRequest({
config: { skillSync: config, config: {} },
user: {},
skillSyncAllowServerCredentials: true,
});
const runnerConfig = await runners[0].input.getConfig();
expect(runnerConfig?.github?.runOnStartup).toBe(true);
expect(runnerConfig?.github?.sources[0]).toEqual(
expect.objectContaining({ id: 'tenant-skills', tenantId: 'other-tenant' }),
);
});
it('preserves configured tenant scope for admin base skillSync runs', async () => {
const config = skillSync();
const { orchestrator, runners } = createHarness();
orchestrator.getRunnerForAdminRequest({
config: { skillSync: config, config: { skillSync: config } },
user: { tenantId: 'tenant-a' },
skillSyncAllowServerCredentials: true,
});
const runnerConfig = await runners[0].input.getConfig();
expect(runnerConfig?.github?.runOnStartup).toBe(true);
expect(runnerConfig?.github?.sources[0]).toEqual(
expect.objectContaining({ id: 'tenant-skills', tenantId: 'other-tenant' }),
);
});
it('does not allow admin override runners to use server credentials by default', async () => {
const config = skillSync();
const { orchestrator, runners } = createHarness();
orchestrator.getRunnerForAdminRequest({
config: { skillSync: config, config: {} },
user: { tenantId: 'tenant-a' },
});
expect(runners[0].input.allowServerCredentials).toBe(false);
});
it('does not start request sync when the configured source is already running', async () => {
const config = skillSync({ runOnStartup: false });
const { orchestrator, runners } = createHarness({
status: {
...statusFromConfig(config),
sources: [
{
...statusFromConfig(config).sources[0],
status: 'running',
startedAt: new Date(),
},
],
},
});
const started = await orchestrator.maybeRunForRequest({
config: { skillSync: config, config: {} },
user: { tenantId: 'tenant-a' },
});
expect(started).toBe(false);
expect(runners[0].runner.runOnce).not.toHaveBeenCalled();
});
it('retries request sync when a running source status is stale', async () => {
const config = skillSync({ runOnStartup: false });
const { orchestrator, runners } = createHarness({
status: {
...statusFromConfig(config),
sources: [
{
...statusFromConfig(config).sources[0],
status: 'running',
credentialPresent: true,
startedAt: new Date(Date.now() - 40 * 60 * 1000),
},
],
},
});
const started = await orchestrator.maybeRunForRequest({
config: { skillSync: config, config: {} },
user: { tenantId: 'tenant-a' },
});
expect(started).toBe(true);
expect(runners[0].runner.runOnce).toHaveBeenCalledTimes(1);
});
it('suppresses duplicate request sync while an equivalent run is in flight', async () => {
const pendingRun = deferred<RunnerRunResult>();
const config = skillSync({ runOnStartup: false });
const { orchestrator, runners } = createHarness({
status: withRunnableCredentials(statusFromConfig(config)),
runOnce: () => pendingRun.promise,
});
const first = await orchestrator.maybeRunForRequest({
config: { skillSync: config, config: {} },
user: { tenantId: 'tenant-a' },
});
const second = await orchestrator.maybeRunForRequest({
config: { skillSync: config, config: {} },
user: { tenantId: 'tenant-a' },
});
pendingRun.resolve(completedRun());
await flushPromises();
expect(first).toBe(true);
expect(second).toBe(false);
expect(runners).toHaveLength(1);
expect(runners[0].runner.runOnce).toHaveBeenCalledTimes(1);
});
});

View file

@ -0,0 +1,256 @@
import { skillSyncConfigSchema } from 'librechat-data-provider';
import type { SkillSyncConfig } from 'librechat-data-provider';
import type { GitHubSkillSyncRunner } from './github';
const REQUEST_SYNC_MIN_INTERVAL_MS = 5 * 60 * 1000;
const REQUEST_SYNC_STALE_RUNNING_MS = 35 * 60 * 1000;
type MaybePromise<T> = T | Promise<T>;
export type SkillSyncAppConfigLike = {
skillSync?: unknown;
config?: {
skillSync?: unknown;
};
};
export type SkillSyncRequestUser = {
tenantId?: string | null;
};
export type SkillSyncRequestLike = {
config?: SkillSyncAppConfigLike;
user?: SkillSyncRequestUser;
skillSyncAllowServerCredentials?: boolean;
};
type ResolvedSkillSyncConfig = NonNullable<SkillSyncConfig>;
type ResolvedGitHubSkillSyncConfig = NonNullable<ResolvedSkillSyncConfig['github']>;
type SkillSyncConfigWithGitHub = ResolvedSkillSyncConfig & {
github: ResolvedGitHubSkillSyncConfig;
};
type SkillSyncRunnerStatus = Awaited<ReturnType<GitHubSkillSyncRunner['getStatus']>>;
type SkillSyncTriggerLogger = {
warn: (message: string, metadata?: object) => void;
error: (message: string, error?: unknown) => void;
};
export type SkillSyncTriggerRunnerFactoryInput = {
getConfig: () => MaybePromise<SkillSyncConfig | undefined>;
loadAppConfig: () => MaybePromise<SkillSyncAppConfigLike | undefined>;
allowServerCredentials?: boolean;
};
export type SkillSyncTriggerOrchestratorDeps = {
createRunner: (input: SkillSyncTriggerRunnerFactoryInput) => GitHubSkillSyncRunner;
logger: SkillSyncTriggerLogger;
minIntervalMs?: number;
staleRunningMs?: number;
inFlight?: Set<string>;
};
export type SkillSyncTriggerOrchestrator = {
getRunnerForAdminRequest: (request: SkillSyncRequestLike) => GitHubSkillSyncRunner;
maybeRunForRequest: (request: SkillSyncRequestLike) => Promise<boolean>;
};
function parseSkillSyncConfig(
raw: unknown,
logger: SkillSyncTriggerLogger,
): SkillSyncConfig | undefined {
if (!raw || typeof raw !== 'object') {
return undefined;
}
const parsed = skillSyncConfigSchema.safeParse(raw);
if (!parsed.success) {
logger.warn('[GitHubSkillSync] Ignoring invalid skill sync config', {
issues: parsed.error.flatten(),
});
return undefined;
}
return parsed.data;
}
function hasGitHubConfig(config: SkillSyncConfig | undefined): config is SkillSyncConfigWithGitHub {
return Boolean(config?.github);
}
function isSameSkillSyncConfig(
left: SkillSyncConfig | undefined,
right: SkillSyncConfig | undefined,
) {
return JSON.stringify(left ?? null) === JSON.stringify(right ?? null);
}
function getRequestTenantId(user: SkillSyncRequestUser | undefined): string | undefined {
return typeof user?.tenantId === 'string' && user.tenantId ? user.tenantId : undefined;
}
function withRequestTenant(
config: SkillSyncConfigWithGitHub,
user: SkillSyncRequestUser | undefined,
{ disableRunOnStartup = false } = {},
): SkillSyncConfig {
const tenantId = getRequestTenantId(user);
return {
...config,
github: {
...config.github,
...(disableRunOnStartup ? { runOnStartup: false } : {}),
sources: config.github.sources.map((source) => ({
...source,
// Tenant-scoped requests derive their tenant from the request; platform
// requests have no tenant and preserve an explicitly configured source.
tenantId: tenantId ?? source.tenantId,
})),
},
};
}
function getRequestSkillSyncConfig(
appConfig: SkillSyncAppConfigLike | undefined,
user: SkillSyncRequestUser | undefined,
logger: SkillSyncTriggerLogger,
): SkillSyncConfig | undefined {
const resolved = parseSkillSyncConfig(appConfig?.skillSync, logger);
if (
!hasGitHubConfig(resolved) ||
!resolved.github.enabled ||
resolved.github.sources.length === 0
) {
return undefined;
}
const base = parseSkillSyncConfig(appConfig?.config?.skillSync, logger);
if (isSameSkillSyncConfig(resolved, base)) {
return undefined;
}
return withRequestTenant(resolved, user, { disableRunOnStartup: true });
}
function getAdminRequestSkillSyncConfig(
appConfig: SkillSyncAppConfigLike | undefined,
user: SkillSyncRequestUser | undefined,
logger: SkillSyncTriggerLogger,
): SkillSyncConfig | undefined {
const resolved = parseSkillSyncConfig(appConfig?.skillSync, logger);
if (!hasGitHubConfig(resolved)) {
return resolved;
}
const base = parseSkillSyncConfig(appConfig?.config?.skillSync, logger);
if (isSameSkillSyncConfig(resolved, base)) {
return resolved;
}
return withRequestTenant(resolved, user);
}
function toTimestamp(value: unknown): number {
if (!value) {
return 0;
}
if (value instanceof Date) {
return value.getTime();
}
const parsed = Date.parse(String(value));
return Number.isNaN(parsed) ? 0 : parsed;
}
function getLastAttemptAt(source: SkillSyncRunnerStatus['sources'][number]): number {
return Math.max(
toTimestamp(source.finishedAt),
toTimestamp(source.startedAt),
toTimestamp(source.updatedAt),
toTimestamp(source.lastSuccessAt),
toTimestamp(source.lastFailureAt),
);
}
function shouldRunRequestSync(
status: SkillSyncRunnerStatus,
{ minIntervalMs, staleRunningMs }: { minIntervalMs: number; staleRunningMs: number },
): boolean {
if (!status.enabled || status.sources.length === 0) {
return false;
}
const intervalMs = Math.max(minIntervalMs, (status.intervalMinutes ?? 60) * 60 * 1000);
const now = Date.now();
return status.sources.some((source) => {
if (!source.credentialPresent) {
return false;
}
if (source.status === 'running') {
const startedAt = toTimestamp(source.startedAt);
return Boolean(startedAt && now - startedAt >= staleRunningMs);
}
const lastAttemptAt = getLastAttemptAt(source);
return !lastAttemptAt || now - lastAttemptAt >= intervalMs;
});
}
function getRequestSyncKey(
config: SkillSyncConfigWithGitHub,
user: SkillSyncRequestUser | undefined,
) {
const tenantId = user?.tenantId ?? '';
const sources = config.github.sources
.map((source) => source.id)
.sort()
.join(',');
return `${tenantId}:${sources}`;
}
export function createSkillSyncTriggerOrchestrator(
deps: SkillSyncTriggerOrchestratorDeps,
): SkillSyncTriggerOrchestrator {
const inFlight = deps.inFlight ?? new Set<string>();
const minIntervalMs = deps.minIntervalMs ?? REQUEST_SYNC_MIN_INTERVAL_MS;
const staleRunningMs = deps.staleRunningMs ?? REQUEST_SYNC_STALE_RUNNING_MS;
function getRunnerForAdminRequest(request: SkillSyncRequestLike): GitHubSkillSyncRunner {
const config = getAdminRequestSkillSyncConfig(request.config, request.user, deps.logger);
return deps.createRunner({
getConfig: async () => config,
loadAppConfig: async () => request.config,
allowServerCredentials: Boolean(request.skillSyncAllowServerCredentials),
});
}
async function maybeRunForRequest(request: SkillSyncRequestLike): Promise<boolean> {
const config = getRequestSkillSyncConfig(request.config, request.user, deps.logger);
if (!hasGitHubConfig(config)) {
return false;
}
const syncKey = getRequestSyncKey(config, request.user);
if (inFlight.has(syncKey)) {
return false;
}
const requestRunner = deps.createRunner({
getConfig: async () => config,
loadAppConfig: async () => request.config,
allowServerCredentials: Boolean(request.skillSyncAllowServerCredentials),
});
const status = await requestRunner.getStatus();
if (!shouldRunRequestSync(status, { minIntervalMs, staleRunningMs })) {
return false;
}
inFlight.add(syncKey);
void requestRunner
.runOnce()
.catch((error) => deps.logger.error('[GitHubSkillSync] Request-scoped sync failed:', error))
.finally(() => inFlight.delete(syncKey));
return true;
}
return {
getRunnerForAdminRequest,
maybeRunForRequest,
};
}

View file

@ -0,0 +1,86 @@
import type { SkillSyncConfig } from 'librechat-data-provider';
import type { GitHubSkillSyncRunner } from './github';
import { SKILL_SYNC_MAX_TIMER_INTERVAL_MINUTES, startGitHubSkillSyncScheduler } from './scheduler';
import { __resetShutdownStateForTests } from '~/app/shutdown';
const source = {
id: 'librechat-skills',
owner: 'LibreChat',
repo: 'skills',
ref: 'main',
paths: ['skills'],
credentialKey: 'github-skills-prod',
};
function config(intervalMinutes: number, enabled = true): SkillSyncConfig {
return {
github: {
enabled,
intervalMinutes,
runOnStartup: false,
sources: [source],
},
};
}
function runner(): GitHubSkillSyncRunner {
return {
getStatus: jest.fn(),
runOnce: jest.fn(async () => ({ status: 'completed', sources: [] })),
} as unknown as GitHubSkillSyncRunner;
}
async function flushPromises() {
await Promise.resolve();
await Promise.resolve();
await Promise.resolve();
await Promise.resolve();
}
describe('startGitHubSkillSyncScheduler', () => {
beforeEach(() => {
jest.useFakeTimers();
__resetShutdownStateForTests();
});
afterEach(() => {
jest.useRealTimers();
jest.restoreAllMocks();
__resetShutdownStateForTests();
});
it('clamps oversized intervals before scheduling a timer', async () => {
const setTimeoutSpy = jest.spyOn(global, 'setTimeout');
const scheduler = startGitHubSkillSyncScheduler({
getConfig: () => config(Number.MAX_SAFE_INTEGER),
runner: runner(),
});
await flushPromises();
expect(setTimeoutSpy).toHaveBeenLastCalledWith(
expect.any(Function),
SKILL_SYNC_MAX_TIMER_INTERVAL_MINUTES * 60 * 1000,
);
scheduler.stop();
});
it('uses fresh config when scheduling the next run', async () => {
let intervalMinutes = 5;
const skillRunner = runner();
const setTimeoutSpy = jest.spyOn(global, 'setTimeout');
const scheduler = startGitHubSkillSyncScheduler({
getConfig: () => config(intervalMinutes),
runner: skillRunner,
});
await flushPromises();
intervalMinutes = 10;
await jest.advanceTimersByTimeAsync(5 * 60 * 1000);
await flushPromises();
expect(skillRunner.runOnce).toHaveBeenCalledTimes(1);
expect(setTimeoutSpy).toHaveBeenLastCalledWith(expect.any(Function), 10 * 60 * 1000);
scheduler.stop();
});
});

View file

@ -0,0 +1,103 @@
import { logger } from '@librechat/data-schemas';
import type { SkillSyncConfig } from 'librechat-data-provider';
import type { GitHubSkillSyncRunner } from './github';
import { registerShutdownTask } from '~/app/shutdown';
const NODE_TIMER_MAX_MS = 2147483647;
const SKILL_SYNC_MIN_INTERVAL_MINUTES = 5;
export const SKILL_SYNC_MAX_TIMER_INTERVAL_MINUTES: number = Math.floor(NODE_TIMER_MAX_MS / 60_000);
type MaybePromise<T> = T | Promise<T>;
export type GitHubSkillSyncScheduler = {
stop: () => void;
};
function normalizeIntervalMinutes(value: unknown): number {
if (typeof value !== 'number' || !Number.isFinite(value)) {
return 60;
}
return Math.min(
SKILL_SYNC_MAX_TIMER_INTERVAL_MINUTES,
Math.max(SKILL_SYNC_MIN_INTERVAL_MINUTES, Math.floor(value)),
);
}
function getSources(config: SkillSyncConfig | undefined) {
const sources = config?.github?.sources;
return Array.isArray(sources) ? sources : [];
}
export function startGitHubSkillSyncScheduler(params: {
getConfig: () => MaybePromise<SkillSyncConfig | undefined>;
runner: GitHubSkillSyncRunner;
}): GitHubSkillSyncScheduler {
let stopped = false;
let timer: NodeJS.Timeout | undefined;
const getConfig = async (): Promise<SkillSyncConfig | undefined> => {
try {
return await params.getConfig();
} catch (error) {
logger.error('[GitHubSkillSync] Failed to load scheduler config:', error);
return undefined;
}
};
const scheduleNext = (config: SkillSyncConfig | undefined) => {
if (stopped) {
return;
}
const delayMs = normalizeIntervalMinutes(config?.github?.intervalMinutes) * 60 * 1000;
timer = setTimeout(tick, delayMs);
timer.unref?.();
};
const runIfEnabled = async () => {
const config = await getConfig();
const github = config?.github;
if (!github?.enabled || getSources(config).length === 0) {
return config;
}
try {
await params.runner.runOnce();
} catch (error) {
logger.error('[GitHubSkillSync] Scheduled run failed:', error);
}
return getConfig();
};
async function tick() {
if (stopped) {
return;
}
const config = await runIfEnabled();
scheduleNext(config);
}
const scheduler = {
stop: () => {
stopped = true;
if (timer) {
clearTimeout(timer);
timer = undefined;
}
},
};
void (async () => {
const config = await getConfig();
if (config?.github?.enabled && config.github.runOnStartup && getSources(config).length > 0) {
void params.runner.runOnce().catch((error) => {
logger.error('[GitHubSkillSync] Scheduled startup run failed:', error);
});
}
scheduleNext(config);
})();
registerShutdownTask('github skill sync scheduler', () => {
scheduler.stop();
});
return scheduler;
}